Control del confort térmico mediante aprendizaje por refuerzo en edificios
Enviado: 21-06-2024
|Aceptado: 25-12-2024
|Publicado: 09-01-2025
Derechos de autor 2025 M. Castilla, Carmen Campoy-Iniesta, Jose Domingo Alvarez
Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial-CompartirIgual 4.0.
Descargas
Palabras clave:
Aprendizaje por refuerzo, Control basado en el conocimiento, Aprendizaje Automático, Confort térmico, Automatización en edificios
Agencias de apoyo:
MCI-N/AEI/10.13039/501100011033
Unión Europea NextGenerationEU
Junta de Andalucía
Resumen:
El confort t ́ermico se puede definir como la sensaci ́on que garantiza la satisfacci ́on de una persona con el ambiente t ́ermicoque le rodea. Por tanto, garantizar esa sensaci ́on de bienestar de forma eficiente es un factor clave desde el punto de vista delahorro de energ ́ıa, ya que, minimiza los costes y el impacto ambiental derivado de asegurar un ambiente confortable. En estetrabajo, se propone un controlador que utiliza aprendizaje por refuerzo para mantener el confort t ́ermico de los usuarios del centrode investigaci ́on CIESOL. Para ello, se ha hecho uso de un modelo lineal simplificado de la temperatura del aire interior de unahabitaci ́on que ha sido validado con datos reales del edificio. Adem ́as, se han entrenado dos agentes diferentes: un agente deGradiente de Pol ́ıtica Determinista Profunda (DDPG) y un agente de Gradiente de Pol ́ıtica Determinista Profunda de Doble Retardo(TD3). Los resultados obtenidos en simulaci ́on muestran c ́omo el controlador propuesto es capaz de mantener la temperatura interioren la referencia establecida, incluso ante la presencia de perturbaciones. Finalmente, el desempe ̃no del controlador propuesto se hacomparado con un cl ́asico controlador Proporcional-Integral-Derivativo (PID).
Citas:
ASHRAE, 2009. ASHRAE Handbook - Fundamentals. Refrigerating American Society of Heating and Air-Conditioning Engineers.
ASHRAE55, 1992. Thermal environment conditions for human occupancy. American Society of Heating Ventilating and Air-conditioning Engineers.
Bretones, M. D., Álvarez, J. D., del Mar Castilla, M., Berenguel, M., 2020. A fuzzy controller for thermal comfort and indoor air quality in a bioclimatic building. In: 2020 European Control Conference (ECC). IEEE, pp. 1029- 1036. https://doi.org/10.23919/ECC51009.2020.9143775
Bueno, A. M., de Paula Xavier, A. A., Broday, E. E., 2021. Evaluating the connection between thermal comfort and productivity in buildings: A systematic literature review. Buildings 11 (6), 244. https://doi.org/10.3390/buildings11060244
Castilla, M.,Álvarez, J., Normey-Rico, J., Rodríguez, F., 2014a. Thermal comfort control using a non-linear mpc strategy: A real case of study in a bioclimatic building. Journal of Process Control 24 (6), 703-713. https://doi.org/10.1016/j.jprocont.2013.08.009
Castilla, M., Álvarez, J. D., Rodríguez, F., Berenguel, M., 2014b. Comfort control in buildings. Springer. https://doi.org/10.1007/978-1-4471-6347-3
Coraci, D., Brandi, S., Hong, T., Capozzoli, A., 2023. Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings. Applied Energy 333, 120598. https://doi.org/10.1016/j.apenergy.2022.120598
Espejel-Blanco, D. F., Hoyo-Montaño, J. A., Arau, J., Valencia-Palomo, G., García-Barrientos, A., Hernández-De-León, H. R., Camas-Anzueto, J. L., 2022. HVAC control system using predicted mean vote index for energy savings in buildings. Buildings 12 (1), 38. https://doi.org/10.3390/buildings12010038
Fanger, P. O., 1972. Thermal comfort analysis and applications in environment engeering. McGraw Hill.
Fanger, P. O., 1973. Assessment of man's thermal comfort in practice. British Journal of Industrial Medicine 30 (4), 313-324. https://doi.org/10.1136/oem.30.4.313
Fu, Q., Li, Z., Ding, Z., Chen, J., Luo, J., Wang, Y., Lu, Y., 2023. ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings. Building and Environment 242, 110546. https://doi.org/10.1016/j.buildenv.2023.110546
Gao, C., Wang, D., 2023. Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems. Journal of Building Engineering 74, 106852. https://doi.org/10.1016/j.jobe.2023.106852
Gao, G., Li, J., Wen, Y., 2020. Deepcomfort: Energy-efficient thermal comfort control in buildings via reinforcement learning. IEEE Internet of Things Journal 7 (9), 8472-8484. https://doi.org/10.1109/JIOT.2020.2992117
Ghaffarianhoseini, A., AlWaer, H., Omrany, H., Ghaffarianhoseini, A., Alalouch, C., Clements-Croome, D., Tookey, J., 2018. Sick building syndrome: are we doing enough? Architectural Science Review 61 (3), 99-121. https://doi.org/10.1080/00038628.2018.1461060
Halhoul Merabet, G., Essaaidi, M., Ben Haddou, M., Qolomany, B., Qadir, J., Anan, M., Al-Fuqaha, A., Abid, M. R., Benhaddou, D., 2021. Intelligent building control systems for thermal comfort and energy-efficiency: A systematic review of artificial intelligence-assisted techniques. Renewable and Sustainable Energy Reviews 144, 110969. https://doi.org/10.1016/j.rser.2021.110969
ISO7730, 1994. Moderate thermal environments. Determination of the PMV and PPD indices and specification of the conditions for thermal comfort. International Organisation for Standardisation.
Li, H., Xu, J., Zhao, Q., Wang, S., 2024. Economic model predictive control in buildings based on piecewise linear approximation of predicted mean vote index. IEEE Transactions on Automation Science and Engineering 21 (3), 3384-3395. https://doi.org/10.1109/TASE.2023.3279278
Li,W., Zhang, J., Zhao, T., Ren, J., 2021. Experimental study of an indoor temperature fuzzy control method for thermal comfort and energy saving using wristband device. Building and Environment 187, 107432. https://doi.org/10.1016/j.buildenv.2020.107432
Liang, Y., Shuqi, Q., Meng, Z., Chao, S., Tao, J., Xiaohong, G., 2021. A review of deep reinforcement learning for smart building energy management. IEEE Internet of Things Journal 8, 12046-12063. https://doi.org/10.1109/JIOT.2021.3078462
MathWorks, 2024. Reinforcement Learning Toolbox - Agents. Disponible online en: https://es.mathworks.com/help/reinforcement-learning/agents.html, (Ultimo acceso: 29 de mayo 2024).
Russell, S., Norvig, P., 2020. Artificial Intelligence: a modern approach. Fourth edition. NJ: Pearson.
Shao, T., 2021. Indoor environment intelligent control system of green building based on PMV index. Advances in Civil Engineering 2021 (1), 6619401. https://doi.org/10.1155/2021/6619401
Sutton, R., Barto, A., 2018. Reinforcement Learning: an introduction. Second edition. The MIT Press, Cambridge.
Taheri, S., Amiri, A. J., Razban, A., 2024. Real-world implementation of a cloud-based MPC for HVAC control in educational buildings. Energy Conversion and Management 305, 118270. https://doi.org/10.1016/j.enconman.2024.118270
Wang, J., Jiang, Y., Tang, C. Y., Song, L., 2023. Analysis of predicted mean vote-based model predictive control for residential HVAC systems. Building and Environment 229, 109952. https://doi.org/10.1016/j.buildenv.2022.109952
Yao, Y., Shekhar, D. K., 2021. State of the art review on model predictive control (MPC) in heating ventilation and air-conditioning (HVAC) field. Building and Environment 200, 107952. https://doi.org/10.1016/j.buildenv.2021.107952
Yue, B., Su, B., Xiao, F., Li, A., Li, K., Li, S., Yan, R., Lian, Q., Li, A., Li, Y., Fang, X., Liang, X., 2023. Energy-oriented control retrofit for existing HVAC system adopting data-driven MPC - Methodology, implementation and field test. Energy and Buildings 295, 113286. DOI: https://doi.org/10.1016/j.enbuild.2023.113286
