Publications

Adaptive Policy Regularization for Offline-to-Online Reinforcement Learning in HVAC Control

Published in NeurIPS CCAI & ACM BuildSys'24, 2024

This paper proposes an adaptive policy regularization approach for transferring policies from offline datasets to online deployment in HVAC control. The method uses weighted increased simple moving average Q-value estimators to stabilize policy updates and improve safety during online fine-tuning.

Recommended citation: Liu, Hsin-Yu. Adaptive Policy Regularization for Offline-to-Online Reinforcement Learning in HVAC Control. NeurIPS CCAI & ACM BuildSys'24. Nov. 2024. https://dl.acm.org/doi/pdf/10.1145/3671127.3698163

Policy Regularization in Model-Free Building Control via Comprehensive Approaches from Offline to Online Reinforcement Learning

Published in Ph.D. Dissertation, University of California San Diego, 2024

This dissertation develops a novel policy regularization framework for reinforcement learning in HVAC control systems, focusing on safe and efficient operation in real-world settings. Contributions include methods for offline-to-online policy regularization, open-source building batch RL datasets for benchmarking, and empirical results demonstrating energy and performance improvements in building control tasks.

Recommended citation: Liu, Hsin-Yu. Policy Regularization in Model-Free Building Control via Comprehensive Approaches from Offline to Online Reinforcement Learning. Ph.D. Dissertation, University of California San Diego, Jun. 2024. https://escholarship.org/content/qt0b23889v/qt0b23889v_noSplash_60b699fcef9c1cfc9988c4c5c4249a14.pdf

Rule-based Policy Regularization for Reinforcement Learning-based Building Control

Published in The 14th ACM International Conference on Future Energy Systems (ACM e-Energy 2023), 2023

Download here

B2RL: An open-source Dataset for Building Batch Reinforcement Learning

Published in RLEM Workshop (BuildSys), 2022

Link to conference presentation video

Recommended citation: Liu, Hsin-Yu, Xiaohan Fu, Bharathan Balaji, Rajesh Gupta, and Dezhi Hong. "B2RL: an open-source dataset for building batch reinforcement learning." In Proceedings of the 9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pp. 462-465. 2022. https://arxiv.org/abs/2209.15626

Safe HVAC Control via Batch Reinforcement Learning

Published in International Conference on Cyber-Physical Systems (ICCPS), 2022

Recommended citation: Liu, H. Y., Balaji, B., Gao, S., Gupta, R., & Hong, D. (2022, May). Safe HVAC Control via Batch Reinforcement Learning. In 2022 ACM/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS) (pp. 181-192). IEEE. https://conferences.computer.org/cpsiot/pdfs/ICCPS2022-ifhdJu28kaMK8qGYbf7d0/096700a181/096700a181.pdf

Offline Reinforcement Learning with Munchausen Regularization

Published in NeurIPS 2021 Offline RL Workshop, 2021

Recommended citation: Liu, Hsin-Yu, Bharathan Balaji, Rajesh Gupta, and Dezhi Hong. " Offline Reinforcement Learning with Munchausen Regularization " Offline Reinforcement Learning Workshop at Neural Information Processing Systems, 2021 (NeurIPS 2021). http://mesl.ucsd.edu/pubs/Hsinyu_NeurIPS2021_OfflineRL.pdf

SVM Learning for GFIS Trimer Health Monitoring in Helium-Neon Ion Beam Microscopy

Published in Advanced Process Control Conference (APC), 2019

Recommended citation: Kahng, A. B., Liu, H. Y., Park, C., Pichumani, R., & Saul, L. SVM Learning for GFIS Trimer Health Monitoring in Helium-Neon Ion Beam Microscopy., Advanced Process Control Conference, 2019 https://vlsicad.ucsd.edu/Publications/Conferences/372/c372.pdf

METRICS 2.0: A machine-learning based optimization system for IC design

Published in Workshop on Open-Source EDA Technology, 2018

Recommended citation: Hashemi, S., Ho, C. T., Kahng, A. B., Liu, H. Y., & Reda, S. (2018). METRICS 2.0: A machine-learning based optimization system for IC design. In Workshop on Open-Source EDA Technology (p. 21). https://woset-workshop.github.io/PDFs/2018/a21.pdf