Published in Workshop on Open-Source EDA Technology, 2018
Recommended citation: Hashemi, S., Ho, C. T., Kahng, A. B., Liu, H. Y., & Reda, S. (2018). METRICS 2.0: A machine-learning based optimization system for IC design. In Workshop on Open-Source EDA Technology (p. 21). https://woset-workshop.github.io/PDFs/2018/a21.pdf
Published in Advanced Process Control Conference (APC), 2019
Recommended citation: Kahng, A. B., Liu, H. Y., Park, C., Pichumani, R., & Saul, L. SVM Learning for GFIS Trimer Health Monitoring in Helium-Neon Ion Beam Microscopy., Advanced Process Control Conference, 2019 https://vlsicad.ucsd.edu/Publications/Conferences/372/c372.pdf
Published in NeurIPS 2021 Offline RL Workshop, 2021
Recommended citation: Liu, Hsin-Yu, Bharathan Balaji, Rajesh Gupta, and Dezhi Hong. " Offline Reinforcement Learning with Munchausen Regularization " Offline Reinforcement Learning Workshop at Neural Information Processing Systems, 2021 (NeurIPS 2021). http://mesl.ucsd.edu/pubs/Hsinyu_NeurIPS2021_OfflineRL.pdf
Published in International Conference on Cyber-Physical Systems (ICCPS), 2022
Recommended citation: Liu, H. Y., Balaji, B., Gao, S., Gupta, R., & Hong, D. (2022, May). Safe HVAC Control via Batch Reinforcement Learning. In 2022 ACM/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS) (pp. 181-192). IEEE. https://conferences.computer.org/cpsiot/pdfs/ICCPS2022-ifhdJu28kaMK8qGYbf7d0/096700a181/096700a181.pdf
Published in RLEM Workshop (BuildSys), 2022
Link to conference presentation video
Recommended citation: Liu, Hsin-Yu, Xiaohan Fu, Bharathan Balaji, Rajesh Gupta, and Dezhi Hong. "B2RL: an open-source dataset for building batch reinforcement learning." In Proceedings of the 9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pp. 462-465. 2022. https://arxiv.org/abs/2209.15626
Published in The 14th ACM International Conference on Future Energy Systems (ACM e-Energy 2023), 2023
Download here
Published in Ph.D. Dissertation, University of California San Diego, 2024
This dissertation develops a novel policy regularization framework for reinforcement learning in HVAC control systems, focusing on safe and efficient operation in real-world settings. Contributions include methods for offline-to-online policy regularization, open-source building batch RL datasets for benchmarking, and empirical results demonstrating energy and performance improvements in building control tasks.
Recommended citation: Liu, Hsin-Yu. Policy Regularization in Model-Free Building Control via Comprehensive Approaches from Offline to Online Reinforcement Learning. Ph.D. Dissertation, University of California San Diego, Jun. 2024. https://escholarship.org/content/qt0b23889v/qt0b23889v_noSplash_60b699fcef9c1cfc9988c4c5c4249a14.pdf
Published in NeurIPS CCAI & ACM BuildSys'24, 2024
This paper proposes an adaptive policy regularization approach for transferring policies from offline datasets to online deployment in HVAC control. The method uses weighted increased simple moving average Q-value estimators to stabilize policy updates and improve safety during online fine-tuning.
Recommended citation: Liu, Hsin-Yu. Adaptive Policy Regularization for Offline-to-Online Reinforcement Learning in HVAC Control. NeurIPS CCAI & ACM BuildSys'24. Nov. 2024. https://dl.acm.org/doi/pdf/10.1145/3671127.3698163
Published:
I was glad to be invited as one of the panelists to discuss how AI/ML integrates with building control, presenting reinforcement learning research conducted at UC San Diego and practical implications for HVAC and building systems.