Hsin-Yu Liu’s Website
About me
- Currently working at Articul8 AI as an Applied AI Researcher.
- Training Domain-Specific Models (DSMs) using SLM to outperform state-of-the-art close-sourced models.
- Post-training alignments with Reinforcement Learning (GRPO, DPO, RLHF) and Rejection Sampling
- Automatic User-Feedback Fine-Tuning system developements
I am an accomplished AI/ML researcher specializing in Deep Reinforcement Learning, Deep Learning, and Machine Learning. I have a proven track record of leveraging cutting-edge AI/ML technologies to tackle complex technical and business challenges, delivering innovative and impactful solutions.
I hold a Ph.D. in Computer Engineering from the University of California, San Diego (UCSD), where my research focused on Reinforcement Learning algorithms development and its applications.
- My Ph.D. PI is Professor Rajesh K. Gupta, and our lab is Microelectronic Embedded Systems Laboratory (MESL).
Research focus
- Domain Specific Models/Agents Training/Evaluations, Synthetic Datasets Generation/Curation Our Domain Specific Models outperform state-of-the-art proprietary models
- Reinforcement Learning: Online/offline policy regularization, offline-to-online RL, transfer-RL
Animation: Agarwal, R., Schuurmans, D. & Norouzi, M.. (2020). An Optimistic Perspective on Offline Reinforcement Learning International Conference on Machine Learning (ICML).
Publications
- Policy Regularization in Model-Free Building Control via Comprehensive Approaches from Offline to Online Reinforcement Learning
Ph.D. Dissertation, Jun. 2024- Developed a novel policy regularization framework for reinforcement learning in HVAC control systems, ensuring safe and efficient operation in real-world settings.
- Released the first open-source building batch reinforcement learning dataset, enabling benchmarking and advancing research in energy-efficient building management.
- Adaptive Policy Regularization for Offline-to-Online Reinforcement Learning in HVAC Control
NeurIPS CCAI & ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys)’ Nov. 2024- Automatic policy regularization fine-tuning from offline to online via average Q-value estimators
- 40.3% performance improvement from the state-of-the-art methods
- Rule-based Policy Regularization for Reinforcement Learning-based Building Control
ACM International Conference on Future Energy Systems (e-Energy). Jun. 2023- Adaptively incorporates existing policy and RL policies with higher estimated values in policy learning, applicable for both online and offline settings
- Larger than 40% of average episode reward increases for both Online and Offline approaches
- B2RL-An open-source dataset for building batch reinforcement learning
ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys’) - RL Energy Management. Nov. 2022- First Released the first open-source Building Batch RL dataset for benchmarking purposes
- Safe HVAC Control via Batch Reinforcement Learning
International Conference on Cyber-Physical Systems. (ICCPS). May. 2022- Pioneered the development and deployment of Batch RL in real-world building environments
- Incorporate KL-divergence for penalizing large policy update with 16.7% energy reduction
- Offline Reinforcement Learning with Munchausen Regularization
NeurIPS Offline RL Workshop. Dec. 2021- Developed RL policy regularization techniques to penalize large policy updates via KL Divergence
- METRICS 2.0: A machine-learning based optimization system for IC design
Workshop on Open-Source EDA Technology (WOFSET) 2018- Proposed new EDA metrics for EDA-ML studies, marking the first integration of such metrics
- SVM Learning for GFIS Trimer Health Monitoring in Helium-Neon Ion Beam Microscopy
Advanced Process Conference (APC), 2019- Developed SVM for image classification for automated trimer monitoring with >95% precisions
Personal Life
I enjoy:
- Heavy Metal Music,
- Baseball (Go! Padres!)
- Hiking
- Exploring nature.
