Reinforcement Learning Thermostat Controller Comparison

Validation & Documentation · Python · NumPy · Matplotlib · PyTorch · Reinforcement Learning

Temperature trajectories for Q-learning, SARSA, DQN, rule-based, and proportional controllers during one shared 200-step simulated episode, with a 25 degree Celsius target line. — Temperature trajectories · representative episodeEnlarge image

This simulated thermostat study compares Q-learning, SARSA and DQN with rule-based and proportional control. The question is how closely each controller maintains a 25°C target, and what that accuracy costs in control effort and frequent changes of action.

The learned controllers were trained for 2,000 episodes, then evaluated alongside the baselines over 100 matched, 200-step episodes. Shared initial conditions and noise sequences made the comparisons consistent. The main figure shows one representative temperature trajectory; the smaller chart summarizes action changes across the evaluation set.

DQN achieved the highest mean reward and a mean absolute temperature error of 0.408°C. Both traditional baselines used less normalized control effort and switched less frequently than the learned controllers. The reward penalized temperature error and action magnitude, but not switching itself. I focused on validation, standardized figures and metrics, documentation and the final comparison write-up. Controller development was shared across the team.

Bar chart of mean action changes per 200-step episode for Q-learning, SARSA, DQN, rule-based, and proportional controllers, with all three learned controllers above both traditional baselines. — Action changes · 100 matched evaluationsEnlarge image

These results describe a simplified single-zone simulation. Normalized action magnitude is not measured electrical energy, and the study does not establish real-building savings or hardware reliability.

Back to Engineering