Search

This paper introduces a benchmark comparison of distributed optimization algorithms based on dual decomposition for model predictive control problems with constraint coupling. Such problems can be viewed as multiple subsystems linked through constraints on shared, limited resources. In a dual decomposition approach, the allocation and use of these resources are coordinated by repeatedly calculating their associated prices and distributing those values to the participating subsystems. After a short overview of model predictive control, various architectures and communication structures suitable for distributed implementations are outlined. The centralized control problem is then decomposed into several subproblems by introducing dual variables. A range of distributed optimization methods is examined, including the recently developed quasi-Newton dual ascent algorithm. Additionally, an epigraph reformulation of the bundle, combined with a line search technique, is proposed for the quasi-Newton dual ascent method; these enhancements improve its numerical stability and accelerate convergence relative to the earlier trust-region approach. Lastly, the quasi-Newton dual ascent algorithm is benchmarked against the subgradient method, the bundle trust method, and the alternating direction method of multipliers across a broad set of test problems. All benchmark instances are openly accessible on GitHub. 

Previous research has highlighted the need for standardized experimental settings in building control studies, and recent work on real-world reinforcement learning (RL) has introduced challenges that remain largely unexplored. Building on these insights, we propose a curated set of nine real-world challenges specifically aimed at RL control in grid-interactive buildings (GIBs). We suggest that studies in this field adopt this challenge-based perspective alongside standardized environments to ensure repeatable results. While advanced control strategies like model predictive control (MPC) and RL each offer unique advantages, both face limitations that hinder real-world deployment, and direct, unbiased comparisons between them are scarce. By framing evaluations around concrete challenges, it becomes possible to systematically test controller performance under diverse conditions and facilitate fairer comparisons. To demonstrate this approach, we focus on the offline learning challenge using CityLearn, an OpenAI Gym environment tailored for RL in demand response scenarios, in which energy storage across a collection of heterogeneous buildings is manipulated to shape the aggregated electricity demand profile. Using this setup, we examine how different levels of domain knowledge and RL algorithm complexity influence outcomes, revealing that the sequence of operations (SOOs) generated by a rule-based controller (RBC)—which provides training logs for offline RL agents—directly affects performance on four energy flexibility metrics. Prolonged offline training using an optimized RBC yields sustained performance gains, whereas training on logs from a simplified RBC can deteriorate agent effectiveness over time. Additionally, sharing information between agents does not appear to affect results significantly. Overall, we call for stronger interdisciplinary collaboration to tackle these real-world challenges and unlock the full potential of GIB control systems.