Search

Smart-grid optimization increasingly integrates digital twins, reinforcement learning, and cyber-physical resilience analytics into a shared research space. This convergence reflects a shift from isolated grid-control studies toward data-intensive, adaptive, and resilience-oriented infrastructure analytics. This bibliometric review maps the research landscape at the intersection of these three domains. It quantifies publication trends, identifies thematic clusters, and interprets emerging fronts in smart-grid optimization research. A structured bibliometric protocol was applied to 28 peer-reviewed journal articles published between 2017 and 2025. Co-word analysis, citation analysis, thematic clustering, and citation burst interpretation were used to organize the evidence base. The analysis reveals rising publication activity after 2019 and three dominant thematic clusters. Digital twin research centers on simulation and cyber-physical representation, reinforcement learning emphasizes adaptive grid control, and cyber-resilience analytics focuses on vulnerability, disruption, and recovery. The field is moving toward integrated frameworks that combine digital twins and reinforcement learning for proactive, cyber-resilient grid optimization. These fronts suggest a growing need for real-time, explainable, and safety-aware analytics.

High penetration of rooftop photovoltaic generation introduces rapid voltage fluctuations in low-voltage distribution feeders. These fluctuations challenge conventional voltage regulation because inverter actions must respond locally while remaining coordinated across electrically coupled nodes. This article proposes a multi-agent reinforcement learning model for grid-edge voltage regulation under distributed PV variability and communication latency. The objective is to maintain feeder voltages within operational limits while reducing unnecessary reactive power support, curtailment, and communication burden. The voltage control task is formulated as a decentralized partially observable Markov decision process. Smart inverter and battery agents use local observations during execution, while centralized training uses feeder-wide voltage, PV, and control information to train coordinated policies. Simulation results show that the proposed MARL controller improves voltage compliance relative to local Volt/VAr control and approaches centralized optimal power flow performance. The controller remains stable under bounded latency and asynchronous agent updates. The proposed framework offers a robust decentralized control strategy for high-PV distribution feeders. It combines grid-edge autonomy with coordinated learning, making it suitable for communication-limited voltage regulation.

Reinforcement learning can autonomously dispatch demand response actions under changing prices, loads, and grid constraints. However, operator acceptance requires explanations that clarify why specific customers or aggregations are selected. This article presents an explainable reinforcement learning framework for demand response dispatch using customer flexibility profiles, decomposed rewards, and operator review. The objective is to preserve dispatch performance while improving transparency and controllability. The framework combines a profile-aware RL dispatcher, a decomposed reward model, an explanation generator, and a structured operator review protocol. Evaluation is performed through simulation on realistic demand response scenarios with household and flexible-load representations. The proposed explainable RL framework is designed to achieve dispatch performance comparable to black-box RL while providing interpretable action rationales. Operator review is incorporated to approve, reject, or modify dispatch recommendations before execution. The framework bridges autonomous RL control and trustworthy grid-edge operation. It provides an algorithmic blueprint for human-centered demand response dispatch.

Reaching carbon neutrality by 2050 will require transmission system operators to accommodate unprecedented levels of variable solar and wind generation while preserving system reliability. To address this systemic constraint, Réseau de Transport d’Électricité (RTE), together with the research community, introduced Learn To Run A Power Network (L2RPN), a crowd-sourced competition designed to accelerate the integration of intermittent renewable sources into grid operations. L2RPN relies on 16 years of weekly scenarios (832 in total) modeled on a 118-node network under realistic operational constraints, framing real-time grid management as a Markov Decision Process. The six competing teams approached the problem by designing autonomous agents that combined heuristics, optimization, data scaling, supervised learning, and reinforcement learning techniques. We deliver a comprehensive assessment of all six teams’ performances under the competition’s stringent framework. Furthermore, we conduct an extensive examination of the publicly released winning solution, which ensures consistent decision-making across scenarios, performs real-time multimodal actions in less than 5 seconds, and achieves efficient topology control through action-space reduction and a neural policy that predicts beneficial grid actions with over 80% accuracy. Concurrently, we trained a neural alert module using 315,000 samples generated from leading agents, obtaining 93.9% recall in identifying hazardous states and enabling agents to anticipate potential failures. Ultimately, this study highlights both the potential and current limitations of AI in real-time grid operation, while establishing a transparent foundation for developing more robust and trustworthy systems to support the energy transition.

The rising priority placed on energy conservation, together with the adoption of renewable sources, has grown essential to sustainable practices across today’s university campuses. This research introduces an intelligent green energy management framework (SGEMS) that integrates machine learning predictions with reinforcement learning (RL) to enhance energy utilization and photovoltaic output on an eco-conscious campus. Leveraging historical records from three campus structures, the study developed a forecasting engine capable of projecting near-term electricity demand and solar yield. When paired with RL, the XGBoost method achieved markedly better accuracy in estimating consumption and production levels than competing algorithms, with root mean square errors (RMSEs) of 14.72, mean absolute errors (MAEs) of 12.00, and mean absolute percentage errors (MAPE) of 2.18%. The work also describes an accessible, browser-based dashboard that enables live energy oversight and supports timely decision-making, empowering facility managers to anticipate deficits and regulate consumption more intelligently. Overall, the presented methodology provides an adaptable blueprint that enables campuses to reduce dependence on utility grids, improve efficiency, and serve as a reference model for subsequent eco-friendly campus programs.

Advanced control strategies have been applied to manage variable power output in standalone microgrids, reducing the risk of frequency instability and power-quality deterioration. Nevertheless, a key difficulty arises: these computationally intensive techniques, when used without stability checks, may fail to cope with the fast dynamics of real-world conditions, particularly under communication delays. This research proposes a versatile real-time solution for frequency regulation that relies on an artificial neural network (ANN) confined to stable operating regions. The proposed method combines stabilizing PID controllers—derived via small-signal analysis—with an automated optimization of ANN weights and reinforcement learning (RL)-guided constraint selection. Initially, stabilizing PID controllers are developed using the stability boundary locus approach and the Mikhailov criterion, with explicit consideration of communication delays. Subsequently, controller parameters are fine-tuned in real time via an automated search for optimal coefficient sets, using a constrained ANN to keep frequency deviations within a predefined parameter space. The framework is further strengthened by an RL-based tuning mechanism trained on an interpolated stability boundary curve, guaranteeing both stability and enhanced performance. This unique integration of ANN, RL, and sophisticated PID tuning represents a significant advancement in addressing frequency control challenges in isolated AC microgrids. Experimental results demonstrate that the proposed approach surpasses conventional techniques thanks to its reduced parameter search domain. Specifically, it achieves greater reductions in both transient and steady-state frequency deviations compared to semi-constrained and unconstrained alternatives. The enhanced performance metrics and stability assessments confirm that the method delivers superior system behavior and robustness under varying operating conditions.

The increasing deployment of wind power plays a key role in addressing climate change. Nevertheless, the intermittent nature of wind generation often leads to curtailment, resulting in substantial financial losses for wind farm operators. Battery energy storage systems (BESS) can help alleviate curtailment by providing local backup power. However, dedicating storage primarily to this supporting function can greatly diminish its profitability in energy markets. Optimal BESS operation requires carefully balancing local curtailment mitigation with participation in market bidding, yet achieving this in practice remains difficult owing to the intricate coordination required and the uncertainty of both electricity prices and wind output. This study explores a combined bidding strategy for a co-located wind-plus-battery setup operating in the spot market and the Regulation Frequency Control Ancillary Service markets. We introduce an innovative deep reinforcement learning method that separates the overall market engagement into two interconnected Markov decision processes—one for each asset. This structure allows the BESS to capture surplus wind energy that would otherwise be curtailed while simultaneously participating in joint bidding across markets to optimize total operational income. Validation was performed using real-world wind farm data. The proposed coordinated bidding approach outperformed a conventional optimization-based reference case, delivering roughly 25% higher revenue and achieving 2.3 times greater reduction in wind curtailment. Findings demonstrate that participating jointly in multiple markets substantially enhances the economic viability of wind-battery installations relative to engaging in individual markets in isolation. Additional simulations indicate that charging the BESS with otherwise curtailed wind power yields further economic benefits. Effective deployment of the developed algorithm is expected to promote greater co-location of renewable generation and storage facilities, thereby realizing broader advantages for the power system.

This paper presents a real-time preventive control approach utilizing deep reinforcement learning (DRL) to identify optimal control actions that maintain an adequate secure operating limit (SOL) in power systems. The SOL represents the boundary of the most stressed pre-contingency operating condition that the system can endure following a specified set of credible contingencies while satisfying predefined stability requirements. The proposed DRL technique employs a hybrid control framework that enables simultaneous management of both discrete and continuous control variables. Its effectiveness is assessed using a modified Nordic32 test system. Findings indicate that the DRL method rapidly acquires a robust control policy capable of securing sufficient SOL across various operating conditions. The approach is benchmarked against a rule-based lookup table method and a DRL variant restricted to discrete actions. Results demonstrate that the hybrid DRL controller consistently outperforms the alternatives across all evaluated test cases, highlighting the advantages of jointly optimizing discrete and continuous variables for greater flexibility and operational efficiency.

Previous research has highlighted the need for standardized experimental settings in building control studies, and recent work on real-world reinforcement learning (RL) has introduced challenges that remain largely unexplored. Building on these insights, we propose a curated set of nine real-world challenges specifically aimed at RL control in grid-interactive buildings (GIBs). We suggest that studies in this field adopt this challenge-based perspective alongside standardized environments to ensure repeatable results. While advanced control strategies like model predictive control (MPC) and RL each offer unique advantages, both face limitations that hinder real-world deployment, and direct, unbiased comparisons between them are scarce. By framing evaluations around concrete challenges, it becomes possible to systematically test controller performance under diverse conditions and facilitate fairer comparisons. To demonstrate this approach, we focus on the offline learning challenge using CityLearn, an OpenAI Gym environment tailored for RL in demand response scenarios, in which energy storage across a collection of heterogeneous buildings is manipulated to shape the aggregated electricity demand profile. Using this setup, we examine how different levels of domain knowledge and RL algorithm complexity influence outcomes, revealing that the sequence of operations (SOOs) generated by a rule-based controller (RBC)—which provides training logs for offline RL agents—directly affects performance on four energy flexibility metrics. Prolonged offline training using an optimized RBC yields sustained performance gains, whereas training on logs from a simplified RBC can deteriorate agent effectiveness over time. Additionally, sharing information between agents does not appear to affect results significantly. Overall, we call for stronger interdisciplinary collaboration to tackle these real-world challenges and unlock the full potential of GIB control systems.