Reinforcement Learning Applied to an EV Charging Scheduling Problem

Abstract

This thesis investigates Maskable Proximal Policy Optimization (PPO) for Electric Vehicle Charging Scheduling (EVCS) within a non-uniform smart grid, partially integrated with renewable energy, aiming to minimize charging tardiness. Using the Polderwijk simulation, which models real-world grid bottlenecks and partial solar energy deployment, we compare Maskable PPO against traditional dispatching strategies and a heuristic approach. Despite experimentation across varied observation spaces, action spaces, and reward functions, the Maskable PPO agent consistently failed to achieve convergence, being unable to respect grid capacity or minimize delays. Our findings show the challenges of applying model-free reinforcement learning (RL) to EVCS problems with strict hard constraints. This research indicates a need for future work to investigate alternative RL architectures for integrating hard constraints to achieve convergence in complex smart grid environments.