Risk-Averse Predictive Maintenance Scheduling with Distributional Reinforcement Learning using Data-Driven Probabilistic Prognostics

Abstract

Maintenance scheduling for complex industrial systems, such as turbo jet engines, is critical for ensuring operational efficiency and safety. While data-driven prognostics have shown potential for improving predictive maintenance planning, existing approaches often fail to explicitly account for the safety-critical nature of these systems, typically addressing it through assigning high costs to failures. This thesis proposes a novel risk-averse approach to integrating data-driven probabilistic prognostics into predictive maintenance scheduling. The proposed methodology is applied to NASA's turbofan engine C-MAPSS data set. A threshold-weighted scoring rule is employed as the loss function in a neural network model to induce aversion to downside-risk when estimating the distribution of the remaining useful life (RUL). Building on these estimates, a Distributional Reinforcement Learning (DRL) model is developed for predictive maintenance scheduling. Here, risk-aversion is introduced by optimizing the agent’s decision-making based on the Conditional Value at Risk (CVaR) of the return distribution, rather than the mean. Results show that the forecasting model incorporating the threshold-weighted scoring rule demonstrates a tendency to underestimate RUL, effectively inducing the desired risk-averse behavior with only minor losses in overall performance. The risk-averse maintenance scheduling models exhibited a noticeable, though slightly inconsistent, trend towards preventing engine failures more effectively, with marginally higher average RUL at scheduled replacements compared to their risk-neutral counterparts. The scheduling agents learned to optimize the use of two out of three maintenance actions to balance failure prevention and operational efficiency. This study demonstrates the feasibility of incorporating downside-risk aversion in both RUL estimation and maintenance scheduling, offering a more robust framework for enhancing safety and performance in predictive maintenance strategies.