ABSTRACT
Optimal management of bridge systems and related transportation infrastructure poses multi-faceted challenges, requiring adept inspection and maintenance policies at both system and individual asset levels, to minimize life-cycle costs while considering various operational, risk, and performance constraints. This demanding type of optimization problems entails, among others, high-dimensional aspects, describing multi-component systems, long-planning horizons, diverse probabilistic and deterministic operational objectives and constraints, and inherent uncertainties associated with inspections and stochastic models. Effective coordination among individual component assets considering various inter-dependencies is also essential to enable a true system-based optimal solution. In this work, this optimization problem is formulated within the framework of Partially Observable Markov Decision Processes (POMDPs) and constrained Multi-Agent Deep Reinforcement Learning (MARL). POMDPs offer a principled mathematical approach for sequential decision-making under uncertainty, incorporating Bayesian inference to address the observation/monitoring data uncertainty, and can be suitably scaled to high-dimensional state and action spaces associated with multi-component systems, exploiting the rich representational capacities of deep learning and decentralized control settings of MARL. In this work, the recently developed DDMAC deep reinforcement learning (DRL) algorithm (Deep Decentralized Multi-Agent Actor-Critic) has been successfully deployed based on the Centralized Training and Decentralized Execution (CTDE) formulation. The efficacy and implementation aspects of the developed framework are originally studied in this work based on two existing real-world transportation networks in Virginia and Pennsylvania, USA, following all regulations imposed by the relevant agencies, as well as their overall practices, in an effort to investigate the use of the suggested framework in practical, actual settings. In both cases, DRL results significantly surpass the ones related to current state-of-practice and state-of-the-art policies, providing further support and insights toward the use of DRL-driven policies for infrastructure management.
