Optimization of sewer flushing programs: A deep reinforcement learning approach

Flushing programs are critical for managing municipal sewer assets and require efficient planning and funding. The multi-year flushing program planning decision is a complex and challenging problem involving a vast number of decision alternatives and dependencies among asset components. This paper presents a deep reinforcement learning (DRL) approach to solve this problem. The sewer network is divided into catchments and flushing decisions are made at the catchment level. The problem is formulated as a Markov decision process with the goal of maximizing the expected cumulative flushing effectiveness reward. The annual budget has a lower and upper bound, and unused budget can be saved for future years. The results show that the DRL approach performs better than a conventional genetic algorithm and is close to the Pareto frontier, particularly for longer planning horizons.