Online Improvement of Condition-Based Maintenance Policy via Monte Carlo Tree Search

Michael Hoffman, Eunhye Song, Michael P. Brundage, Soundar Kumara

Research output: Contribution to journalArticlepeer-review

Abstract

Often in manufacturing systems, scenarios arise where the demand for maintenance exceeds the capacity of maintenance resources. This results in the problem of allocating the limited resources among machines competing for them. This maintenance scheduling problem can be formulated as a Markov decision process (MDP) with the goal of finding the optimal dynamic maintenance action given the current system state. However, as the system becomes more complex, solving an MDP suffers from the curse of dimensionality. To overcome this issue, we propose a two-stage approach that first optimizes a static condition-based maintenance (CBM) policy using a genetic algorithm (GA) and then improves the policy online via Monte Carlo tree search (MCTS). The static policy significantly reduces the state space of the online problem by allowing us to ignore machines that are not sufficiently degraded. Furthermore, we formulate MCTS to seek a maintenance schedule that maximizes the long-term production volume of the system to reconcile the conflict between maintenance and production objectives. We demonstrate that the resulting online policy is an improvement over the static CBM policy found by GA.

Original languageEnglish (US)
JournalIEEE Transactions on Automation Science and Engineering
DOIs
StateAccepted/In press - 2021

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Online Improvement of Condition-Based Maintenance Policy via Monte Carlo Tree Search'. Together they form a unique fingerprint.

Cite this