Life-cycle policies for large engineering systems under complete and partial observability

C. P. Andriotis, K. G. Papakonstantinou

Research output: Contribution to conferencePaperpeer-review

1 Scopus citations

Abstract

Management of structures and infrastructure systems has gained significant attention in the pursuit of optimal inspection and maintenance life-cycle policies that are able to handle diverse deteriorating effects of stochastic nature and satisfy long-term objectives. Such sequential decision problems can be efficiently formulated along the premises of Markov Decision Processes (MDP) and Partially Observable Markov Decision Processes (POMDP), which describe agent-based acting in environments with Markovian dynamics, equipped with rewards, actions, and complete or partial observations. In systems with relatively low dimensional state and action spaces, MDPs and POMDPs can be satisfactorily solved using different dynamic programming algorithms, such as value iteration with or without synchronous updates and point-based approaches for partial observability cases. However, optimal planning for large systems with multiple components is computationally hard and severely suffers from the curse of dimensionality. Namely, the system states and actions can grow exponentially with the number of components, in the most general and adverse case, making the problem intractable by conventional dynamic programming schemes. In this work, Deep Reinforcement Learning (DRL) is implemented, with emphasis in the development and application of deep architectures, suitable for large engineering systems. The developed approach leverages component-wise information to prescribe component-wise actions, while maintaining global optimality on the system level. Thereby, the system life-cycle cost functions are efficiently parametrized for large state and action spaces through nonlinear approximations, enabling adept planning in complex decision problems. Results are presented for a multi-component system, evaluated against various condition-based policies.

Original languageEnglish (US)
StatePublished - Jan 1 2019
Event13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP 2019 - Seoul, Korea, Republic of
Duration: May 26 2019May 30 2019

Conference

Conference13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP 2019
CountryKorea, Republic of
CitySeoul
Period5/26/195/30/19

All Science Journal Classification (ASJC) codes

  • Civil and Structural Engineering
  • Statistics and Probability

Fingerprint Dive into the research topics of 'Life-cycle policies for large engineering systems under complete and partial observability'. Together they form a unique fingerprint.

Cite this