TY - JOUR

T1 - On the global convergence of relative value iteration for infinite-horizon risk-sensitive control of diffusions

AU - Hmedi, Hassan

AU - Arapostathis, Ari

AU - Pang, Guodong

N1 - Funding Information:
The work of Ari Arapostathis was supported in part by the Army Research Office (ARO), United States of America through grant W911NF-17-1-001 , and in part by the National Science Foundation (NSF), United States of America through grant DMS-1715210 . Guodong Pang was partly supported by the ARO, United States of America grant W911NF-17-1-001 and by the NSF, United States of America grant DMS-2216765 .
Publisher Copyright:
© 2022 Elsevier B.V.

PY - 2023/1

Y1 - 2023/1

N2 - In Arapostathis and Borkar (2020), a multiplicative relative value iteration algorithm (RVI) for infinite-horizon risk-sensitive control of diffusions in Rd is studied. Assuming that there exists a control for which the diffusion is positive recurrent, the authors have established that the multiplicative value iteration (VI) algorithm converges to the solution of the multiplicative (risk-sensitive) HJB equation starting from an initial condition within the neighborhood of the solution (local convergence). Under a blanket (uniform) exponential ergodicity assumption, the authors have also shown that the RVI algorithm converges to the solution of the multiplicative HJB equation starting from any positive initial condition (global convergence). In this paper, we revisit this problem without assuming the blanket (uniform) condition. We instead assume a near-monotone running cost, and in addition, a structural assumption relating the running cost function to the solution of the multiplicative HJB equation. We show that this structural assumption implies the existence of a control under which the ground state diffusion is exponentially ergodic. More importantly, a global convergence result of the multiplicative VI/RVI algorithms is established; thus, extending upon the results in Arapostathis and Borkar (2020).

AB - In Arapostathis and Borkar (2020), a multiplicative relative value iteration algorithm (RVI) for infinite-horizon risk-sensitive control of diffusions in Rd is studied. Assuming that there exists a control for which the diffusion is positive recurrent, the authors have established that the multiplicative value iteration (VI) algorithm converges to the solution of the multiplicative (risk-sensitive) HJB equation starting from an initial condition within the neighborhood of the solution (local convergence). Under a blanket (uniform) exponential ergodicity assumption, the authors have also shown that the RVI algorithm converges to the solution of the multiplicative HJB equation starting from any positive initial condition (global convergence). In this paper, we revisit this problem without assuming the blanket (uniform) condition. We instead assume a near-monotone running cost, and in addition, a structural assumption relating the running cost function to the solution of the multiplicative HJB equation. We show that this structural assumption implies the existence of a control under which the ground state diffusion is exponentially ergodic. More importantly, a global convergence result of the multiplicative VI/RVI algorithms is established; thus, extending upon the results in Arapostathis and Borkar (2020).

UR - http://www.scopus.com/inward/record.url?scp=85142485992&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85142485992&partnerID=8YFLogxK

U2 - 10.1016/j.sysconle.2022.105413

DO - 10.1016/j.sysconle.2022.105413

M3 - Article

AN - SCOPUS:85142485992

SN - 0167-6911

VL - 171

JO - Systems and Control Letters

JF - Systems and Control Letters

M1 - 105413

ER -