Constrained Markov Decision Process Modeling for Optimal Sensing of Cardiac Events in Mobile Health

Bing Yao, Yun Chen, Hui Yang

Research output: Contribution to journalArticlepeer-review


Rapid advances in the smartphone, wearable sensing, and wireless communication provide an unprecedented opportunity to develop mobile systems for smart health management. Mobile cardiac sensing collects health-related data from individuals and enables the extraction of information pertinent to cardiac conditions. However, wireless sensors in ambulatory care settings operate on batteries. All-time sensing and monitoring will result in fast depletion of the battery in the mobile system. There is an urgent need to develop optimal sensing schemes that will reduce energy consumption while satisfying the requirements in the detection of cardiac events. In this article, we develop a constrained Markov decision process (CMDP) framework to optimize mobile electrocardiography (ECG) sensing under the constraint of the energy budget. We first characterize the cardiac states from ECG signals using the heterogeneous recurrence analysis. Second, we model the stochastic dynamics in cardiac processes as a continuous-time Markov chain (CTMC). Third, we optimize the ECG sensing through a CMDP framework under the constraint of energy budget. Finally, we validate and evaluate the performance of our CMDP policy in both simulation and real-world case studies. Experimental results demonstrate that the proposed CMDP policy significantly outperforms the traditional uniform and mean-time-to-event (MTTE) policies. Specifically, the error of state estimation is reduced by 34.0% in the real-world case study for energy-constrained sensing of cardiac events.

Original languageEnglish (US)
JournalIEEE Transactions on Automation Science and Engineering
StateAccepted/In press - 2021

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Constrained Markov Decision Process Modeling for Optimal Sensing of Cardiac Events in Mobile Health'. Together they form a unique fingerprint.

Cite this