Optimal control of infinite horizon partially observable decision processes modelled as generators of probabilistic regular languages

Ishanu Chattopadhyay, Asok Ray

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Decision processes with incomplete state feedback have been traditionally modelled as partially observable Markov decision processes. In this article, we present an alternative formulation based on probabilistic regular languages. The proposed approach generalises the recently reported work on language measure theoretic optimal control for perfectly observable situations and shows that such a framework is far more computationally tractable to the classical alternative. In particular, we show that the infinite horizon decision problem under partial observation, modelled in the proposed framework, is λ-approximable and, in general, is not harder to solve compared to the fully observable case. The approach is illustrated via two simple examples.

Original languageEnglish (US)
Pages (from-to)457-483
Number of pages27
JournalInternational Journal of Control
Volume83
Issue number3
DOIs
StatePublished - Mar 1 2010

Fingerprint

Formal languages
State feedback

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Computer Science Applications

Cite this

@article{4d363d0202c94a79b3b61f1e12540090,
title = "Optimal control of infinite horizon partially observable decision processes modelled as generators of probabilistic regular languages",
abstract = "Decision processes with incomplete state feedback have been traditionally modelled as partially observable Markov decision processes. In this article, we present an alternative formulation based on probabilistic regular languages. The proposed approach generalises the recently reported work on language measure theoretic optimal control for perfectly observable situations and shows that such a framework is far more computationally tractable to the classical alternative. In particular, we show that the infinite horizon decision problem under partial observation, modelled in the proposed framework, is λ-approximable and, in general, is not harder to solve compared to the fully observable case. The approach is illustrated via two simple examples.",
author = "Ishanu Chattopadhyay and Asok Ray",
year = "2010",
month = "3",
day = "1",
doi = "10.1080/00207170903193458",
language = "English (US)",
volume = "83",
pages = "457--483",
journal = "International Journal of Control",
issn = "0020-7179",
publisher = "Taylor and Francis Ltd.",
number = "3",

}

Optimal control of infinite horizon partially observable decision processes modelled as generators of probabilistic regular languages. / Chattopadhyay, Ishanu; Ray, Asok.

In: International Journal of Control, Vol. 83, No. 3, 01.03.2010, p. 457-483.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Optimal control of infinite horizon partially observable decision processes modelled as generators of probabilistic regular languages

AU - Chattopadhyay, Ishanu

AU - Ray, Asok

PY - 2010/3/1

Y1 - 2010/3/1

N2 - Decision processes with incomplete state feedback have been traditionally modelled as partially observable Markov decision processes. In this article, we present an alternative formulation based on probabilistic regular languages. The proposed approach generalises the recently reported work on language measure theoretic optimal control for perfectly observable situations and shows that such a framework is far more computationally tractable to the classical alternative. In particular, we show that the infinite horizon decision problem under partial observation, modelled in the proposed framework, is λ-approximable and, in general, is not harder to solve compared to the fully observable case. The approach is illustrated via two simple examples.

AB - Decision processes with incomplete state feedback have been traditionally modelled as partially observable Markov decision processes. In this article, we present an alternative formulation based on probabilistic regular languages. The proposed approach generalises the recently reported work on language measure theoretic optimal control for perfectly observable situations and shows that such a framework is far more computationally tractable to the classical alternative. In particular, we show that the infinite horizon decision problem under partial observation, modelled in the proposed framework, is λ-approximable and, in general, is not harder to solve compared to the fully observable case. The approach is illustrated via two simple examples.

UR - http://www.scopus.com/inward/record.url?scp=77951115539&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77951115539&partnerID=8YFLogxK

U2 - 10.1080/00207170903193458

DO - 10.1080/00207170903193458

M3 - Article

AN - SCOPUS:77951115539

VL - 83

SP - 457

EP - 483

JO - International Journal of Control

JF - International Journal of Control

SN - 0020-7179

IS - 3

ER -