Reinforcement learning framework for modeling spatial sequential decisions under uncertainty

Truc Viet Le, Siyuan Liu, Hoong Chuin Lau

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

We consider the problem of trajectory prediction, where a trajectory is an ordered sequence of location visits and corresponding timestamps. The problem arises when an agent makes sequential decisions to visit a set of spatial locations of interest. Each location bears a stochastic utility and the agent has a limited budget to spend. Given the agent's observed partial trajectory, our goal is to predict the remaining trajectory. We propose a solution framework to the problem considering both the uncertainty of utility and the budget constraint. We use reinforcement learning (RL) to model the underlying decision processes and inverse RL to learn the utility distributions of the locations. We then propose two decision models to make predictions: one is based on long-term optimal planning of RL and another uses myopic heuristics. We finally apply the framework to predict real-world human trajectories and are able to explain the underlying processes of the observed actions.

Original languageEnglish (US)
Title of host publicationAAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Pages1449-1450
Number of pages2
ISBN (Electronic)9781450342391
StatePublished - Jan 1 2016
Event15th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2016 - Singapore, Singapore
Duration: May 9 2016May 13 2016

Publication series

NameProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
ISSN (Print)1548-8403
ISSN (Electronic)1558-2914

Other

Other15th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2016
CountrySingapore
CitySingapore
Period5/9/165/13/16

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering

Cite this

Le, T. V., Liu, S., & Lau, H. C. (2016). Reinforcement learning framework for modeling spatial sequential decisions under uncertainty. In AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems (pp. 1449-1450). (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).