A reinforcement learning framework for trajectory prediction under uncertainty and budget constraint

Truc Viet Le, Siyuan Liu, Hoong Chuin Lau

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

We consider the problem of trajectory prediction, where a trajectory is an ordered sequence of location visits and corresponding timestamps. The problem arises when an agent makes sequential decisions to visit a set of spatial locations of interest. Each location bears a stochastic utility and the agent has a limited budget to spend. Given the agent's observed partial trajectory, our goal is to predict the agent's remaining trajectory. We propose a solution framework to the problem that incorporates both the stochastic utility of each location and the budget constraint. We first cluster the agents into groups of homogeneous behaviors called "agent types". Depending on its type, each agent's trajectory is then transformed into a discretestate sequence representation. Based on such representations, we use reinforcement learning (RL) to model the underlying decision processes and inverse RL to learn the utility distributions of the spatial locations. We finally propose two decision models to make predictions: one is based on long-term optimal planning of RL and another uses myopic heuristics. We apply the framework to predict real-world human trajectories collected in a large theme park and are able to explain the underlying processes of the observed actions.

Original languageEnglish (US)
Title of host publicationFrontiers in Artificial Intelligence and Applications
EditorsGal A. Kaminka, Frank Dignum, Eyke Hullermeier, Paolo Bouquet, Virginia Dignum, Maria Fox, Frank van Harmelen
PublisherIOS Press
Pages347-354
Number of pages8
ISBN (Electronic)9781614996712
DOIs
StatePublished - 2016
Event22nd European Conference on Artificial Intelligence, ECAI 2016 - The Hague, Netherlands
Duration: Aug 29 2016Sep 2 2016

Publication series

NameFrontiers in Artificial Intelligence and Applications
Volume285
ISSN (Print)0922-6389

Other

Other22nd European Conference on Artificial Intelligence, ECAI 2016
CountryNetherlands
CityThe Hague
Period8/29/169/2/16

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence

Fingerprint Dive into the research topics of 'A reinforcement learning framework for trajectory prediction under uncertainty and budget constraint'. Together they form a unique fingerprint.

Cite this