Learning traffic signal control from demonstrations

Yuanhao Xiong, Kai Xu, Guanjie Zheng, Zhenhui Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reinforcement learning (RL) has recently become a promising approach in various decision-making tasks. Among them, traffic signal control is the one where RL makes a great breakthrough. However, these methods always suffer from the prominent exploration problem and even fail to converge. To resolve this issue, we make an analogy between agents and humans. Agents can learn from demonstrations generated by traditional traffic signal control methods, in the similar way as people master a skill from expert knowledge. Therefore, we propose DemoLight, for the first time, to leverage demonstrations collected from classic methods to accelerate learning. Based on the state-of-the-art deep RL method Advantage Actor-Critic (A2C), training with demos are carried out for both the actor and the critic and reinforcement learning is followed for further improvement. Results under real-world datasets show that DemoLight enables a more efficient exploration and outperforms existing baselines with faster convergence and better performance.

Original languageEnglish (US)
Title of host publicationCIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management
PublisherAssociation for Computing Machinery
Pages2289-2292
Number of pages4
ISBN (Electronic)9781450369763
DOIs
StatePublished - Nov 3 2019
Event28th ACM International Conference on Information and Knowledge Management, CIKM 2019 - Beijing, China
Duration: Nov 3 2019Nov 7 2019

Publication series

NameInternational Conference on Information and Knowledge Management, Proceedings

Conference

Conference28th ACM International Conference on Information and Knowledge Management, CIKM 2019
CountryChina
CityBeijing
Period11/3/1911/7/19

Fingerprint

Reinforcement learning
Decision making
Learning methods
Leverage
Expert knowledge

All Science Journal Classification (ASJC) codes

  • Business, Management and Accounting(all)
  • Decision Sciences(all)

Cite this

Xiong, Y., Xu, K., Zheng, G., & Li, Z. (2019). Learning traffic signal control from demonstrations. In CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management (pp. 2289-2292). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3357384.3358079
Xiong, Yuanhao ; Xu, Kai ; Zheng, Guanjie ; Li, Zhenhui. / Learning traffic signal control from demonstrations. CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2019. pp. 2289-2292 (International Conference on Information and Knowledge Management, Proceedings).
@inproceedings{4717fc94dfb6449bb57f45e4237fd660,
title = "Learning traffic signal control from demonstrations",
abstract = "Reinforcement learning (RL) has recently become a promising approach in various decision-making tasks. Among them, traffic signal control is the one where RL makes a great breakthrough. However, these methods always suffer from the prominent exploration problem and even fail to converge. To resolve this issue, we make an analogy between agents and humans. Agents can learn from demonstrations generated by traditional traffic signal control methods, in the similar way as people master a skill from expert knowledge. Therefore, we propose DemoLight, for the first time, to leverage demonstrations collected from classic methods to accelerate learning. Based on the state-of-the-art deep RL method Advantage Actor-Critic (A2C), training with demos are carried out for both the actor and the critic and reinforcement learning is followed for further improvement. Results under real-world datasets show that DemoLight enables a more efficient exploration and outperforms existing baselines with faster convergence and better performance.",
author = "Yuanhao Xiong and Kai Xu and Guanjie Zheng and Zhenhui Li",
year = "2019",
month = "11",
day = "3",
doi = "10.1145/3357384.3358079",
language = "English (US)",
series = "International Conference on Information and Knowledge Management, Proceedings",
publisher = "Association for Computing Machinery",
pages = "2289--2292",
booktitle = "CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management",

}

Xiong, Y, Xu, K, Zheng, G & Li, Z 2019, Learning traffic signal control from demonstrations. in CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, pp. 2289-2292, 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, 11/3/19. https://doi.org/10.1145/3357384.3358079

Learning traffic signal control from demonstrations. / Xiong, Yuanhao; Xu, Kai; Zheng, Guanjie; Li, Zhenhui.

CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2019. p. 2289-2292 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Learning traffic signal control from demonstrations

AU - Xiong, Yuanhao

AU - Xu, Kai

AU - Zheng, Guanjie

AU - Li, Zhenhui

PY - 2019/11/3

Y1 - 2019/11/3

N2 - Reinforcement learning (RL) has recently become a promising approach in various decision-making tasks. Among them, traffic signal control is the one where RL makes a great breakthrough. However, these methods always suffer from the prominent exploration problem and even fail to converge. To resolve this issue, we make an analogy between agents and humans. Agents can learn from demonstrations generated by traditional traffic signal control methods, in the similar way as people master a skill from expert knowledge. Therefore, we propose DemoLight, for the first time, to leverage demonstrations collected from classic methods to accelerate learning. Based on the state-of-the-art deep RL method Advantage Actor-Critic (A2C), training with demos are carried out for both the actor and the critic and reinforcement learning is followed for further improvement. Results under real-world datasets show that DemoLight enables a more efficient exploration and outperforms existing baselines with faster convergence and better performance.

AB - Reinforcement learning (RL) has recently become a promising approach in various decision-making tasks. Among them, traffic signal control is the one where RL makes a great breakthrough. However, these methods always suffer from the prominent exploration problem and even fail to converge. To resolve this issue, we make an analogy between agents and humans. Agents can learn from demonstrations generated by traditional traffic signal control methods, in the similar way as people master a skill from expert knowledge. Therefore, we propose DemoLight, for the first time, to leverage demonstrations collected from classic methods to accelerate learning. Based on the state-of-the-art deep RL method Advantage Actor-Critic (A2C), training with demos are carried out for both the actor and the critic and reinforcement learning is followed for further improvement. Results under real-world datasets show that DemoLight enables a more efficient exploration and outperforms existing baselines with faster convergence and better performance.

UR - http://www.scopus.com/inward/record.url?scp=85075457757&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075457757&partnerID=8YFLogxK

U2 - 10.1145/3357384.3358079

DO - 10.1145/3357384.3358079

M3 - Conference contribution

AN - SCOPUS:85075457757

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 2289

EP - 2292

BT - CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management

PB - Association for Computing Machinery

ER -

Xiong Y, Xu K, Zheng G, Li Z. Learning traffic signal control from demonstrations. In CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery. 2019. p. 2289-2292. (International Conference on Information and Knowledge Management, Proceedings). https://doi.org/10.1145/3357384.3358079