CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario

Huichu Zhang, Yaoyao Ding, Weinan Zhang, Siyuan Feng, Yichen Zhu, Yong Yu, Zhenhui Li, Chang Liu, Zihan Zhou, Haiming Jin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.

Original languageEnglish (US)
Title of host publicationThe Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019
PublisherAssociation for Computing Machinery, Inc
Pages3620-3624
Number of pages5
ISBN (Electronic)9781450366748
DOIs
StatePublished - May 13 2019
Event2019 World Wide Web Conference, WWW 2019 - San Francisco, United States
Duration: May 13 2019May 17 2019

Publication series

NameThe Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019

Conference

Conference2019 World Wide Web Conference, WWW 2019
CountryUnited States
CitySan Francisco
Period5/13/195/17/19

Fingerprint

Reinforcement learning
Traffic signals
Simulators
User interfaces
Data structures
Learning systems
Monitoring

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software

Cite this

Zhang, H., Ding, Y., Zhang, W., Feng, S., Zhu, Y., Yu, Y., ... Jin, H. (2019). CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019 (pp. 3620-3624). (The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019). Association for Computing Machinery, Inc. https://doi.org/10.1145/3308558.3314139
Zhang, Huichu ; Ding, Yaoyao ; Zhang, Weinan ; Feng, Siyuan ; Zhu, Yichen ; Yu, Yong ; Li, Zhenhui ; Liu, Chang ; Zhou, Zihan ; Jin, Haiming. / CityFlow : A multi-agent reinforcement learning environment for large scale city traffic scenario. The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019. Association for Computing Machinery, Inc, 2019. pp. 3620-3624 (The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019).
@inproceedings{6a17c6eb18b44bb183e5bf730c409f9b,
title = "CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario",
abstract = "Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.",
author = "Huichu Zhang and Yaoyao Ding and Weinan Zhang and Siyuan Feng and Yichen Zhu and Yong Yu and Zhenhui Li and Chang Liu and Zihan Zhou and Haiming Jin",
year = "2019",
month = "5",
day = "13",
doi = "10.1145/3308558.3314139",
language = "English (US)",
series = "The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019",
publisher = "Association for Computing Machinery, Inc",
pages = "3620--3624",
booktitle = "The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019",

}

Zhang, H, Ding, Y, Zhang, W, Feng, S, Zhu, Y, Yu, Y, Li, Z, Liu, C, Zhou, Z & Jin, H 2019, CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario. in The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019. The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019, Association for Computing Machinery, Inc, pp. 3620-3624, 2019 World Wide Web Conference, WWW 2019, San Francisco, United States, 5/13/19. https://doi.org/10.1145/3308558.3314139

CityFlow : A multi-agent reinforcement learning environment for large scale city traffic scenario. / Zhang, Huichu; Ding, Yaoyao; Zhang, Weinan; Feng, Siyuan; Zhu, Yichen; Yu, Yong; Li, Zhenhui; Liu, Chang; Zhou, Zihan; Jin, Haiming.

The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019. Association for Computing Machinery, Inc, 2019. p. 3620-3624 (The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - CityFlow

T2 - A multi-agent reinforcement learning environment for large scale city traffic scenario

AU - Zhang, Huichu

AU - Ding, Yaoyao

AU - Zhang, Weinan

AU - Feng, Siyuan

AU - Zhu, Yichen

AU - Yu, Yong

AU - Li, Zhenhui

AU - Liu, Chang

AU - Zhou, Zihan

AU - Jin, Haiming

PY - 2019/5/13

Y1 - 2019/5/13

N2 - Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.

AB - Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.

UR - http://www.scopus.com/inward/record.url?scp=85066891747&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066891747&partnerID=8YFLogxK

U2 - 10.1145/3308558.3314139

DO - 10.1145/3308558.3314139

M3 - Conference contribution

AN - SCOPUS:85066891747

T3 - The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019

SP - 3620

EP - 3624

BT - The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019

PB - Association for Computing Machinery, Inc

ER -

Zhang H, Ding Y, Zhang W, Feng S, Zhu Y, Yu Y et al. CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019. Association for Computing Machinery, Inc. 2019. p. 3620-3624. (The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019). https://doi.org/10.1145/3308558.3314139