On convergence rates of game theoretic reinforcement learning algorithms

Zhisheng Hu, Minghui Zhu, Ping Chen, Peng Liu

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

This paper investigates a class of multi-player discrete games where each player aims to maximize its own utility function. Each player does not know the other players’ action sets, their deployed actions or the structures of its own or the others’ utility functions. Instead, each player only knows its own deployed actions and its received utility values in recent history. We propose a reinforcement learning algorithm which converges to the set of action profiles which have maximal stochastic potential with probability one. Furthermore, an upper bound on the convergence rate is derived and is minimized when the exploration rates are restricted to p-series. The algorithm performance is verified using a case study in the smart grid.

Original languageEnglish (US)
Pages (from-to)90-101
Number of pages12
JournalAutomatica
Volume104
DOIs
StatePublished - Jun 2019

Fingerprint

Reinforcement learning
Learning algorithms

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Cite this

@article{02446a0f3187430d873c0c44bc78ac1e,
title = "On convergence rates of game theoretic reinforcement learning algorithms",
abstract = "This paper investigates a class of multi-player discrete games where each player aims to maximize its own utility function. Each player does not know the other players’ action sets, their deployed actions or the structures of its own or the others’ utility functions. Instead, each player only knows its own deployed actions and its received utility values in recent history. We propose a reinforcement learning algorithm which converges to the set of action profiles which have maximal stochastic potential with probability one. Furthermore, an upper bound on the convergence rate is derived and is minimized when the exploration rates are restricted to p-series. The algorithm performance is verified using a case study in the smart grid.",
author = "Zhisheng Hu and Minghui Zhu and Ping Chen and Peng Liu",
year = "2019",
month = "6",
doi = "10.1016/j.automatica.2019.02.032",
language = "English (US)",
volume = "104",
pages = "90--101",
journal = "Automatica",
issn = "0005-1098",
publisher = "Elsevier Limited",

}

On convergence rates of game theoretic reinforcement learning algorithms. / Hu, Zhisheng; Zhu, Minghui; Chen, Ping; Liu, Peng.

In: Automatica, Vol. 104, 06.2019, p. 90-101.

Research output: Contribution to journalArticle

TY - JOUR

T1 - On convergence rates of game theoretic reinforcement learning algorithms

AU - Hu, Zhisheng

AU - Zhu, Minghui

AU - Chen, Ping

AU - Liu, Peng

PY - 2019/6

Y1 - 2019/6

N2 - This paper investigates a class of multi-player discrete games where each player aims to maximize its own utility function. Each player does not know the other players’ action sets, their deployed actions or the structures of its own or the others’ utility functions. Instead, each player only knows its own deployed actions and its received utility values in recent history. We propose a reinforcement learning algorithm which converges to the set of action profiles which have maximal stochastic potential with probability one. Furthermore, an upper bound on the convergence rate is derived and is minimized when the exploration rates are restricted to p-series. The algorithm performance is verified using a case study in the smart grid.

AB - This paper investigates a class of multi-player discrete games where each player aims to maximize its own utility function. Each player does not know the other players’ action sets, their deployed actions or the structures of its own or the others’ utility functions. Instead, each player only knows its own deployed actions and its received utility values in recent history. We propose a reinforcement learning algorithm which converges to the set of action profiles which have maximal stochastic potential with probability one. Furthermore, an upper bound on the convergence rate is derived and is minimized when the exploration rates are restricted to p-series. The algorithm performance is verified using a case study in the smart grid.

UR - http://www.scopus.com/inward/record.url?scp=85062684426&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062684426&partnerID=8YFLogxK

U2 - 10.1016/j.automatica.2019.02.032

DO - 10.1016/j.automatica.2019.02.032

M3 - Article

AN - SCOPUS:85062684426

VL - 104

SP - 90

EP - 101

JO - Automatica

JF - Automatica

SN - 0005-1098

ER -