Accelerating Stochastic Composition Optimization

Mengdi Wang, Ji Liu, Xingyuan Fang

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

We consider the stochastic nested composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method. This algorithm updates the solution based on noisy gradient queries using a two-timescale iteration. The ASC-PG is the first proximal gradient method for the stochastic composition problem that can deal with nonsmooth regularization penalty. We show that the ASC-PG exhibits faster convergence than the best known algorithms, and that it achieves the optimal sample-error complexity in several important special cases. We demonstrate the application of ASC-PG to reinforcement learning and conduct numerical experiments.

Original languageEnglish (US)
Pages (from-to)1-23
Number of pages23
JournalJournal of Machine Learning Research
Volume18
StatePublished - Oct 1 2017

Fingerprint

Gradient methods
Optimization
Chemical analysis
Gradient
Proximal Methods
Reinforcement learning
Gradient Method
Reinforcement Learning
Expected Value
Value Function
Experiments
Penalty
Regularization
Time Scales
Update
Numerical Experiment
Query
Optimization Problem
First-order
Iteration

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Statistics and Probability
  • Artificial Intelligence

Cite this

@article{1bfdb35b790b437cb27fa3b726d86960,
title = "Accelerating Stochastic Composition Optimization",
abstract = "We consider the stochastic nested composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method. This algorithm updates the solution based on noisy gradient queries using a two-timescale iteration. The ASC-PG is the first proximal gradient method for the stochastic composition problem that can deal with nonsmooth regularization penalty. We show that the ASC-PG exhibits faster convergence than the best known algorithms, and that it achieves the optimal sample-error complexity in several important special cases. We demonstrate the application of ASC-PG to reinforcement learning and conduct numerical experiments.",
author = "Mengdi Wang and Ji Liu and Xingyuan Fang",
year = "2017",
month = "10",
day = "1",
language = "English (US)",
volume = "18",
pages = "1--23",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

Accelerating Stochastic Composition Optimization. / Wang, Mengdi; Liu, Ji; Fang, Xingyuan.

In: Journal of Machine Learning Research, Vol. 18, 01.10.2017, p. 1-23.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Accelerating Stochastic Composition Optimization

AU - Wang, Mengdi

AU - Liu, Ji

AU - Fang, Xingyuan

PY - 2017/10/1

Y1 - 2017/10/1

N2 - We consider the stochastic nested composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method. This algorithm updates the solution based on noisy gradient queries using a two-timescale iteration. The ASC-PG is the first proximal gradient method for the stochastic composition problem that can deal with nonsmooth regularization penalty. We show that the ASC-PG exhibits faster convergence than the best known algorithms, and that it achieves the optimal sample-error complexity in several important special cases. We demonstrate the application of ASC-PG to reinforcement learning and conduct numerical experiments.

AB - We consider the stochastic nested composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method. This algorithm updates the solution based on noisy gradient queries using a two-timescale iteration. The ASC-PG is the first proximal gradient method for the stochastic composition problem that can deal with nonsmooth regularization penalty. We show that the ASC-PG exhibits faster convergence than the best known algorithms, and that it achieves the optimal sample-error complexity in several important special cases. We demonstrate the application of ASC-PG to reinforcement learning and conduct numerical experiments.

UR - http://www.scopus.com/inward/record.url?scp=85032913200&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85032913200&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:85032913200

VL - 18

SP - 1

EP - 23

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -