TY - JOUR

T1 - On the Analysis of Variance-reduced and Randomized Projection Variants of Single Projection Schemes for Monotone Stochastic Variational Inequality Problems

AU - Cui, Shisheng

AU - Shanbhag, Uday V.

N1 - Funding Information:
The authors have been partly funded by NSF Grants 1246887 and 1400217, DOE ARPA-E DE-AR0001076 as well as the Gary and Sheila Bello Chair funds. A preliminary conference version of this work appeared in []. Acknowledgements
Publisher Copyright:
© 2021, The Author(s), under exclusive licence to Springer Nature B.V. part of Springer Nature.

PY - 2021/6

Y1 - 2021/6

N2 - Classical extragradient schemes and their stochastic counterpart represent a cornerstone for resolving monotone variational inequality problems. Yet, such schemes have a per-iteration complexity of two projections onto a convex set and require two evaluations of the map, the former of which could be relatively expensive. We consider two related avenues where the per-iteration complexity is significantly reduced: (i) A stochastic projected reflected gradient method requiring a single evaluation of the map and a single projection; and (ii) A stochastic subgradient extragradient method that requires two evaluations of the map, a single projection onto the associated feasibility set, and a significantly cheaper projection (onto a halfspace) computable in closed form. Under a variance-reduced framework reliant on a sample-average of the map based on an increasing batch-size, we prove almost sure convergence of the iterates to a random point in the solution set for both schemes. Additionally, non-asymptotic rate guarantees are derived for both schemes in terms of the gap function; notably, both rates match the best-known rates obtained in deterministic regimes. To address feasibility sets given by the intersection of a large number of convex constraints, we adapt both of the aforementioned schemes to a random projection framework. We then show that the random projection analogs of both schemes also display almost sure convergence under a weak-sharpness requirement; furthermore, without imposing the weak-sharpness requirement, both schemes are characterized by the optimal rate in terms of the gap function of the projection of the averaged sequence onto the set as well as the infeasibility of this sequence. Preliminary numerics support theoretical findings and the schemes outperform standard extragradient schemes in terms of the per-iteration complexity.

AB - Classical extragradient schemes and their stochastic counterpart represent a cornerstone for resolving monotone variational inequality problems. Yet, such schemes have a per-iteration complexity of two projections onto a convex set and require two evaluations of the map, the former of which could be relatively expensive. We consider two related avenues where the per-iteration complexity is significantly reduced: (i) A stochastic projected reflected gradient method requiring a single evaluation of the map and a single projection; and (ii) A stochastic subgradient extragradient method that requires two evaluations of the map, a single projection onto the associated feasibility set, and a significantly cheaper projection (onto a halfspace) computable in closed form. Under a variance-reduced framework reliant on a sample-average of the map based on an increasing batch-size, we prove almost sure convergence of the iterates to a random point in the solution set for both schemes. Additionally, non-asymptotic rate guarantees are derived for both schemes in terms of the gap function; notably, both rates match the best-known rates obtained in deterministic regimes. To address feasibility sets given by the intersection of a large number of convex constraints, we adapt both of the aforementioned schemes to a random projection framework. We then show that the random projection analogs of both schemes also display almost sure convergence under a weak-sharpness requirement; furthermore, without imposing the weak-sharpness requirement, both schemes are characterized by the optimal rate in terms of the gap function of the projection of the averaged sequence onto the set as well as the infeasibility of this sequence. Preliminary numerics support theoretical findings and the schemes outperform standard extragradient schemes in terms of the per-iteration complexity.

UR - http://www.scopus.com/inward/record.url?scp=85104055602&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85104055602&partnerID=8YFLogxK

U2 - 10.1007/s11228-021-00572-6

DO - 10.1007/s11228-021-00572-6

M3 - Article

AN - SCOPUS:85104055602

VL - 29

SP - 453

EP - 499

JO - Set-Valued and Variational Analysis

JF - Set-Valued and Variational Analysis

SN - 1877-0533

IS - 2

ER -