Sketching meets random projection in the dual: A provable recovery algorithm for big and high-dimensional data

Jialei Wang, Jason D. Lee, Mehrdad Mahdavi, Mladen Kolar, Nathan Srebro

Research output: Contribution to conferencePaper

3 Citations (Scopus)

Abstract

We provide a unified optimization view of iterative Hessian sketch (IHS) and iterative dual random projection (IDRP). We establish a primal-dual connection between the Hessian sketch and dual random projection, and show that their iterative extensions are optimization processes with preconditioning. We develop accelerated versions of IHS and IDRP based on this insight together with conjugate gradient descent, and propose a primal-dual sketch method that simultaneously reduces the sample size and dimensionality.

Original languageEnglish (US)
StatePublished - Jan 1 2017
Event20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017 - Fort Lauderdale, United States
Duration: Apr 20 2017Apr 22 2017

Conference

Conference20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017
CountryUnited States
CityFort Lauderdale
Period4/20/174/22/17

Fingerprint

Random Projection
Sketching
High-dimensional Data
Recovery
Primal-dual Method
Conjugate Gradient
Gradient Descent
Primal-dual
Preconditioning
Process Optimization
Dimensionality
Sample Size
Optimization

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Statistics and Probability

Cite this

Wang, J., Lee, J. D., Mahdavi, M., Kolar, M., & Srebro, N. (2017). Sketching meets random projection in the dual: A provable recovery algorithm for big and high-dimensional data. Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States.
Wang, Jialei ; Lee, Jason D. ; Mahdavi, Mehrdad ; Kolar, Mladen ; Srebro, Nathan. / Sketching meets random projection in the dual : A provable recovery algorithm for big and high-dimensional data. Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States.
@conference{26587fb3c3554c82bdb20dbdb3ada15e,
title = "Sketching meets random projection in the dual: A provable recovery algorithm for big and high-dimensional data",
abstract = "We provide a unified optimization view of iterative Hessian sketch (IHS) and iterative dual random projection (IDRP). We establish a primal-dual connection between the Hessian sketch and dual random projection, and show that their iterative extensions are optimization processes with preconditioning. We develop accelerated versions of IHS and IDRP based on this insight together with conjugate gradient descent, and propose a primal-dual sketch method that simultaneously reduces the sample size and dimensionality.",
author = "Jialei Wang and Lee, {Jason D.} and Mehrdad Mahdavi and Mladen Kolar and Nathan Srebro",
year = "2017",
month = "1",
day = "1",
language = "English (US)",
note = "20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017 ; Conference date: 20-04-2017 Through 22-04-2017",

}

Wang, J, Lee, JD, Mahdavi, M, Kolar, M & Srebro, N 2017, 'Sketching meets random projection in the dual: A provable recovery algorithm for big and high-dimensional data' Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States, 4/20/17 - 4/22/17, .

Sketching meets random projection in the dual : A provable recovery algorithm for big and high-dimensional data. / Wang, Jialei; Lee, Jason D.; Mahdavi, Mehrdad; Kolar, Mladen; Srebro, Nathan.

2017. Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States.

Research output: Contribution to conferencePaper

TY - CONF

T1 - Sketching meets random projection in the dual

T2 - A provable recovery algorithm for big and high-dimensional data

AU - Wang, Jialei

AU - Lee, Jason D.

AU - Mahdavi, Mehrdad

AU - Kolar, Mladen

AU - Srebro, Nathan

PY - 2017/1/1

Y1 - 2017/1/1

N2 - We provide a unified optimization view of iterative Hessian sketch (IHS) and iterative dual random projection (IDRP). We establish a primal-dual connection between the Hessian sketch and dual random projection, and show that their iterative extensions are optimization processes with preconditioning. We develop accelerated versions of IHS and IDRP based on this insight together with conjugate gradient descent, and propose a primal-dual sketch method that simultaneously reduces the sample size and dimensionality.

AB - We provide a unified optimization view of iterative Hessian sketch (IHS) and iterative dual random projection (IDRP). We establish a primal-dual connection between the Hessian sketch and dual random projection, and show that their iterative extensions are optimization processes with preconditioning. We develop accelerated versions of IHS and IDRP based on this insight together with conjugate gradient descent, and propose a primal-dual sketch method that simultaneously reduces the sample size and dimensionality.

UR - http://www.scopus.com/inward/record.url?scp=85067578192&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067578192&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85067578192

ER -

Wang J, Lee JD, Mahdavi M, Kolar M, Srebro N. Sketching meets random projection in the dual: A provable recovery algorithm for big and high-dimensional data. 2017. Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States.