Kernel mean shrinkage estimators

Krikamol Muandet, Bharath Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard Schölkopf

Research output: Contribution to journalReview article

10 Citations (Scopus)

Abstract

A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in RKHSs. Given a finite sample, an empirical average has been used commonly as a standard estimator of the true kernel mean. Despite a widespread use of this estimator, we show that it can be improved thanks to the well-known Stein phenomenon. We propose a new family of estimators called kernel mean shrinkage estimators (KMSEs), which benefit from both theoretical justifications and good empirical performance. The results demonstrate that the proposed estimators outperform the standard one, especially in a "large d, small n" paradigm.

Original languageEnglish (US)
JournalJournal of Machine Learning Research
Volume17
StatePublished - Apr 1 2016

Fingerprint

Shrinkage Estimator
kernel
Kernel Methods
Hilbert spaces
Estimator
Principal component analysis
Probability distributions
Kernel Principal Component Analysis
Reproducing Kernel Hilbert Space
Kernel Estimator
Justification
Probability Distribution
Paradigm
Demonstrate
Standards

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Statistics and Probability
  • Artificial Intelligence

Cite this

Muandet, K., Sriperumbudur, B., Fukumizu, K., Gretton, A., & Schölkopf, B. (2016). Kernel mean shrinkage estimators. Journal of Machine Learning Research, 17.
Muandet, Krikamol ; Sriperumbudur, Bharath ; Fukumizu, Kenji ; Gretton, Arthur ; Schölkopf, Bernhard. / Kernel mean shrinkage estimators. In: Journal of Machine Learning Research. 2016 ; Vol. 17.
@article{192e1c6446d84763a88a501760e26002,
title = "Kernel mean shrinkage estimators",
abstract = "A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in RKHSs. Given a finite sample, an empirical average has been used commonly as a standard estimator of the true kernel mean. Despite a widespread use of this estimator, we show that it can be improved thanks to the well-known Stein phenomenon. We propose a new family of estimators called kernel mean shrinkage estimators (KMSEs), which benefit from both theoretical justifications and good empirical performance. The results demonstrate that the proposed estimators outperform the standard one, especially in a {"}large d, small n{"} paradigm.",
author = "Krikamol Muandet and Bharath Sriperumbudur and Kenji Fukumizu and Arthur Gretton and Bernhard Sch{\"o}lkopf",
year = "2016",
month = "4",
day = "1",
language = "English (US)",
volume = "17",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

Muandet, K, Sriperumbudur, B, Fukumizu, K, Gretton, A & Schölkopf, B 2016, 'Kernel mean shrinkage estimators', Journal of Machine Learning Research, vol. 17.

Kernel mean shrinkage estimators. / Muandet, Krikamol; Sriperumbudur, Bharath; Fukumizu, Kenji; Gretton, Arthur; Schölkopf, Bernhard.

In: Journal of Machine Learning Research, Vol. 17, 01.04.2016.

Research output: Contribution to journalReview article

TY - JOUR

T1 - Kernel mean shrinkage estimators

AU - Muandet, Krikamol

AU - Sriperumbudur, Bharath

AU - Fukumizu, Kenji

AU - Gretton, Arthur

AU - Schölkopf, Bernhard

PY - 2016/4/1

Y1 - 2016/4/1

N2 - A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in RKHSs. Given a finite sample, an empirical average has been used commonly as a standard estimator of the true kernel mean. Despite a widespread use of this estimator, we show that it can be improved thanks to the well-known Stein phenomenon. We propose a new family of estimators called kernel mean shrinkage estimators (KMSEs), which benefit from both theoretical justifications and good empirical performance. The results demonstrate that the proposed estimators outperform the standard one, especially in a "large d, small n" paradigm.

AB - A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in RKHSs. Given a finite sample, an empirical average has been used commonly as a standard estimator of the true kernel mean. Despite a widespread use of this estimator, we show that it can be improved thanks to the well-known Stein phenomenon. We propose a new family of estimators called kernel mean shrinkage estimators (KMSEs), which benefit from both theoretical justifications and good empirical performance. The results demonstrate that the proposed estimators outperform the standard one, especially in a "large d, small n" paradigm.

UR - http://www.scopus.com/inward/record.url?scp=84979917601&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84979917601&partnerID=8YFLogxK

M3 - Review article

AN - SCOPUS:84979917601

VL - 17

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -

Muandet K, Sriperumbudur B, Fukumizu K, Gretton A, Schölkopf B. Kernel mean shrinkage estimators. Journal of Machine Learning Research. 2016 Apr 1;17.