On the generalization ability of online learning algorithms for pairwise loss functions

Purushottam Kar, Bharath Kumar Sriperumbudur, Prateek Jain, Harish C. Karnick

Research output: Contribution to conferencePaper

21 Citations (Scopus)

Abstract

In this paper, we study the generalization properties of online learning based stochastic methods for supervised learning problems where the loss function is dependent on more than one training sample (e.g., metric learning, ranking). We present a generic decoupling technique that enables us to provide Rademacher complexity-based generalization error bounds. Our bounds are in general tighter than those obtained by Wang et al. (2012) for the same problem. Using our decoupling technique, we are further able to obtain fast convergence rates for strongly convex pairwise loss functions. We are also able to analyze a class of memory efficient online learning algorithms for pairwise learning problems that use only a bounded subset of past training samples to update the hypothesis at each step. Finally, in order to complement our generalization bounds, we propose a novel memory efficient online learning algorithm for higher order learning problems with bounded regret guarantees.

Original languageEnglish (US)
Pages1478-1486
Number of pages9
StatePublished - Jan 1 2013
Event30th International Conference on Machine Learning, ICML 2013 - Atlanta, GA, United States
Duration: Jun 16 2013Jun 21 2013

Other

Other30th International Conference on Machine Learning, ICML 2013
CountryUnited States
CityAtlanta, GA
Period6/16/136/21/13

Fingerprint

Learning algorithms
Data storage equipment
Supervised learning
ability
learning
ranking
guarantee

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Sociology and Political Science

Cite this

Kar, P., Sriperumbudur, B. K., Jain, P., & Karnick, H. C. (2013). On the generalization ability of online learning algorithms for pairwise loss functions. 1478-1486. Paper presented at 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, United States.
Kar, Purushottam ; Sriperumbudur, Bharath Kumar ; Jain, Prateek ; Karnick, Harish C. / On the generalization ability of online learning algorithms for pairwise loss functions. Paper presented at 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, United States.9 p.
@conference{15fe5a0d602a47e1a5c79a1b639bbd20,
title = "On the generalization ability of online learning algorithms for pairwise loss functions",
abstract = "In this paper, we study the generalization properties of online learning based stochastic methods for supervised learning problems where the loss function is dependent on more than one training sample (e.g., metric learning, ranking). We present a generic decoupling technique that enables us to provide Rademacher complexity-based generalization error bounds. Our bounds are in general tighter than those obtained by Wang et al. (2012) for the same problem. Using our decoupling technique, we are further able to obtain fast convergence rates for strongly convex pairwise loss functions. We are also able to analyze a class of memory efficient online learning algorithms for pairwise learning problems that use only a bounded subset of past training samples to update the hypothesis at each step. Finally, in order to complement our generalization bounds, we propose a novel memory efficient online learning algorithm for higher order learning problems with bounded regret guarantees.",
author = "Purushottam Kar and Sriperumbudur, {Bharath Kumar} and Prateek Jain and Karnick, {Harish C.}",
year = "2013",
month = "1",
day = "1",
language = "English (US)",
pages = "1478--1486",
note = "30th International Conference on Machine Learning, ICML 2013 ; Conference date: 16-06-2013 Through 21-06-2013",

}

Kar, P, Sriperumbudur, BK, Jain, P & Karnick, HC 2013, 'On the generalization ability of online learning algorithms for pairwise loss functions', Paper presented at 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, United States, 6/16/13 - 6/21/13 pp. 1478-1486.

On the generalization ability of online learning algorithms for pairwise loss functions. / Kar, Purushottam; Sriperumbudur, Bharath Kumar; Jain, Prateek; Karnick, Harish C.

2013. 1478-1486 Paper presented at 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, United States.

Research output: Contribution to conferencePaper

TY - CONF

T1 - On the generalization ability of online learning algorithms for pairwise loss functions

AU - Kar, Purushottam

AU - Sriperumbudur, Bharath Kumar

AU - Jain, Prateek

AU - Karnick, Harish C.

PY - 2013/1/1

Y1 - 2013/1/1

N2 - In this paper, we study the generalization properties of online learning based stochastic methods for supervised learning problems where the loss function is dependent on more than one training sample (e.g., metric learning, ranking). We present a generic decoupling technique that enables us to provide Rademacher complexity-based generalization error bounds. Our bounds are in general tighter than those obtained by Wang et al. (2012) for the same problem. Using our decoupling technique, we are further able to obtain fast convergence rates for strongly convex pairwise loss functions. We are also able to analyze a class of memory efficient online learning algorithms for pairwise learning problems that use only a bounded subset of past training samples to update the hypothesis at each step. Finally, in order to complement our generalization bounds, we propose a novel memory efficient online learning algorithm for higher order learning problems with bounded regret guarantees.

AB - In this paper, we study the generalization properties of online learning based stochastic methods for supervised learning problems where the loss function is dependent on more than one training sample (e.g., metric learning, ranking). We present a generic decoupling technique that enables us to provide Rademacher complexity-based generalization error bounds. Our bounds are in general tighter than those obtained by Wang et al. (2012) for the same problem. Using our decoupling technique, we are further able to obtain fast convergence rates for strongly convex pairwise loss functions. We are also able to analyze a class of memory efficient online learning algorithms for pairwise learning problems that use only a bounded subset of past training samples to update the hypothesis at each step. Finally, in order to complement our generalization bounds, we propose a novel memory efficient online learning algorithm for higher order learning problems with bounded regret guarantees.

UR - http://www.scopus.com/inward/record.url?scp=84897547320&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84897547320&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:84897547320

SP - 1478

EP - 1486

ER -

Kar P, Sriperumbudur BK, Jain P, Karnick HC. On the generalization ability of online learning algorithms for pairwise loss functions. 2013. Paper presented at 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, United States.