Multiple label prediction for image annotation with multiple kernel correlation models

Oksana Yakhnenko, Vasant Honavar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Citations (Scopus)

Abstract

Image annotation is a challenging task that allows to correlate text keywords with an image. In this paper we address the problem of image annotation using Kernel Multiple Linear Regression model. Multiple Linear Regression (MLR) model reconstructs image caption from an image by performing a linear transformation of an image into some semantic space, and then recovers the caption by performing another linear transformation from the semantic space into the label space. The model is trained so that model parameters minimize the error of reconstruction directly. This model is related to Canonical Correlation Analysis (CCA) which maps both images and caption into the semantic space to minimize the distance of mapping in the semantic space. Kernel trick is then used for the MLR resulting in Kernel Multiple Linear Regression model. The solution to KMLR is a solution to the generalized eigen-value problem, related to KCCA (Kernel Canonical Correlation Analysis). We then extend Kernel Multiple Linear Regression and Kernel Canonical Correlation analysis models to multiple kernel setting, to allow various representations of images and captions. We present results for image annotation using Multiple Kernel Learning CCA and MLR on Oliva and Torralba [21] scene recognition that show kernel selection behaviour.

Original languageEnglish (US)
Title of host publication2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009
Pages8-15
Number of pages8
DOIs
StatePublished - Nov 20 2009
Event2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009 - Miami, FL, United States
Duration: Jun 20 2009Jun 25 2009

Publication series

Name2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009

Other

Other2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009
CountryUnited States
CityMiami, FL
Period6/20/096/25/09

Fingerprint

Labels
Linear regression
Semantics
Linear transformations

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition
  • Biomedical Engineering

Cite this

Yakhnenko, O., & Honavar, V. (2009). Multiple label prediction for image annotation with multiple kernel correlation models. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009 (pp. 8-15). [5204274] (2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009). https://doi.org/10.1109/CVPR.2009.5204274
Yakhnenko, Oksana ; Honavar, Vasant. / Multiple label prediction for image annotation with multiple kernel correlation models. 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009. 2009. pp. 8-15 (2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009).
@inproceedings{297af244c89a4ba0a6972a9f0d78e83b,
title = "Multiple label prediction for image annotation with multiple kernel correlation models",
abstract = "Image annotation is a challenging task that allows to correlate text keywords with an image. In this paper we address the problem of image annotation using Kernel Multiple Linear Regression model. Multiple Linear Regression (MLR) model reconstructs image caption from an image by performing a linear transformation of an image into some semantic space, and then recovers the caption by performing another linear transformation from the semantic space into the label space. The model is trained so that model parameters minimize the error of reconstruction directly. This model is related to Canonical Correlation Analysis (CCA) which maps both images and caption into the semantic space to minimize the distance of mapping in the semantic space. Kernel trick is then used for the MLR resulting in Kernel Multiple Linear Regression model. The solution to KMLR is a solution to the generalized eigen-value problem, related to KCCA (Kernel Canonical Correlation Analysis). We then extend Kernel Multiple Linear Regression and Kernel Canonical Correlation analysis models to multiple kernel setting, to allow various representations of images and captions. We present results for image annotation using Multiple Kernel Learning CCA and MLR on Oliva and Torralba [21] scene recognition that show kernel selection behaviour.",
author = "Oksana Yakhnenko and Vasant Honavar",
year = "2009",
month = "11",
day = "20",
doi = "10.1109/CVPR.2009.5204274",
language = "English (US)",
isbn = "9781424439911",
series = "2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009",
pages = "8--15",
booktitle = "2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009",

}

Yakhnenko, O & Honavar, V 2009, Multiple label prediction for image annotation with multiple kernel correlation models. in 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009., 5204274, 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 8-15, 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, Miami, FL, United States, 6/20/09. https://doi.org/10.1109/CVPR.2009.5204274

Multiple label prediction for image annotation with multiple kernel correlation models. / Yakhnenko, Oksana; Honavar, Vasant.

2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009. 2009. p. 8-15 5204274 (2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Multiple label prediction for image annotation with multiple kernel correlation models

AU - Yakhnenko, Oksana

AU - Honavar, Vasant

PY - 2009/11/20

Y1 - 2009/11/20

N2 - Image annotation is a challenging task that allows to correlate text keywords with an image. In this paper we address the problem of image annotation using Kernel Multiple Linear Regression model. Multiple Linear Regression (MLR) model reconstructs image caption from an image by performing a linear transformation of an image into some semantic space, and then recovers the caption by performing another linear transformation from the semantic space into the label space. The model is trained so that model parameters minimize the error of reconstruction directly. This model is related to Canonical Correlation Analysis (CCA) which maps both images and caption into the semantic space to minimize the distance of mapping in the semantic space. Kernel trick is then used for the MLR resulting in Kernel Multiple Linear Regression model. The solution to KMLR is a solution to the generalized eigen-value problem, related to KCCA (Kernel Canonical Correlation Analysis). We then extend Kernel Multiple Linear Regression and Kernel Canonical Correlation analysis models to multiple kernel setting, to allow various representations of images and captions. We present results for image annotation using Multiple Kernel Learning CCA and MLR on Oliva and Torralba [21] scene recognition that show kernel selection behaviour.

AB - Image annotation is a challenging task that allows to correlate text keywords with an image. In this paper we address the problem of image annotation using Kernel Multiple Linear Regression model. Multiple Linear Regression (MLR) model reconstructs image caption from an image by performing a linear transformation of an image into some semantic space, and then recovers the caption by performing another linear transformation from the semantic space into the label space. The model is trained so that model parameters minimize the error of reconstruction directly. This model is related to Canonical Correlation Analysis (CCA) which maps both images and caption into the semantic space to minimize the distance of mapping in the semantic space. Kernel trick is then used for the MLR resulting in Kernel Multiple Linear Regression model. The solution to KMLR is a solution to the generalized eigen-value problem, related to KCCA (Kernel Canonical Correlation Analysis). We then extend Kernel Multiple Linear Regression and Kernel Canonical Correlation analysis models to multiple kernel setting, to allow various representations of images and captions. We present results for image annotation using Multiple Kernel Learning CCA and MLR on Oliva and Torralba [21] scene recognition that show kernel selection behaviour.

UR - http://www.scopus.com/inward/record.url?scp=70449586741&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70449586741&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2009.5204274

DO - 10.1109/CVPR.2009.5204274

M3 - Conference contribution

SN - 9781424439911

T3 - 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009

SP - 8

EP - 15

BT - 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009

ER -

Yakhnenko O, Honavar V. Multiple label prediction for image annotation with multiple kernel correlation models. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009. 2009. p. 8-15. 5204274. (2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009). https://doi.org/10.1109/CVPR.2009.5204274