Tagging over time: Real-world image annotation by lightweight meta-learning

Ritendra Datta, Dhiraj Joshi, Jia Li, James Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Citations (Scopus)

Abstract

Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic approach to meta-learning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as black-boxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this meta-learning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.

Original languageEnglish (US)
Title of host publicationProceedings of the Fifteenth ACM International Conference on Multimedia, MM'07
Pages393-402
Number of pages10
DOIs
StatePublished - Dec 1 2007
Event15th ACM International Conference on Multimedia, MM'07 - Augsburg, Bavaria, Germany
Duration: Sep 24 2007Sep 29 2007

Other

Other15th ACM International Conference on Multimedia, MM'07
CountryGermany
CityAugsburg, Bavaria
Period9/24/079/29/07

Fingerprint

Online systems
Ontology
Feedback
Experiments

All Science Journal Classification (ASJC) codes

  • Computer Science(all)

Cite this

Datta, R., Joshi, D., Li, J., & Wang, J. (2007). Tagging over time: Real-world image annotation by lightweight meta-learning. In Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07 (pp. 393-402) https://doi.org/10.1145/1291233.1291328
Datta, Ritendra ; Joshi, Dhiraj ; Li, Jia ; Wang, James. / Tagging over time : Real-world image annotation by lightweight meta-learning. Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07. 2007. pp. 393-402
@inproceedings{46c508a7ae6d4512a6502dbe90f3d479,
title = "Tagging over time: Real-world image annotation by lightweight meta-learning",
abstract = "Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic approach to meta-learning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as black-boxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this meta-learning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.",
author = "Ritendra Datta and Dhiraj Joshi and Jia Li and James Wang",
year = "2007",
month = "12",
day = "1",
doi = "10.1145/1291233.1291328",
language = "English (US)",
isbn = "9781595937025",
pages = "393--402",
booktitle = "Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07",

}

Datta, R, Joshi, D, Li, J & Wang, J 2007, Tagging over time: Real-world image annotation by lightweight meta-learning. in Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07. pp. 393-402, 15th ACM International Conference on Multimedia, MM'07, Augsburg, Bavaria, Germany, 9/24/07. https://doi.org/10.1145/1291233.1291328

Tagging over time : Real-world image annotation by lightweight meta-learning. / Datta, Ritendra; Joshi, Dhiraj; Li, Jia; Wang, James.

Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07. 2007. p. 393-402.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Tagging over time

T2 - Real-world image annotation by lightweight meta-learning

AU - Datta, Ritendra

AU - Joshi, Dhiraj

AU - Li, Jia

AU - Wang, James

PY - 2007/12/1

Y1 - 2007/12/1

N2 - Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic approach to meta-learning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as black-boxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this meta-learning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.

AB - Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic approach to meta-learning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as black-boxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this meta-learning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.

UR - http://www.scopus.com/inward/record.url?scp=37849021285&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=37849021285&partnerID=8YFLogxK

U2 - 10.1145/1291233.1291328

DO - 10.1145/1291233.1291328

M3 - Conference contribution

AN - SCOPUS:37849021285

SN - 9781595937025

SP - 393

EP - 402

BT - Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07

ER -

Datta R, Joshi D, Li J, Wang J. Tagging over time: Real-world image annotation by lightweight meta-learning. In Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07. 2007. p. 393-402 https://doi.org/10.1145/1291233.1291328