Tagging over time: Real-world image annotation by lightweight meta-learning

Ritendra Datta, Dhiraj Joshi, Jia Li, James Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Scopus citations


Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic approach to meta-learning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as black-boxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this meta-learning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.

Original languageEnglish (US)
Title of host publicationProceedings of the Fifteenth ACM International Conference on Multimedia, MM'07
Number of pages10
StatePublished - Dec 1 2007
Event15th ACM International Conference on Multimedia, MM'07 - Augsburg, Bavaria, Germany
Duration: Sep 24 2007Sep 29 2007


Other15th ACM International Conference on Multimedia, MM'07
CityAugsburg, Bavaria

All Science Journal Classification (ASJC) codes

  • Computer Science(all)


Dive into the research topics of 'Tagging over time: Real-world image annotation by lightweight meta-learning'. Together they form a unique fingerprint.

Cite this