Abstract
Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic approach to meta-learning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as black-boxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this meta-learning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.
| Original language | English (US) |
|---|---|
| Title of host publication | Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07 |
| Pages | 393-402 |
| Number of pages | 10 |
| DOIs | |
| State | Published - Dec 1 2007 |
| Event | 15th ACM International Conference on Multimedia, MM'07 - Augsburg, Bavaria, Germany Duration: Sep 24 2007 → Sep 29 2007 |
Other
| Other | 15th ACM International Conference on Multimedia, MM'07 |
|---|---|
| Country | Germany |
| City | Augsburg, Bavaria |
| Period | 9/24/07 → 9/29/07 |
Fingerprint
All Science Journal Classification (ASJC) codes
- Computer Science(all)
Cite this
}
Tagging over time : Real-world image annotation by lightweight meta-learning. / Datta, Ritendra; Joshi, Dhiraj; Li, Jia; Wang, James.
Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07. 2007. p. 393-402.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
TY - GEN
T1 - Tagging over time
T2 - Real-world image annotation by lightweight meta-learning
AU - Datta, Ritendra
AU - Joshi, Dhiraj
AU - Li, Jia
AU - Wang, James
PY - 2007/12/1
Y1 - 2007/12/1
N2 - Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic approach to meta-learning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as black-boxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this meta-learning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.
AB - Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic approach to meta-learning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as black-boxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this meta-learning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.
UR - http://www.scopus.com/inward/record.url?scp=37849021285&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=37849021285&partnerID=8YFLogxK
U2 - 10.1145/1291233.1291328
DO - 10.1145/1291233.1291328
M3 - Conference contribution
AN - SCOPUS:37849021285
SN - 9781595937025
SP - 393
EP - 402
BT - Proceedings of the Fifteenth ACM International Conference on Multimedia, MM'07
ER -