Assessing tracking performance in complex scenarios using mean time between failures

Peter Carr, Robert Collins

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Existing measures for evaluating the performance of tracking algorithms are difficult to interpret, which makes it hard to identify the best approach for a particular situation. As we show, a dummy algorithm which does not actually track scores well under most existing measures. Although some measures characterize specific error sources quite well, combining them into a single aggregate measure for comparing approaches or tuning parameters is not straightforward. In this work we propose 'mean time between failures' as a viable summary of solution quality - especially when the goal is to follow objects for as long as possible. In addition to being sensitive to all tracking errors, the performance numbers are directly interpretable: how long can an algorithm operate before a mistake has likely occurred (the object is lost, its identity is confused, etc.)? We illustrate the merits of this measure by assessing solutions from different algorithms on a challenging dataset.

Original languageEnglish (US)
Title of host publication2016 IEEE Winter Conference on Applications of Computer Vision, WACV 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509006410
DOIs
StatePublished - May 23 2016
EventIEEE Winter Conference on Applications of Computer Vision, WACV 2016 - Lake Placid, United States
Duration: Mar 7 2016Mar 10 2016

Publication series

Name2016 IEEE Winter Conference on Applications of Computer Vision, WACV 2016

Other

OtherIEEE Winter Conference on Applications of Computer Vision, WACV 2016
CountryUnited States
CityLake Placid
Period3/7/163/10/16

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this

Carr, P., & Collins, R. (2016). Assessing tracking performance in complex scenarios using mean time between failures. In 2016 IEEE Winter Conference on Applications of Computer Vision, WACV 2016 [7477617] (2016 IEEE Winter Conference on Applications of Computer Vision, WACV 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV.2016.7477617