Statistical debugging for real-world performance problems

Linhai Song, Shan Lu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

32 Scopus citations

Abstract

Design and implementation defects that lead to inefficient computation widely exist in software. These defects are difficult to avoid and discover. They lead to severe performance degradation and energy waste during production runs, and are becoming increasingly critical with the meager increase of single-core hardware performance and the increasing concerns about energy constraints. Effective tools that diagnose performance problems and point out the inefficiency root cause are sorely needed.

The state of the art of performance diagnosis is preliminary. Profiling can identify the functions that consume the most computation resources, but can neither identify the ones that waste the most resources nor explain why. Performance-bug detectors can identify specific type of inefficient computation, but are not suited for diagnosing general performance problems. Effective failure diagnosis techniques, such as statistical debugging, have been proposed for functional bugs. However, whether they work for performance problems is still an open question.

In this paper, we first conduct an empirical study to understand how performance problems are observed and reported by real-world users. Our study shows that statistical debugging is a natural fit for diagnosing performance problems, which are often observed through comparison-based approaches and reported together with both good and bad inputs. We then thoroughly investigate different design points in statistical debugging, including three different predicates and two different types of statistical models, to understand which design point works the best for performance diagnosis. Finally, we study how some unique nature of performance bugs allows sampling techniques to lower the overhead of run-time performance diagnosis without extending the diagnosis latency.

Original languageEnglish (US)
Title of host publicationProceedings of the Conference on Object-Oriented Programming Systems, Languages, and Applications, OOPSLA
PublisherAssociation for Computing Machinery
Pages561-578
Number of pages18
ISBN (Electronic)9781450325851
DOIs
StatePublished - Oct 15 2014
Event2014 ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2014 - Portland, United States
Duration: Oct 20 2014Oct 24 2014

Publication series

NameProceedings of the Conference on Object-Oriented Programming Systems, Languages, and Applications, OOPSLA

Other

Other2014 ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2014
CountryUnited States
CityPortland
Period10/20/1410/24/14

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Software

Cite this

Song, L., & Lu, S. (2014). Statistical debugging for real-world performance problems. In Proceedings of the Conference on Object-Oriented Programming Systems, Languages, and Applications, OOPSLA (pp. 561-578). (Proceedings of the Conference on Object-Oriented Programming Systems, Languages, and Applications, OOPSLA). Association for Computing Machinery. https://doi.org/10.1145/2660193.2660234