Prediction in the Presence of Response-Dependent Missing Labels

Hyebin Song, Garvesh Raskutti, Rebecca Willett

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In various settings, limitations of sensing technologies or other sampling mechanisms result in missing labels, where the likelihood of a missing label is an unknown function of the data. For example, satellites used to detect forest fires cannot sense fires below a certain size threshold. In such cases, training datasets consist of positive and pseudo-negative observations (true negatives or undetected positives with small magnitudes). We develop a new methodology and non-convex algorithm which jointly estimates the magnitude and occurrence of events, utilizing prior knowledge of the detection mechanism. We provide conditions under which our model is identifiable. We prove that even though our approach leads to a non-convex objective, any local minimizer has an optimal statistical error (up to a log term) and the projected gradient descent algorithm has geometric convergence rates. We demonstrate on both synthetic data and a California wildfire dataset that our method outperforms existing state-of-the-art approaches.

Original languageEnglish (US)
Title of host publication2021 IEEE Statistical Signal Processing Workshop, SSP 2021
PublisherIEEE Computer Society
Pages451-455
Number of pages5
ISBN (Electronic)9781728157672
DOIs
StatePublished - Jul 11 2021
Event21st IEEE Statistical Signal Processing Workshop, SSP 2021 - Virtual, Rio de Janeiro, Brazil
Duration: Jul 11 2021Jul 14 2021

Publication series

NameIEEE Workshop on Statistical Signal Processing Proceedings
Volume2021-July

Conference

Conference21st IEEE Statistical Signal Processing Workshop, SSP 2021
Country/TerritoryBrazil
CityVirtual, Rio de Janeiro
Period7/11/217/14/21

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • Applied Mathematics
  • Signal Processing
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Prediction in the Presence of Response-Dependent Missing Labels'. Together they form a unique fingerprint.

Cite this