Max-Information, Differential Privacy, and Post-selection Hypothesis Testing

Ryan Rogers, Aaron Roth, Adam Davison Smith, Om Thakkar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

In this paper, we initiate a principled study of how the generalization properties of approximate differential privacy can be used to perform adaptive hypothesis testing, while giving statistically valid p-value corrections. We do this by observing that the guarantees of algorithms with bounded approximate max-information are sufficient to correct the p-values of adaptively chosen hypotheses, and then by proving that algorithms that satisfy (ϵ,δ)-differential privacy have bounded approximate max information when their inputs are drawn from a product distribution. This substantially extends the known connection between differential privacy and max-information, which previously was only known to hold for (pure) (ϵ,0)-differential privacy. It also extends our understanding of max-information as a partially unifying measure controlling the generalization properties of adaptive data analyses. We also show a lower bound, proving that (despite the strong composition properties of max-information), when data is drawn from a product distribution, (ϵ,δ)-differentially private algorithms can come first in a composition with other algorithms satisfying max-information bounds, but not necessarily second if the composition is required to itself satisfy a nontrivial max-information bound. This, in particular, implies that the connection between (ϵ,δ)-differential privacy and max-information holds only for inputs drawn from product distributions, unlike the connection between (ϵ,0)-differential privacy and max-information.

Original languageEnglish (US)
Title of host publicationProceedings - 57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016
PublisherIEEE Computer Society
Pages487-494
Number of pages8
ISBN (Electronic)9781509039333
DOIs
StatePublished - Dec 14 2016
Event57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016 - New Brunswick, United States
Duration: Oct 9 2016Oct 11 2016

Publication series

NameProceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS
Volume2016-December
ISSN (Print)0272-5428

Other

Other57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016
CountryUnited States
CityNew Brunswick
Period10/9/1610/11/16

Fingerprint

Testing
Chemical analysis

All Science Journal Classification (ASJC) codes

  • Computer Science(all)

Cite this

Rogers, R., Roth, A., Smith, A. D., & Thakkar, O. (2016). Max-Information, Differential Privacy, and Post-selection Hypothesis Testing. In Proceedings - 57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016 (pp. 487-494). [7782963] (Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS; Vol. 2016-December). IEEE Computer Society. https://doi.org/10.1109/FOCS.2016.59
Rogers, Ryan ; Roth, Aaron ; Smith, Adam Davison ; Thakkar, Om. / Max-Information, Differential Privacy, and Post-selection Hypothesis Testing. Proceedings - 57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016. IEEE Computer Society, 2016. pp. 487-494 (Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS).
@inproceedings{ec471d5c16244bf8b0cab8f9bf3d58eb,
title = "Max-Information, Differential Privacy, and Post-selection Hypothesis Testing",
abstract = "In this paper, we initiate a principled study of how the generalization properties of approximate differential privacy can be used to perform adaptive hypothesis testing, while giving statistically valid p-value corrections. We do this by observing that the guarantees of algorithms with bounded approximate max-information are sufficient to correct the p-values of adaptively chosen hypotheses, and then by proving that algorithms that satisfy (ϵ,δ)-differential privacy have bounded approximate max information when their inputs are drawn from a product distribution. This substantially extends the known connection between differential privacy and max-information, which previously was only known to hold for (pure) (ϵ,0)-differential privacy. It also extends our understanding of max-information as a partially unifying measure controlling the generalization properties of adaptive data analyses. We also show a lower bound, proving that (despite the strong composition properties of max-information), when data is drawn from a product distribution, (ϵ,δ)-differentially private algorithms can come first in a composition with other algorithms satisfying max-information bounds, but not necessarily second if the composition is required to itself satisfy a nontrivial max-information bound. This, in particular, implies that the connection between (ϵ,δ)-differential privacy and max-information holds only for inputs drawn from product distributions, unlike the connection between (ϵ,0)-differential privacy and max-information.",
author = "Ryan Rogers and Aaron Roth and Smith, {Adam Davison} and Om Thakkar",
year = "2016",
month = "12",
day = "14",
doi = "10.1109/FOCS.2016.59",
language = "English (US)",
series = "Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS",
publisher = "IEEE Computer Society",
pages = "487--494",
booktitle = "Proceedings - 57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016",
address = "United States",

}

Rogers, R, Roth, A, Smith, AD & Thakkar, O 2016, Max-Information, Differential Privacy, and Post-selection Hypothesis Testing. in Proceedings - 57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016., 7782963, Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS, vol. 2016-December, IEEE Computer Society, pp. 487-494, 57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016, New Brunswick, United States, 10/9/16. https://doi.org/10.1109/FOCS.2016.59

Max-Information, Differential Privacy, and Post-selection Hypothesis Testing. / Rogers, Ryan; Roth, Aaron; Smith, Adam Davison; Thakkar, Om.

Proceedings - 57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016. IEEE Computer Society, 2016. p. 487-494 7782963 (Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS; Vol. 2016-December).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Max-Information, Differential Privacy, and Post-selection Hypothesis Testing

AU - Rogers, Ryan

AU - Roth, Aaron

AU - Smith, Adam Davison

AU - Thakkar, Om

PY - 2016/12/14

Y1 - 2016/12/14

N2 - In this paper, we initiate a principled study of how the generalization properties of approximate differential privacy can be used to perform adaptive hypothesis testing, while giving statistically valid p-value corrections. We do this by observing that the guarantees of algorithms with bounded approximate max-information are sufficient to correct the p-values of adaptively chosen hypotheses, and then by proving that algorithms that satisfy (ϵ,δ)-differential privacy have bounded approximate max information when their inputs are drawn from a product distribution. This substantially extends the known connection between differential privacy and max-information, which previously was only known to hold for (pure) (ϵ,0)-differential privacy. It also extends our understanding of max-information as a partially unifying measure controlling the generalization properties of adaptive data analyses. We also show a lower bound, proving that (despite the strong composition properties of max-information), when data is drawn from a product distribution, (ϵ,δ)-differentially private algorithms can come first in a composition with other algorithms satisfying max-information bounds, but not necessarily second if the composition is required to itself satisfy a nontrivial max-information bound. This, in particular, implies that the connection between (ϵ,δ)-differential privacy and max-information holds only for inputs drawn from product distributions, unlike the connection between (ϵ,0)-differential privacy and max-information.

AB - In this paper, we initiate a principled study of how the generalization properties of approximate differential privacy can be used to perform adaptive hypothesis testing, while giving statistically valid p-value corrections. We do this by observing that the guarantees of algorithms with bounded approximate max-information are sufficient to correct the p-values of adaptively chosen hypotheses, and then by proving that algorithms that satisfy (ϵ,δ)-differential privacy have bounded approximate max information when their inputs are drawn from a product distribution. This substantially extends the known connection between differential privacy and max-information, which previously was only known to hold for (pure) (ϵ,0)-differential privacy. It also extends our understanding of max-information as a partially unifying measure controlling the generalization properties of adaptive data analyses. We also show a lower bound, proving that (despite the strong composition properties of max-information), when data is drawn from a product distribution, (ϵ,δ)-differentially private algorithms can come first in a composition with other algorithms satisfying max-information bounds, but not necessarily second if the composition is required to itself satisfy a nontrivial max-information bound. This, in particular, implies that the connection between (ϵ,δ)-differential privacy and max-information holds only for inputs drawn from product distributions, unlike the connection between (ϵ,0)-differential privacy and max-information.

UR - http://www.scopus.com/inward/record.url?scp=85009347998&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85009347998&partnerID=8YFLogxK

U2 - 10.1109/FOCS.2016.59

DO - 10.1109/FOCS.2016.59

M3 - Conference contribution

AN - SCOPUS:85009347998

T3 - Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS

SP - 487

EP - 494

BT - Proceedings - 57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016

PB - IEEE Computer Society

ER -

Rogers R, Roth A, Smith AD, Thakkar O. Max-Information, Differential Privacy, and Post-selection Hypothesis Testing. In Proceedings - 57th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2016. IEEE Computer Society. 2016. p. 487-494. 7782963. (Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS). https://doi.org/10.1109/FOCS.2016.59