Detecting violations of differential privacy

Zeyu Ding, Yuxin Wang, Guanhong Wang, Danfeng Zhang, Daniel Kifer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

The widespread acceptance of differential privacy has led to the publication of many sophisticated algorithms for protecting privacy. However, due to the subtle nature of this privacy definition, many such algorithms have bugs that make them violate their claimed privacy. In this paper, we consider the problem of producing counterexamples for such incorrect algorithms. The counterexamples are designed to be short and human-understandable so that the counterexample generator can be used in the development process - a developer could quickly explore variations of an algorithm and investigate where they break down. Our approach is statistical in nature. It runs a candidate algorithm many times and uses statistical tests to try to detect violations of differential privacy. An evaluation on a variety of incorrect published algorithms validates the usefulness of our approach: it correctly rejects incorrect algorithms and provides counterexamples for them within a few seconds.

Original languageEnglish (US)
Title of host publicationCCS 2018 - Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security
PublisherAssociation for Computing Machinery
Pages475-489
Number of pages15
ISBN (Electronic)9781450356930
DOIs
StatePublished - Oct 15 2018
Event25th ACM Conference on Computer and Communications Security, CCS 2018 - Toronto, Canada
Duration: Oct 15 2018 → …

Publication series

NameProceedings of the ACM Conference on Computer and Communications Security
ISSN (Print)1543-7221

Other

Other25th ACM Conference on Computer and Communications Security, CCS 2018
CountryCanada
CityToronto
Period10/15/18 → …

Fingerprint

Statistical tests

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Networks and Communications

Cite this

Ding, Z., Wang, Y., Wang, G., Zhang, D., & Kifer, D. (2018). Detecting violations of differential privacy. In CCS 2018 - Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (pp. 475-489). (Proceedings of the ACM Conference on Computer and Communications Security). Association for Computing Machinery. https://doi.org/10.1145/3243734.3243818
Ding, Zeyu ; Wang, Yuxin ; Wang, Guanhong ; Zhang, Danfeng ; Kifer, Daniel. / Detecting violations of differential privacy. CCS 2018 - Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. Association for Computing Machinery, 2018. pp. 475-489 (Proceedings of the ACM Conference on Computer and Communications Security).
@inproceedings{7618b9ce1dda4cd9ab0a6cc06d7ce0c0,
title = "Detecting violations of differential privacy",
abstract = "The widespread acceptance of differential privacy has led to the publication of many sophisticated algorithms for protecting privacy. However, due to the subtle nature of this privacy definition, many such algorithms have bugs that make them violate their claimed privacy. In this paper, we consider the problem of producing counterexamples for such incorrect algorithms. The counterexamples are designed to be short and human-understandable so that the counterexample generator can be used in the development process - a developer could quickly explore variations of an algorithm and investigate where they break down. Our approach is statistical in nature. It runs a candidate algorithm many times and uses statistical tests to try to detect violations of differential privacy. An evaluation on a variety of incorrect published algorithms validates the usefulness of our approach: it correctly rejects incorrect algorithms and provides counterexamples for them within a few seconds.",
author = "Zeyu Ding and Yuxin Wang and Guanhong Wang and Danfeng Zhang and Daniel Kifer",
year = "2018",
month = "10",
day = "15",
doi = "10.1145/3243734.3243818",
language = "English (US)",
series = "Proceedings of the ACM Conference on Computer and Communications Security",
publisher = "Association for Computing Machinery",
pages = "475--489",
booktitle = "CCS 2018 - Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security",

}

Ding, Z, Wang, Y, Wang, G, Zhang, D & Kifer, D 2018, Detecting violations of differential privacy. in CCS 2018 - Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. Proceedings of the ACM Conference on Computer and Communications Security, Association for Computing Machinery, pp. 475-489, 25th ACM Conference on Computer and Communications Security, CCS 2018, Toronto, Canada, 10/15/18. https://doi.org/10.1145/3243734.3243818

Detecting violations of differential privacy. / Ding, Zeyu; Wang, Yuxin; Wang, Guanhong; Zhang, Danfeng; Kifer, Daniel.

CCS 2018 - Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. Association for Computing Machinery, 2018. p. 475-489 (Proceedings of the ACM Conference on Computer and Communications Security).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Detecting violations of differential privacy

AU - Ding, Zeyu

AU - Wang, Yuxin

AU - Wang, Guanhong

AU - Zhang, Danfeng

AU - Kifer, Daniel

PY - 2018/10/15

Y1 - 2018/10/15

N2 - The widespread acceptance of differential privacy has led to the publication of many sophisticated algorithms for protecting privacy. However, due to the subtle nature of this privacy definition, many such algorithms have bugs that make them violate their claimed privacy. In this paper, we consider the problem of producing counterexamples for such incorrect algorithms. The counterexamples are designed to be short and human-understandable so that the counterexample generator can be used in the development process - a developer could quickly explore variations of an algorithm and investigate where they break down. Our approach is statistical in nature. It runs a candidate algorithm many times and uses statistical tests to try to detect violations of differential privacy. An evaluation on a variety of incorrect published algorithms validates the usefulness of our approach: it correctly rejects incorrect algorithms and provides counterexamples for them within a few seconds.

AB - The widespread acceptance of differential privacy has led to the publication of many sophisticated algorithms for protecting privacy. However, due to the subtle nature of this privacy definition, many such algorithms have bugs that make them violate their claimed privacy. In this paper, we consider the problem of producing counterexamples for such incorrect algorithms. The counterexamples are designed to be short and human-understandable so that the counterexample generator can be used in the development process - a developer could quickly explore variations of an algorithm and investigate where they break down. Our approach is statistical in nature. It runs a candidate algorithm many times and uses statistical tests to try to detect violations of differential privacy. An evaluation on a variety of incorrect published algorithms validates the usefulness of our approach: it correctly rejects incorrect algorithms and provides counterexamples for them within a few seconds.

UR - http://www.scopus.com/inward/record.url?scp=85056847220&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056847220&partnerID=8YFLogxK

U2 - 10.1145/3243734.3243818

DO - 10.1145/3243734.3243818

M3 - Conference contribution

AN - SCOPUS:85056847220

T3 - Proceedings of the ACM Conference on Computer and Communications Security

SP - 475

EP - 489

BT - CCS 2018 - Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security

PB - Association for Computing Machinery

ER -

Ding Z, Wang Y, Wang G, Zhang D, Kifer D. Detecting violations of differential privacy. In CCS 2018 - Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. Association for Computing Machinery. 2018. p. 475-489. (Proceedings of the ACM Conference on Computer and Communications Security). https://doi.org/10.1145/3243734.3243818