Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks

Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, Ananthram Swami

Research output: Chapter in Book/Report/Conference proceedingConference contribution

707 Scopus citations

Abstract

Deep learning algorithms have been shown to perform extremely well on manyclassical machine learning problems. However, recent studies have shown thatdeep learning, like other machine learning techniques, is vulnerable to adversarial samples: inputs crafted to force adeep neural network (DNN) to provide adversary-selected outputs. Such attackscan seriously undermine the security of the system supported by the DNN, sometimes with devastating consequences. For example, autonomous vehicles canbe crashed, illicit or illegal content can bypass content filters, or biometricauthentication systems can be manipulated to allow improper access. In thiswork, we introduce a defensive mechanism called defensive distillationto reduce the effectiveness of adversarial samples on DNNs. We analyticallyinvestigate the generalizability and robustness properties granted by the useof defensive distillation when training DNNs. We also empirically study theeffectiveness of our defense mechanisms on two DNNs placed in adversarialsettings. The study shows that defensive distillation can reduce effectivenessof sample creation from 95% to less than 0.5% on a studied DNN. Such dramaticgains can be explained by the fact that distillation leads gradients used inadversarial sample creation to be reduced by a factor of 1030. We alsofind that distillation increases the average minimum number of features thatneed to be modified to create adversarial samples by about 800% on one of theDNNs we tested.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE Symposium on Security and Privacy, SP 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages582-597
Number of pages16
ISBN (Electronic)9781509008247
DOIs
StatePublished - Aug 16 2016
Event2016 IEEE Symposium on Security and Privacy, SP 2016 - San Jose, United States
Duration: May 23 2016May 25 2016

Publication series

NameProceedings - 2016 IEEE Symposium on Security and Privacy, SP 2016

Other

Other2016 IEEE Symposium on Security and Privacy, SP 2016
CountryUnited States
CitySan Jose
Period5/23/165/25/16

All Science Journal Classification (ASJC) codes

  • Safety, Risk, Reliability and Quality
  • Computer Networks and Communications
  • Software

Fingerprint Dive into the research topics of 'Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks'. Together they form a unique fingerprint.

Cite this