Improving Neural Network Robustness Through Neighborhood Preserving Layers

Bingyuan Liu, Christopher Malon, Lingzhou Xue, Erik Kruus

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

One major source of vulnerability of neural nets in classification tasks is from overparameterized fully connected layers near the end of the network. In this paper, we propose a new neighborhood preserving layer which can replace these fully connected layers to improve the network robustness. Networks including these neighborhood preserving layers can be trained efficiently. We theoretically prove that our proposed layers are more robust against distortion because they effectively control the magnitude of gradients. Finally, we empirically show that networks with our proposed layers are more robust against state-of-the-art gradient descent based attacks, such as a PGD attack on the benchmark image classification datasets MNIST and CIFAR10.

Original languageEnglish (US)
Title of host publicationPattern Recognition. ICPR International Workshops and Challenges, 2021, Proceedings
EditorsAlberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, Roberto Vezzani
PublisherSpringer Science and Business Media Deutschland GmbH
Pages179-195
Number of pages17
ISBN (Print)9783030687793
DOIs
StatePublished - 2021
Event25th International Conference on Pattern Recognition Workshops, ICPR 2020 - Virtual, Online
Duration: Jan 10 2021Jan 15 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12666 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference25th International Conference on Pattern Recognition Workshops, ICPR 2020
CityVirtual, Online
Period1/10/211/15/21

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Improving Neural Network Robustness Through Neighborhood Preserving Layers'. Together they form a unique fingerprint.

Cite this