Empirical analysis of the expected source values rule

Richard Spartz, Vasant Honavar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Despite its notoriously slow learning time, back-propagation (BP) is one of the most widely used neural network training algorithms. Two major reasons for this slow convergence are the step size problem and the flat spot problem [Fahlman, 1988]. In [Samad, 1991] a simple modification, the expected source values (ESV) rule, is proposed for speeding up the BP algorithm. We have extended the ESV rule by coupling it with a flat-spot removal strategy presented in [Fahlman, 1988], as well as incorporating a momentum term to combat the step size problem. The resulting rule has shown dramatically improved learning time over standard BP, measured in training epochs. Two versions of the ESV modification are mentioned in [Samad. 1991], on-demand and up-front, but simulation results are given mostly for the on-demand case. Our results indicate that the up-front version works somewhat better than the on-demand version in terms of learning speed. We have also analyzed the interactions between the three modifications as they are used in various combinations.

Original languageEnglish (US)
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
EditorsMarry Lou Padgett
PublisherPubl by Society of Photo-Optical Instrumentation Engineers
Pages95-100
Number of pages6
ISBN (Print)1565550072
StatePublished - Dec 1 1993
EventProceedings of the 3rd Workshop on Neural Networks: Academic/Industrial/NASA/Defense - Alabama, AL, USA
Duration: Feb 10 1992Feb 12 1992

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume1721
ISSN (Print)0277-786X

Other

OtherProceedings of the 3rd Workshop on Neural Networks: Academic/Industrial/NASA/Defense
CityAlabama, AL, USA
Period2/10/922/12/92

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Empirical analysis of the expected source values rule'. Together they form a unique fingerprint.

Cite this