Scalable massively parallel artificial neural networks

Lyle N. Long, Ankur Gupta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

There is renewed interest in computational intelligence, due to advances in algorithms, neuroscience, and computer hardware. In addition there is enormous interest in autonomous vehicles (air, ground, and sea) and robotics, which need significant onboard intelligence. Work in this area could not only lead to better understanding of the human brain but also very useful engineering applications. The functioning of the human brain is not well understood, but enormous progress has been made in understanding it and, in particular, the neocortex. There are many reasons to develop models of the brain. Artificial Neural Networks (ANN), one type of model, can be very effective for pattern recognition, function approximation, scientific classification, control, and the analysis of time series data. ANNs often use the back-propagation algorithm for training, and can require large training times especially for large networks, but there are many other types of ANNs. Once the network is trained for a particular problem, however, it can produce results in a very short time. Parallelization of ANNs could drastically reduce the training time. An object-oriented, massively-parallel ANN (Artificial Neural Network) software package SPANN (Scalable Parallel Artificial Neural Network) has been developed and is described here. MPI was used to parallelize the C++ code. Only the neurons on tlie edges of the domains were involved in communication, in order to reduce the communication costs and maintain scalability. The back-propagation algorithm was used to train the network. In preliminary tests, the software was used to identify character sets. The code correctly identified all the characters wlien adequate training was used in the network. The code was run on up to 500 Intel Itanium processors with 25,000 neurons and more than 2 billion neuron weights. Various comparisons in training time, forward propagation time, and error reduction were also made.

Original languageEnglish (US)
Title of host publicationCollection of Technical Papers - InfoTech at Aerospace
Subtitle of host publicationAdvancing Contemporary Aerospace Technologies and Their Integration
Pages2359-2369
Number of pages11
StatePublished - Dec 1 2005
EventInfoTech at Aerospace: Advancing Contemporary Aerospace Technologies and Their Integration - Arlington, VA, United States
Duration: Sep 26 2005Sep 29 2005

Publication series

NameCollection of Technical Papers - InfoTech at Aerospace: Advancing Contemporary Aerospace Technologies and Their Integration
Volume4

Other

OtherInfoTech at Aerospace: Advancing Contemporary Aerospace Technologies and Their Integration
CountryUnited States
CityArlington, VA
Period9/26/059/29/05

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Fingerprint Dive into the research topics of 'Scalable massively parallel artificial neural networks'. Together they form a unique fingerprint.

Cite this