A comparison of gradient ascent, gradient descent and genetic-algorithm- based artificial neural networks for the binary classification problem

Research output: Contribution to journalReview article

14 Citations (Scopus)

Abstract

We compare log maximum likelihood gradient ascent, root-mean-square error minimizing gradient descent and genetic-algorithm-based artificial neural network procedures for a binary classification problem. We use simulated data and real-world data sets, and four different performance metrics of correct classification, sensitivity, specificity and reliability for our comparisons. Our experiments indicate that a genetic-algorithm-based artificial neural network that maximizes the total number of correct classifications generally fares well for tlw binary classification problem. However, if the training data set contains inconsistent decisions or noise then the log maximum likelihood maximizing gradient ascent may be the best classification approach to use. The root-mean-square minimizing gradient descent approach appears to overfit training data and has the lowest reliability among the approaches considered for our research. At the end of the paper, we provide a few guidelines, including computational complexity, for selection of an appropriate technique for a given binary classification problem.

Original languageEnglish (US)
Pages (from-to)65-86
Number of pages22
JournalExpert Systems
Volume24
Issue number2
DOIs
StatePublished - May 1 2007

Fingerprint

Binary Classification
Descent Algorithm
Ascent
Gradient Algorithm
Gradient Descent
Classification Problems
Artificial Neural Network
Genetic algorithms
Genetic Algorithm
Gradient
Neural networks
Maximum Likelihood
Roots
Maximum likelihood
Performance Metrics
Mean square error
Inconsistent
Mean Square
Specificity
Lowest

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Theoretical Computer Science
  • Computational Theory and Mathematics
  • Artificial Intelligence

Cite this

@article{4054984f34654263876dc81f255fb0e0,
title = "A comparison of gradient ascent, gradient descent and genetic-algorithm- based artificial neural networks for the binary classification problem",
abstract = "We compare log maximum likelihood gradient ascent, root-mean-square error minimizing gradient descent and genetic-algorithm-based artificial neural network procedures for a binary classification problem. We use simulated data and real-world data sets, and four different performance metrics of correct classification, sensitivity, specificity and reliability for our comparisons. Our experiments indicate that a genetic-algorithm-based artificial neural network that maximizes the total number of correct classifications generally fares well for tlw binary classification problem. However, if the training data set contains inconsistent decisions or noise then the log maximum likelihood maximizing gradient ascent may be the best classification approach to use. The root-mean-square minimizing gradient descent approach appears to overfit training data and has the lowest reliability among the approaches considered for our research. At the end of the paper, we provide a few guidelines, including computational complexity, for selection of an appropriate technique for a given binary classification problem.",
author = "Pendharkar, {Parag C.}",
year = "2007",
month = "5",
day = "1",
doi = "10.1111/j.1468-0394.2007.00421.x",
language = "English (US)",
volume = "24",
pages = "65--86",
journal = "Expert Systems",
issn = "0266-4720",
publisher = "Wiley-Blackwell",
number = "2",

}

TY - JOUR

T1 - A comparison of gradient ascent, gradient descent and genetic-algorithm- based artificial neural networks for the binary classification problem

AU - Pendharkar, Parag C.

PY - 2007/5/1

Y1 - 2007/5/1

N2 - We compare log maximum likelihood gradient ascent, root-mean-square error minimizing gradient descent and genetic-algorithm-based artificial neural network procedures for a binary classification problem. We use simulated data and real-world data sets, and four different performance metrics of correct classification, sensitivity, specificity and reliability for our comparisons. Our experiments indicate that a genetic-algorithm-based artificial neural network that maximizes the total number of correct classifications generally fares well for tlw binary classification problem. However, if the training data set contains inconsistent decisions or noise then the log maximum likelihood maximizing gradient ascent may be the best classification approach to use. The root-mean-square minimizing gradient descent approach appears to overfit training data and has the lowest reliability among the approaches considered for our research. At the end of the paper, we provide a few guidelines, including computational complexity, for selection of an appropriate technique for a given binary classification problem.

AB - We compare log maximum likelihood gradient ascent, root-mean-square error minimizing gradient descent and genetic-algorithm-based artificial neural network procedures for a binary classification problem. We use simulated data and real-world data sets, and four different performance metrics of correct classification, sensitivity, specificity and reliability for our comparisons. Our experiments indicate that a genetic-algorithm-based artificial neural network that maximizes the total number of correct classifications generally fares well for tlw binary classification problem. However, if the training data set contains inconsistent decisions or noise then the log maximum likelihood maximizing gradient ascent may be the best classification approach to use. The root-mean-square minimizing gradient descent approach appears to overfit training data and has the lowest reliability among the approaches considered for our research. At the end of the paper, we provide a few guidelines, including computational complexity, for selection of an appropriate technique for a given binary classification problem.

UR - http://www.scopus.com/inward/record.url?scp=34247180155&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34247180155&partnerID=8YFLogxK

U2 - 10.1111/j.1468-0394.2007.00421.x

DO - 10.1111/j.1468-0394.2007.00421.x

M3 - Review article

AN - SCOPUS:34247180155

VL - 24

SP - 65

EP - 86

JO - Expert Systems

JF - Expert Systems

SN - 0266-4720

IS - 2

ER -