Conquering the needle-in-a-haystack

How correlated input variables beneficially alter the fitness landscape for neural networks

Stephen D. Turner, Marylyn Deriggi Ritchie, William S. Bush

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Evolutionary algorithms such as genetic programming and grammatical evolution have been used for simultaneously optimizing network architecture, variable selection, and weights for artificial neural networks. Using an evolutionary algorithm to perform variable selection while searching for non-linear interactions is akin to searching for a needle in a haystack. There is, however, a considerable amount of correlation among variables in biological datasets, such as in microarray or genetic studies. Using the XOR problem, we show that correlation between non-functional and functional variables alters the variable selection fitness landscape by broadening the fitness peak over a wider range of potential input variables. Furthermore, when sub-optimal weights are used, local optima in the variable selection fitness landscape appear centered on each of the two functional variables. These attributes of the fitness landscape may supply building blocks for evolutionary search procedures, and may provide a rationale for conducting a local search for variable selection.

Original languageEnglish (US)
Title of host publicationEvolutionary Computation, Machine Learning and Data Mining in Bioinformatics - 7th European Conference, EvoBIO 2009, Proceedings
Pages80-91
Number of pages12
DOIs
StatePublished - Jul 23 2009
Event7th European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics, EvoBIO 2009 - Tubingen, Germany
Duration: Apr 15 2009Apr 17 2009

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5483 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other7th European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics, EvoBIO 2009
CountryGermany
CityTubingen
Period4/15/094/17/09

Fingerprint

Fitness Landscape
Variable Selection
Evolutionary algorithms
Neural Networks
Neural networks
Genetic programming
Microarrays
Network architecture
Evolutionary Algorithms
Grammatical Evolution
Nonlinear Interaction
Network Architecture
Genetic Programming
Microarray
Building Blocks
Local Search
Fitness
Artificial Neural Network
Attribute
Range of data

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Turner, S. D., Ritchie, M. D., & Bush, W. S. (2009). Conquering the needle-in-a-haystack: How correlated input variables beneficially alter the fitness landscape for neural networks. In Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics - 7th European Conference, EvoBIO 2009, Proceedings (pp. 80-91). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5483 LNCS). https://doi.org/10.1007/978-3-642-01184-9_8
Turner, Stephen D. ; Ritchie, Marylyn Deriggi ; Bush, William S. / Conquering the needle-in-a-haystack : How correlated input variables beneficially alter the fitness landscape for neural networks. Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics - 7th European Conference, EvoBIO 2009, Proceedings. 2009. pp. 80-91 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{c9dcc8dd94044c49b865c069915322ff,
title = "Conquering the needle-in-a-haystack: How correlated input variables beneficially alter the fitness landscape for neural networks",
abstract = "Evolutionary algorithms such as genetic programming and grammatical evolution have been used for simultaneously optimizing network architecture, variable selection, and weights for artificial neural networks. Using an evolutionary algorithm to perform variable selection while searching for non-linear interactions is akin to searching for a needle in a haystack. There is, however, a considerable amount of correlation among variables in biological datasets, such as in microarray or genetic studies. Using the XOR problem, we show that correlation between non-functional and functional variables alters the variable selection fitness landscape by broadening the fitness peak over a wider range of potential input variables. Furthermore, when sub-optimal weights are used, local optima in the variable selection fitness landscape appear centered on each of the two functional variables. These attributes of the fitness landscape may supply building blocks for evolutionary search procedures, and may provide a rationale for conducting a local search for variable selection.",
author = "Turner, {Stephen D.} and Ritchie, {Marylyn Deriggi} and Bush, {William S.}",
year = "2009",
month = "7",
day = "23",
doi = "10.1007/978-3-642-01184-9_8",
language = "English (US)",
isbn = "3642011837",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "80--91",
booktitle = "Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics - 7th European Conference, EvoBIO 2009, Proceedings",

}

Turner, SD, Ritchie, MD & Bush, WS 2009, Conquering the needle-in-a-haystack: How correlated input variables beneficially alter the fitness landscape for neural networks. in Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics - 7th European Conference, EvoBIO 2009, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5483 LNCS, pp. 80-91, 7th European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics, EvoBIO 2009, Tubingen, Germany, 4/15/09. https://doi.org/10.1007/978-3-642-01184-9_8

Conquering the needle-in-a-haystack : How correlated input variables beneficially alter the fitness landscape for neural networks. / Turner, Stephen D.; Ritchie, Marylyn Deriggi; Bush, William S.

Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics - 7th European Conference, EvoBIO 2009, Proceedings. 2009. p. 80-91 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5483 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Conquering the needle-in-a-haystack

T2 - How correlated input variables beneficially alter the fitness landscape for neural networks

AU - Turner, Stephen D.

AU - Ritchie, Marylyn Deriggi

AU - Bush, William S.

PY - 2009/7/23

Y1 - 2009/7/23

N2 - Evolutionary algorithms such as genetic programming and grammatical evolution have been used for simultaneously optimizing network architecture, variable selection, and weights for artificial neural networks. Using an evolutionary algorithm to perform variable selection while searching for non-linear interactions is akin to searching for a needle in a haystack. There is, however, a considerable amount of correlation among variables in biological datasets, such as in microarray or genetic studies. Using the XOR problem, we show that correlation between non-functional and functional variables alters the variable selection fitness landscape by broadening the fitness peak over a wider range of potential input variables. Furthermore, when sub-optimal weights are used, local optima in the variable selection fitness landscape appear centered on each of the two functional variables. These attributes of the fitness landscape may supply building blocks for evolutionary search procedures, and may provide a rationale for conducting a local search for variable selection.

AB - Evolutionary algorithms such as genetic programming and grammatical evolution have been used for simultaneously optimizing network architecture, variable selection, and weights for artificial neural networks. Using an evolutionary algorithm to perform variable selection while searching for non-linear interactions is akin to searching for a needle in a haystack. There is, however, a considerable amount of correlation among variables in biological datasets, such as in microarray or genetic studies. Using the XOR problem, we show that correlation between non-functional and functional variables alters the variable selection fitness landscape by broadening the fitness peak over a wider range of potential input variables. Furthermore, when sub-optimal weights are used, local optima in the variable selection fitness landscape appear centered on each of the two functional variables. These attributes of the fitness landscape may supply building blocks for evolutionary search procedures, and may provide a rationale for conducting a local search for variable selection.

UR - http://www.scopus.com/inward/record.url?scp=67650660993&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67650660993&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-01184-9_8

DO - 10.1007/978-3-642-01184-9_8

M3 - Conference contribution

SN - 3642011837

SN - 9783642011832

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 80

EP - 91

BT - Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics - 7th European Conference, EvoBIO 2009, Proceedings

ER -

Turner SD, Ritchie MD, Bush WS. Conquering the needle-in-a-haystack: How correlated input variables beneficially alter the fitness landscape for neural networks. In Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics - 7th European Conference, EvoBIO 2009, Proceedings. 2009. p. 80-91. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-01184-9_8