Failure-driven learning in the control of ill-defined continuous systems

David W. Russell

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

It is customary to expect that learning systems will frequently underperform or even fail in their initial stages of maturity. In fact, heuristic systems are somewhat encouraged to “fail” in order to expose the algorithm to the entire problem space, in the belief that this turbulent, knowledge-building process is an essential precursor to better performance later. This is clearly illustrated in learning a new game, where initial, erratic play is essential if all the rules are to be learned. Good and bad “moves” must be identified and cataloged so that strategies for avoiding, or at least postponing, losses (failures) can be formulated. Intelligent systems that use game-playing paradigms pose profound questions that must be addressed before any level of confidence can be placed in their ability to perform in the real world. This paper describes a game-playing methodology that has been used to control short-duration tasks and discusses its adaptation to continuous (and possibly ill-defined) processes. The paper postulates that system failures can be turned into opportunities for positive statistical reinforcement.

Original languageEnglish (US)
Pages (from-to)555-566
Number of pages12
JournalCybernetics and Systems
Volume25
Issue number4
DOIs
StatePublished - Jan 1 1994

Fingerprint

Intelligent systems
Learning systems
Reinforcement

All Science Journal Classification (ASJC) codes

  • Software
  • Information Systems
  • Artificial Intelligence

Cite this

@article{2cada399381f419b80d06231fa9af754,
title = "Failure-driven learning in the control of ill-defined continuous systems",
abstract = "It is customary to expect that learning systems will frequently underperform or even fail in their initial stages of maturity. In fact, heuristic systems are somewhat encouraged to “fail” in order to expose the algorithm to the entire problem space, in the belief that this turbulent, knowledge-building process is an essential precursor to better performance later. This is clearly illustrated in learning a new game, where initial, erratic play is essential if all the rules are to be learned. Good and bad “moves” must be identified and cataloged so that strategies for avoiding, or at least postponing, losses (failures) can be formulated. Intelligent systems that use game-playing paradigms pose profound questions that must be addressed before any level of confidence can be placed in their ability to perform in the real world. This paper describes a game-playing methodology that has been used to control short-duration tasks and discusses its adaptation to continuous (and possibly ill-defined) processes. The paper postulates that system failures can be turned into opportunities for positive statistical reinforcement.",
author = "Russell, {David W.}",
year = "1994",
month = "1",
day = "1",
doi = "10.1080/01969729408902341",
language = "English (US)",
volume = "25",
pages = "555--566",
journal = "Cybernetics and Systems",
issn = "0196-9722",
publisher = "Taylor and Francis Ltd.",
number = "4",

}

Failure-driven learning in the control of ill-defined continuous systems. / Russell, David W.

In: Cybernetics and Systems, Vol. 25, No. 4, 01.01.1994, p. 555-566.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Failure-driven learning in the control of ill-defined continuous systems

AU - Russell, David W.

PY - 1994/1/1

Y1 - 1994/1/1

N2 - It is customary to expect that learning systems will frequently underperform or even fail in their initial stages of maturity. In fact, heuristic systems are somewhat encouraged to “fail” in order to expose the algorithm to the entire problem space, in the belief that this turbulent, knowledge-building process is an essential precursor to better performance later. This is clearly illustrated in learning a new game, where initial, erratic play is essential if all the rules are to be learned. Good and bad “moves” must be identified and cataloged so that strategies for avoiding, or at least postponing, losses (failures) can be formulated. Intelligent systems that use game-playing paradigms pose profound questions that must be addressed before any level of confidence can be placed in their ability to perform in the real world. This paper describes a game-playing methodology that has been used to control short-duration tasks and discusses its adaptation to continuous (and possibly ill-defined) processes. The paper postulates that system failures can be turned into opportunities for positive statistical reinforcement.

AB - It is customary to expect that learning systems will frequently underperform or even fail in their initial stages of maturity. In fact, heuristic systems are somewhat encouraged to “fail” in order to expose the algorithm to the entire problem space, in the belief that this turbulent, knowledge-building process is an essential precursor to better performance later. This is clearly illustrated in learning a new game, where initial, erratic play is essential if all the rules are to be learned. Good and bad “moves” must be identified and cataloged so that strategies for avoiding, or at least postponing, losses (failures) can be formulated. Intelligent systems that use game-playing paradigms pose profound questions that must be addressed before any level of confidence can be placed in their ability to perform in the real world. This paper describes a game-playing methodology that has been used to control short-duration tasks and discusses its adaptation to continuous (and possibly ill-defined) processes. The paper postulates that system failures can be turned into opportunities for positive statistical reinforcement.

UR - http://www.scopus.com/inward/record.url?scp=0028462392&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0028462392&partnerID=8YFLogxK

U2 - 10.1080/01969729408902341

DO - 10.1080/01969729408902341

M3 - Article

AN - SCOPUS:0028462392

VL - 25

SP - 555

EP - 566

JO - Cybernetics and Systems

JF - Cybernetics and Systems

SN - 0196-9722

IS - 4

ER -