Failure-driven learning in the control of ill-defined continuous systems

David W. Russell

Research output: Contribution to journalArticle

1 Scopus citations

Abstract

It is customary to expect that learning systems will frequently underperform or even fail in their initial stages of maturity. In fact, heuristic systems are somewhat encouraged to “fail” in order to expose the algorithm to the entire problem space, in the belief that this turbulent, knowledge-building process is an essential precursor to better performance later. This is clearly illustrated in learning a new game, where initial, erratic play is essential if all the rules are to be learned. Good and bad “moves” must be identified and cataloged so that strategies for avoiding, or at least postponing, losses (failures) can be formulated. Intelligent systems that use game-playing paradigms pose profound questions that must be addressed before any level of confidence can be placed in their ability to perform in the real world. This paper describes a game-playing methodology that has been used to control short-duration tasks and discusses its adaptation to continuous (and possibly ill-defined) processes. The paper postulates that system failures can be turned into opportunities for positive statistical reinforcement.

Original languageEnglish (US)
Pages (from-to)555-566
Number of pages12
JournalCybernetics and Systems
Volume25
Issue number4
DOIs
StatePublished - Jan 1 1994

All Science Journal Classification (ASJC) codes

  • Software
  • Information Systems
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Failure-driven learning in the control of ill-defined continuous systems'. Together they form a unique fingerprint.

Cite this