Noisy time series prediction using recurrent neural networks and grammatical inference

C. Lee Giles, Steve Lawrence, Ah Chung Tsoi

Research output: Contribution to journalArticle

203 Citations (Scopus)

Abstract

Financial forecasting is an example of a signal processing problem which is challenging due to small sample sizes, high noise, non-stationarily, and non-linearity. Neural networks have been very successful in a number of signal processing applications. We discuss fundamental limitations and inherent difficulties when using neural networks for the processing of high noise, small sample size signals. We introduce a new intelligent signal processing method which addresses the difficulties. The method proposed uses conversion into a symbolic representation with a self-organizing map, and grammatical inference with recurrent neural networks. We apply the method to the prediction of daily foreign exchange rates, addressing difficulties with non-stationarily, overfitting, and unequal a priori class probabilities, and we find significant predictability in comprehensive experiments covering 5 different foreign exchange rates. The method correctly predicts the direction of change for the next day with an error rate of 47.1%. The error rate reduces to around 40% when rejecting examples where the system has low confidence in its prediction. We show that the symbolic representation aids the extraction of symbolic knowledge from the trained recurrent neural networks in the form of deterministic finite state automata. These automata explain the operation of the system and are often relatively simple. Automata rules related to well known behavior such as trend following and mean reversal are extracted.

Original languageEnglish (US)
Pages (from-to)161-183
Number of pages23
JournalMachine Learning
Volume44
Issue number1-2
DOIs
StatePublished - Jul 1 2001

Fingerprint

Recurrent neural networks
Time series
Signal processing
Neural networks
Self organizing maps
Finite automata
Processing
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Cite this

Giles, C. Lee ; Lawrence, Steve ; Tsoi, Ah Chung. / Noisy time series prediction using recurrent neural networks and grammatical inference. In: Machine Learning. 2001 ; Vol. 44, No. 1-2. pp. 161-183.
@article{73ec922b2afb463895204e18387aa7fd,
title = "Noisy time series prediction using recurrent neural networks and grammatical inference",
abstract = "Financial forecasting is an example of a signal processing problem which is challenging due to small sample sizes, high noise, non-stationarily, and non-linearity. Neural networks have been very successful in a number of signal processing applications. We discuss fundamental limitations and inherent difficulties when using neural networks for the processing of high noise, small sample size signals. We introduce a new intelligent signal processing method which addresses the difficulties. The method proposed uses conversion into a symbolic representation with a self-organizing map, and grammatical inference with recurrent neural networks. We apply the method to the prediction of daily foreign exchange rates, addressing difficulties with non-stationarily, overfitting, and unequal a priori class probabilities, and we find significant predictability in comprehensive experiments covering 5 different foreign exchange rates. The method correctly predicts the direction of change for the next day with an error rate of 47.1{\%}. The error rate reduces to around 40{\%} when rejecting examples where the system has low confidence in its prediction. We show that the symbolic representation aids the extraction of symbolic knowledge from the trained recurrent neural networks in the form of deterministic finite state automata. These automata explain the operation of the system and are often relatively simple. Automata rules related to well known behavior such as trend following and mean reversal are extracted.",
author = "Giles, {C. Lee} and Steve Lawrence and Tsoi, {Ah Chung}",
year = "2001",
month = "7",
day = "1",
doi = "10.1023/A:1010884214864",
language = "English (US)",
volume = "44",
pages = "161--183",
journal = "Machine Learning",
issn = "0885-6125",
publisher = "Springer Netherlands",
number = "1-2",

}

Noisy time series prediction using recurrent neural networks and grammatical inference. / Giles, C. Lee; Lawrence, Steve; Tsoi, Ah Chung.

In: Machine Learning, Vol. 44, No. 1-2, 01.07.2001, p. 161-183.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Noisy time series prediction using recurrent neural networks and grammatical inference

AU - Giles, C. Lee

AU - Lawrence, Steve

AU - Tsoi, Ah Chung

PY - 2001/7/1

Y1 - 2001/7/1

N2 - Financial forecasting is an example of a signal processing problem which is challenging due to small sample sizes, high noise, non-stationarily, and non-linearity. Neural networks have been very successful in a number of signal processing applications. We discuss fundamental limitations and inherent difficulties when using neural networks for the processing of high noise, small sample size signals. We introduce a new intelligent signal processing method which addresses the difficulties. The method proposed uses conversion into a symbolic representation with a self-organizing map, and grammatical inference with recurrent neural networks. We apply the method to the prediction of daily foreign exchange rates, addressing difficulties with non-stationarily, overfitting, and unequal a priori class probabilities, and we find significant predictability in comprehensive experiments covering 5 different foreign exchange rates. The method correctly predicts the direction of change for the next day with an error rate of 47.1%. The error rate reduces to around 40% when rejecting examples where the system has low confidence in its prediction. We show that the symbolic representation aids the extraction of symbolic knowledge from the trained recurrent neural networks in the form of deterministic finite state automata. These automata explain the operation of the system and are often relatively simple. Automata rules related to well known behavior such as trend following and mean reversal are extracted.

AB - Financial forecasting is an example of a signal processing problem which is challenging due to small sample sizes, high noise, non-stationarily, and non-linearity. Neural networks have been very successful in a number of signal processing applications. We discuss fundamental limitations and inherent difficulties when using neural networks for the processing of high noise, small sample size signals. We introduce a new intelligent signal processing method which addresses the difficulties. The method proposed uses conversion into a symbolic representation with a self-organizing map, and grammatical inference with recurrent neural networks. We apply the method to the prediction of daily foreign exchange rates, addressing difficulties with non-stationarily, overfitting, and unequal a priori class probabilities, and we find significant predictability in comprehensive experiments covering 5 different foreign exchange rates. The method correctly predicts the direction of change for the next day with an error rate of 47.1%. The error rate reduces to around 40% when rejecting examples where the system has low confidence in its prediction. We show that the symbolic representation aids the extraction of symbolic knowledge from the trained recurrent neural networks in the form of deterministic finite state automata. These automata explain the operation of the system and are often relatively simple. Automata rules related to well known behavior such as trend following and mean reversal are extracted.

UR - http://www.scopus.com/inward/record.url?scp=0035399722&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0035399722&partnerID=8YFLogxK

U2 - 10.1023/A:1010884214864

DO - 10.1023/A:1010884214864

M3 - Article

VL - 44

SP - 161

EP - 183

JO - Machine Learning

JF - Machine Learning

SN - 0885-6125

IS - 1-2

ER -