### Abstract

A discussion is presented of the advantage of using a linear recurrent network to encode and recognize sequential data. The hidden Markov model (HMM) is shown to be a special case of such linear recurrent second-order neural networks. The Baum-Welch reestimation formula, which has proved very useful in training HMM, can also be used to learn a linear recurrent network. As an example, a network has successfully learned the stochastic Reber grammar with only a few hundred sample strings in about 14 iterations. The relative merits and limitations of the Baum-Welch optimal ascent algorithm in comparison with the error correction-gradient descent-learning algorithm are discussed.

Original language | English (US) |
---|---|

Title of host publication | 90 Int Jt Conf Neural Networks IJCNN 90 |

Publisher | Publ by IEEE |

Pages | 729-734 |

Number of pages | 6 |

State | Published - 1990 |

Event | 1990 International Joint Conference on Neural Networks - IJCNN 90 - San Diego, CA, USA Duration: Jun 17 1990 → Jun 21 1990 |

### Other

Other | 1990 International Joint Conference on Neural Networks - IJCNN 90 |
---|---|

City | San Diego, CA, USA |

Period | 6/17/90 → 6/21/90 |

### Fingerprint

### All Science Journal Classification (ASJC) codes

- Engineering(all)

### Cite this

*90 Int Jt Conf Neural Networks IJCNN 90*(pp. 729-734). Publ by IEEE.

}

*90 Int Jt Conf Neural Networks IJCNN 90.*Publ by IEEE, pp. 729-734, 1990 International Joint Conference on Neural Networks - IJCNN 90, San Diego, CA, USA, 6/17/90.

**Recurrent neural networks, hidden Markov models and stochastic grammars.** / Sun, G. Z.; Chen, H. H.; Lee, Y. C.; Giles, C. L.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Recurrent neural networks, hidden Markov models and stochastic grammars

AU - Sun, G. Z.

AU - Chen, H. H.

AU - Lee, Y. C.

AU - Giles, C. L.

PY - 1990

Y1 - 1990

N2 - A discussion is presented of the advantage of using a linear recurrent network to encode and recognize sequential data. The hidden Markov model (HMM) is shown to be a special case of such linear recurrent second-order neural networks. The Baum-Welch reestimation formula, which has proved very useful in training HMM, can also be used to learn a linear recurrent network. As an example, a network has successfully learned the stochastic Reber grammar with only a few hundred sample strings in about 14 iterations. The relative merits and limitations of the Baum-Welch optimal ascent algorithm in comparison with the error correction-gradient descent-learning algorithm are discussed.

AB - A discussion is presented of the advantage of using a linear recurrent network to encode and recognize sequential data. The hidden Markov model (HMM) is shown to be a special case of such linear recurrent second-order neural networks. The Baum-Welch reestimation formula, which has proved very useful in training HMM, can also be used to learn a linear recurrent network. As an example, a network has successfully learned the stochastic Reber grammar with only a few hundred sample strings in about 14 iterations. The relative merits and limitations of the Baum-Welch optimal ascent algorithm in comparison with the error correction-gradient descent-learning algorithm are discussed.

UR - http://www.scopus.com/inward/record.url?scp=0025547722&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0025547722&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:0025547722

SP - 729

EP - 734

BT - 90 Int Jt Conf Neural Networks IJCNN 90

PB - Publ by IEEE

ER -