Combining hidden Markov model and fuzzy neural network for continuous recognition of complex dynamic gestures

Huiyue Wu, Jianmin Wang, Xiaolong Zhang

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

In the design of gesture-based user interfaces, continuously recognizing complex dynamic gestures is a challenging task, because of the high-dimensional information of gestures, ambiguous semantic meanings of gestures, and the presence of unpredictable non-gesture body motions. In this paper, we propose a hybrid model that can leverage the time-series modeling ability of hidden Markov model and the fuzzy inference ability of fuzzy neural network. First, a complex dynamic gesture is decomposed and fed into the hybrid model. The likelihood probability of an observation sequence estimated by the hidden Markov model is used as fuzzy membership degree of the corresponding fuzzy class variable in fuzzy neural network. Next, fuzzy rule modeling and fuzzy inference are performed by fuzzy neural network for gesture classification. To spot key gestures accurately, a threshold model is introduced to calculate the likelihood threshold of an input pattern and provide a reliability measure of whether to accept the pattern as a gesture. Finally, the proposed method is applied to recognize ten user-defined dynamic gestures for controlling interactive digital television in a smart room. Results of our experiment show that the proposed method performed better in terms of spotting reliability and recognition accuracy than conventional gesture recognition methods.

Original languageEnglish (US)
Pages (from-to)1265-1278
Number of pages14
JournalVisual Computer
Volume33
Issue number10
DOIs
StatePublished - Oct 1 2017

Fingerprint

Fuzzy neural networks
Hidden Markov models
Fuzzy inference
Interactive television
Digital television
Gesture recognition
Fuzzy rules
User interfaces
Time series
Semantics
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Cite this

@article{b7544bc96619403d87952f06cab4d441,
title = "Combining hidden Markov model and fuzzy neural network for continuous recognition of complex dynamic gestures",
abstract = "In the design of gesture-based user interfaces, continuously recognizing complex dynamic gestures is a challenging task, because of the high-dimensional information of gestures, ambiguous semantic meanings of gestures, and the presence of unpredictable non-gesture body motions. In this paper, we propose a hybrid model that can leverage the time-series modeling ability of hidden Markov model and the fuzzy inference ability of fuzzy neural network. First, a complex dynamic gesture is decomposed and fed into the hybrid model. The likelihood probability of an observation sequence estimated by the hidden Markov model is used as fuzzy membership degree of the corresponding fuzzy class variable in fuzzy neural network. Next, fuzzy rule modeling and fuzzy inference are performed by fuzzy neural network for gesture classification. To spot key gestures accurately, a threshold model is introduced to calculate the likelihood threshold of an input pattern and provide a reliability measure of whether to accept the pattern as a gesture. Finally, the proposed method is applied to recognize ten user-defined dynamic gestures for controlling interactive digital television in a smart room. Results of our experiment show that the proposed method performed better in terms of spotting reliability and recognition accuracy than conventional gesture recognition methods.",
author = "Huiyue Wu and Jianmin Wang and Xiaolong Zhang",
year = "2017",
month = "10",
day = "1",
doi = "10.1007/s00371-015-1147-2",
language = "English (US)",
volume = "33",
pages = "1265--1278",
journal = "Visual Computer",
issn = "0178-2789",
publisher = "Springer Verlag",
number = "10",

}

Combining hidden Markov model and fuzzy neural network for continuous recognition of complex dynamic gestures. / Wu, Huiyue; Wang, Jianmin; Zhang, Xiaolong.

In: Visual Computer, Vol. 33, No. 10, 01.10.2017, p. 1265-1278.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Combining hidden Markov model and fuzzy neural network for continuous recognition of complex dynamic gestures

AU - Wu, Huiyue

AU - Wang, Jianmin

AU - Zhang, Xiaolong

PY - 2017/10/1

Y1 - 2017/10/1

N2 - In the design of gesture-based user interfaces, continuously recognizing complex dynamic gestures is a challenging task, because of the high-dimensional information of gestures, ambiguous semantic meanings of gestures, and the presence of unpredictable non-gesture body motions. In this paper, we propose a hybrid model that can leverage the time-series modeling ability of hidden Markov model and the fuzzy inference ability of fuzzy neural network. First, a complex dynamic gesture is decomposed and fed into the hybrid model. The likelihood probability of an observation sequence estimated by the hidden Markov model is used as fuzzy membership degree of the corresponding fuzzy class variable in fuzzy neural network. Next, fuzzy rule modeling and fuzzy inference are performed by fuzzy neural network for gesture classification. To spot key gestures accurately, a threshold model is introduced to calculate the likelihood threshold of an input pattern and provide a reliability measure of whether to accept the pattern as a gesture. Finally, the proposed method is applied to recognize ten user-defined dynamic gestures for controlling interactive digital television in a smart room. Results of our experiment show that the proposed method performed better in terms of spotting reliability and recognition accuracy than conventional gesture recognition methods.

AB - In the design of gesture-based user interfaces, continuously recognizing complex dynamic gestures is a challenging task, because of the high-dimensional information of gestures, ambiguous semantic meanings of gestures, and the presence of unpredictable non-gesture body motions. In this paper, we propose a hybrid model that can leverage the time-series modeling ability of hidden Markov model and the fuzzy inference ability of fuzzy neural network. First, a complex dynamic gesture is decomposed and fed into the hybrid model. The likelihood probability of an observation sequence estimated by the hidden Markov model is used as fuzzy membership degree of the corresponding fuzzy class variable in fuzzy neural network. Next, fuzzy rule modeling and fuzzy inference are performed by fuzzy neural network for gesture classification. To spot key gestures accurately, a threshold model is introduced to calculate the likelihood threshold of an input pattern and provide a reliability measure of whether to accept the pattern as a gesture. Finally, the proposed method is applied to recognize ten user-defined dynamic gestures for controlling interactive digital television in a smart room. Results of our experiment show that the proposed method performed better in terms of spotting reliability and recognition accuracy than conventional gesture recognition methods.

UR - http://www.scopus.com/inward/record.url?scp=84940055081&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940055081&partnerID=8YFLogxK

U2 - 10.1007/s00371-015-1147-2

DO - 10.1007/s00371-015-1147-2

M3 - Article

AN - SCOPUS:84940055081

VL - 33

SP - 1265

EP - 1278

JO - Visual Computer

JF - Visual Computer

SN - 0178-2789

IS - 10

ER -