Support Vector Learning for Fuzzy Rule-Based Classification Systems

Yixin Chen, James Z. Wang

Research output: Contribution to journalArticle

191 Citations (Scopus)

Abstract

To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.

Original languageEnglish (US)
Pages (from-to)716-728
Number of pages13
JournalIEEE Transactions on Fuzzy Systems
Volume11
Issue number6
DOIs
StatePublished - Dec 1 2003

Fingerprint

Fuzzy Classifier
Support Vector
Fuzzy rules
Fuzzy Rules
Classifiers
Positive definite
kernel
Kernel Function
Feature Space
Support vector machines
Support Vector Machine
Kernel Machines
Fuzzy Inference
Learning
Curse of Dimensionality
Infinite-dimensional Spaces
Training Samples
Fuzzy inference
Fuzzy systems
Prediction Error

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Cite this

@article{744a1b2e1b2e43b28b2a342c57b1bb60,
title = "Support Vector Learning for Fuzzy Rule-Based Classification Systems",
abstract = "To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the {"}curse of dimensionality.{"} Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.",
author = "Yixin Chen and Wang, {James Z.}",
year = "2003",
month = "12",
day = "1",
doi = "10.1109/TFUZZ.2003.819843",
language = "English (US)",
volume = "11",
pages = "716--728",
journal = "IEEE Transactions on Fuzzy Systems",
issn = "1063-6706",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "6",

}

Support Vector Learning for Fuzzy Rule-Based Classification Systems. / Chen, Yixin; Wang, James Z.

In: IEEE Transactions on Fuzzy Systems, Vol. 11, No. 6, 01.12.2003, p. 716-728.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Support Vector Learning for Fuzzy Rule-Based Classification Systems

AU - Chen, Yixin

AU - Wang, James Z.

PY - 2003/12/1

Y1 - 2003/12/1

N2 - To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.

AB - To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.

UR - http://www.scopus.com/inward/record.url?scp=0346076782&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0346076782&partnerID=8YFLogxK

U2 - 10.1109/TFUZZ.2003.819843

DO - 10.1109/TFUZZ.2003.819843

M3 - Article

AN - SCOPUS:0346076782

VL - 11

SP - 716

EP - 728

JO - IEEE Transactions on Fuzzy Systems

JF - IEEE Transactions on Fuzzy Systems

SN - 1063-6706

IS - 6

ER -