Online learning of deep hybrid architectures for semi-supervised categorization

Alexander G. Ororbia, David Reitter, Jian Wu, C. Leegiles

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

A hybrid architecture is presented capable of online learning from both labeled and unlabeled samples. It combines both generative and discriminative objectives to derive a new variant of the Deep Belief Network, i.e., the Stacked Boltzmann Experts Network model. The model’s training algorithm is built on principles developed from hybrid discriminative Boltzmann machines and composes deep architectures in a greedy fashion. It makes use of its inherent “layer-wise ensemble” nature to perform useful classification work. We (1) compare this architecture against a hybrid denoising autoencoder version of itself as well as several other models and (2) investigate training in the context of an incremental learning procedure. The best-performing hybrid model, the Stacked Boltzmann Experts Network, consistently outperforms all others.

Original languageEnglish (US)
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings
EditorsAnnalisa Appice, João Gama, Vitor Santos Costa, João Gama, Alípio Jorge, Annalisa Appice, Annalisa Appice, Vitor Santos Costa, Alípio Jorge, Annalisa Appice, Pedro Pereira Rodrigues, Pedro Pereira Rodrigues, João Gama, Vitor Santos Costa, Soares Soares, Pedro Pereira Rodrigues, Soares Soares, Soares Soares, João Gama, Soares Soares, Alípio Jorge, Alípio Jorge, Pedro Pereira Rodrigues, Vitor Santos Costa
PublisherSpringer Verlag
Pages516-532
Number of pages17
ISBN (Print)9783319235271, 9783319235271, 9783319235271, 9783319235271
DOIs
StatePublished - Jan 1 2015
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015 - Porto, Portugal
Duration: Sep 7 2015Sep 11 2015

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9284
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015
CountryPortugal
CityPorto
Period9/7/159/11/15

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Online learning of deep hybrid architectures for semi-supervised categorization'. Together they form a unique fingerprint.

  • Cite this

    Ororbia, A. G., Reitter, D., Wu, J., & Leegiles, C. (2015). Online learning of deep hybrid architectures for semi-supervised categorization. In A. Appice, J. Gama, V. S. Costa, J. Gama, A. Jorge, A. Appice, A. Appice, V. S. Costa, A. Jorge, A. Appice, P. P. Rodrigues, P. P. Rodrigues, J. Gama, V. S. Costa, S. Soares, P. P. Rodrigues, S. Soares, S. Soares, J. Gama, S. Soares, A. Jorge, A. Jorge, P. P. Rodrigues, ... V. S. Costa (Eds.), Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings (pp. 516-532). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9284). Springer Verlag. https://doi.org/10.1007/978-3-319-23528-8_32