Embedding black-box regression techniques into hierarchical Bayesian models

Benjamin A. Shaby, Daniel Fink

Research output: Contribution to journalArticle

Abstract

Hierarchical models enable the encoding of a variety of parametric structures. However, when presented with a large number of covariates upon which some component of a model hierarchy depends, the modeller may be unwilling or unable to specify a form for that dependence. Data-mining methods are designed to automatically discover relationships between many covariates and a response surface, easily accommodating non-linearities and higher-order interactions. We present a method of wrapping hierarchical models around data-mining methods, preserving the best qualities of the two paradigms. We fit the resulting semi-parametric models using an approximate Gibbs sampler called HEBBRU. Using a simulated dataset, we show that HEBBRU is useful for exploratory analysis and displays excellent predictive accuracy. Finally, we apply HEBBRU to an ornithological dataset drawn from the eBird database.

Original languageEnglish (US)
Pages (from-to)1753-1766
Number of pages14
JournalJournal of Statistical Computation and Simulation
Volume82
Issue number12
DOIs
StatePublished - Dec 1 2012

Fingerprint

Hierarchical Bayesian Model
Black Box
Regression
Hierarchical Model
Covariates
Data Mining
Data mining
Exploratory Analysis
Gibbs Sampler
Response Surface
Semiparametric Model
Encoding
Paradigm
Nonlinearity
Higher Order
Interaction
Black box
Hierarchical Bayesian model
Hierarchical model
Model

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Modeling and Simulation
  • Statistics, Probability and Uncertainty
  • Applied Mathematics

Cite this

Shaby, Benjamin A. ; Fink, Daniel. / Embedding black-box regression techniques into hierarchical Bayesian models. In: Journal of Statistical Computation and Simulation. 2012 ; Vol. 82, No. 12. pp. 1753-1766.
@article{37a48ffbbe8c4c2c9a30910fd62d4fcb,
title = "Embedding black-box regression techniques into hierarchical Bayesian models",
abstract = "Hierarchical models enable the encoding of a variety of parametric structures. However, when presented with a large number of covariates upon which some component of a model hierarchy depends, the modeller may be unwilling or unable to specify a form for that dependence. Data-mining methods are designed to automatically discover relationships between many covariates and a response surface, easily accommodating non-linearities and higher-order interactions. We present a method of wrapping hierarchical models around data-mining methods, preserving the best qualities of the two paradigms. We fit the resulting semi-parametric models using an approximate Gibbs sampler called HEBBRU. Using a simulated dataset, we show that HEBBRU is useful for exploratory analysis and displays excellent predictive accuracy. Finally, we apply HEBBRU to an ornithological dataset drawn from the eBird database.",
author = "Shaby, {Benjamin A.} and Daniel Fink",
year = "2012",
month = "12",
day = "1",
doi = "10.1080/00949655.2011.594052",
language = "English (US)",
volume = "82",
pages = "1753--1766",
journal = "Journal of Statistical Computation and Simulation",
issn = "0094-9655",
publisher = "Taylor and Francis Ltd.",
number = "12",

}

Embedding black-box regression techniques into hierarchical Bayesian models. / Shaby, Benjamin A.; Fink, Daniel.

In: Journal of Statistical Computation and Simulation, Vol. 82, No. 12, 01.12.2012, p. 1753-1766.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Embedding black-box regression techniques into hierarchical Bayesian models

AU - Shaby, Benjamin A.

AU - Fink, Daniel

PY - 2012/12/1

Y1 - 2012/12/1

N2 - Hierarchical models enable the encoding of a variety of parametric structures. However, when presented with a large number of covariates upon which some component of a model hierarchy depends, the modeller may be unwilling or unable to specify a form for that dependence. Data-mining methods are designed to automatically discover relationships between many covariates and a response surface, easily accommodating non-linearities and higher-order interactions. We present a method of wrapping hierarchical models around data-mining methods, preserving the best qualities of the two paradigms. We fit the resulting semi-parametric models using an approximate Gibbs sampler called HEBBRU. Using a simulated dataset, we show that HEBBRU is useful for exploratory analysis and displays excellent predictive accuracy. Finally, we apply HEBBRU to an ornithological dataset drawn from the eBird database.

AB - Hierarchical models enable the encoding of a variety of parametric structures. However, when presented with a large number of covariates upon which some component of a model hierarchy depends, the modeller may be unwilling or unable to specify a form for that dependence. Data-mining methods are designed to automatically discover relationships between many covariates and a response surface, easily accommodating non-linearities and higher-order interactions. We present a method of wrapping hierarchical models around data-mining methods, preserving the best qualities of the two paradigms. We fit the resulting semi-parametric models using an approximate Gibbs sampler called HEBBRU. Using a simulated dataset, we show that HEBBRU is useful for exploratory analysis and displays excellent predictive accuracy. Finally, we apply HEBBRU to an ornithological dataset drawn from the eBird database.

UR - http://www.scopus.com/inward/record.url?scp=84869148695&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84869148695&partnerID=8YFLogxK

U2 - 10.1080/00949655.2011.594052

DO - 10.1080/00949655.2011.594052

M3 - Article

AN - SCOPUS:84869148695

VL - 82

SP - 1753

EP - 1766

JO - Journal of Statistical Computation and Simulation

JF - Journal of Statistical Computation and Simulation

SN - 0094-9655

IS - 12

ER -