Abstract

Reliable forecasts of extreme but rare events, such as earthquakes, financial crashes, and epileptic seizures, would render interventions and precautions possible. Therefore, forecasting methods have been developed which intend to raise an alarm if an extreme event is about to occur. In order to statistically validate the performance of a prediction system, it must be compared to the performance of a random predictor, which raises alarms independent of the events. Such a random predictor can be obtained by bootstrapping or analytically. We propose an analytic statistical framework which, in contrast to conventional methods, allows for validating independently the sensitivity and specificity of a forecasting method. Moreover, our method accounts for the periods during which an event has to remain absent or occur after a respective forecast.

Original languageEnglish (US)
Article number022133
JournalPhysical Review E - Statistical, Nonlinear, and Soft Matter Physics
Volume90
Issue number2
DOIs
StatePublished - Aug 26 2014

Fingerprint

forecasting
Forecast
Extreme Events
evaluation
warning systems
Evaluation
Forecasting
Predictors
predictions
seizures
crashes
accident prevention
Rare Events
Bootstrapping
Crash
Earthquake
Specificity
earthquakes
Prediction
sensitivity

All Science Journal Classification (ASJC) codes

  • Statistical and Nonlinear Physics
  • Statistics and Probability
  • Condensed Matter Physics

Cite this

Mader, Malenka ; Mader, Wolfgang ; Gluckman, Bruce J. ; Timmer, Jens ; Schelter, Björn. / Statistical evaluation of forecasts. In: Physical Review E - Statistical, Nonlinear, and Soft Matter Physics. 2014 ; Vol. 90, No. 2.
@article{22df89dffd61421ebc4cf447938eab4e,
title = "Statistical evaluation of forecasts",
abstract = "Reliable forecasts of extreme but rare events, such as earthquakes, financial crashes, and epileptic seizures, would render interventions and precautions possible. Therefore, forecasting methods have been developed which intend to raise an alarm if an extreme event is about to occur. In order to statistically validate the performance of a prediction system, it must be compared to the performance of a random predictor, which raises alarms independent of the events. Such a random predictor can be obtained by bootstrapping or analytically. We propose an analytic statistical framework which, in contrast to conventional methods, allows for validating independently the sensitivity and specificity of a forecasting method. Moreover, our method accounts for the periods during which an event has to remain absent or occur after a respective forecast.",
author = "Malenka Mader and Wolfgang Mader and Gluckman, {Bruce J.} and Jens Timmer and Bj{\"o}rn Schelter",
year = "2014",
month = "8",
day = "26",
doi = "10.1103/PhysRevE.90.022133",
language = "English (US)",
volume = "90",
journal = "Physical Review E",
issn = "2470-0045",
publisher = "American Physical Society",
number = "2",

}

Statistical evaluation of forecasts. / Mader, Malenka; Mader, Wolfgang; Gluckman, Bruce J.; Timmer, Jens; Schelter, Björn.

In: Physical Review E - Statistical, Nonlinear, and Soft Matter Physics, Vol. 90, No. 2, 022133, 26.08.2014.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Statistical evaluation of forecasts

AU - Mader, Malenka

AU - Mader, Wolfgang

AU - Gluckman, Bruce J.

AU - Timmer, Jens

AU - Schelter, Björn

PY - 2014/8/26

Y1 - 2014/8/26

N2 - Reliable forecasts of extreme but rare events, such as earthquakes, financial crashes, and epileptic seizures, would render interventions and precautions possible. Therefore, forecasting methods have been developed which intend to raise an alarm if an extreme event is about to occur. In order to statistically validate the performance of a prediction system, it must be compared to the performance of a random predictor, which raises alarms independent of the events. Such a random predictor can be obtained by bootstrapping or analytically. We propose an analytic statistical framework which, in contrast to conventional methods, allows for validating independently the sensitivity and specificity of a forecasting method. Moreover, our method accounts for the periods during which an event has to remain absent or occur after a respective forecast.

AB - Reliable forecasts of extreme but rare events, such as earthquakes, financial crashes, and epileptic seizures, would render interventions and precautions possible. Therefore, forecasting methods have been developed which intend to raise an alarm if an extreme event is about to occur. In order to statistically validate the performance of a prediction system, it must be compared to the performance of a random predictor, which raises alarms independent of the events. Such a random predictor can be obtained by bootstrapping or analytically. We propose an analytic statistical framework which, in contrast to conventional methods, allows for validating independently the sensitivity and specificity of a forecasting method. Moreover, our method accounts for the periods during which an event has to remain absent or occur after a respective forecast.

UR - http://www.scopus.com/inward/record.url?scp=84940360678&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940360678&partnerID=8YFLogxK

U2 - 10.1103/PhysRevE.90.022133

DO - 10.1103/PhysRevE.90.022133

M3 - Article

C2 - 25215714

AN - SCOPUS:84940360678

VL - 90

JO - Physical Review E

JF - Physical Review E

SN - 2470-0045

IS - 2

M1 - 022133

ER -