Correcting biased observation model error in data assimilation

Tyrus Berry, John Harlim

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

While the formulation of most data assimilation schemes assumes an unbiased observation model error, in real applications model error with nontrivial biases is unavoidable. A practical example is errors in the radiative transfer model (which is used to assimilate satellite measurements) in the presence of clouds. Together with the dynamical model error, the result is that many (in fact 99%) of the cloudy observed measurements are not being used although they may contain useful information. This paper presents a novel nonparametric Bayesian scheme that is able to learn the observation model error distribution and correct the bias in incoming observations. This scheme can be used in tandem with any data assimilation forecasting system. The proposed model error estimator uses nonparametric likelihood functions constructed with data-driven basis functions based on the theory of kernel embeddings of conditional distributions developed in the machine learning community. Numerically, positive results are shown with two examples. The first example is designed to produce a bimodality in the observation model error (typical of "cloudy" observations) by introducing obstructions to the observations that occur randomly in space and time. The second example, which is physically more realistic, is to assimilate cloudy satellite brightness temperature-like quantities, generated from a stochastic multicloud model for tropical convection and a simple radiative transfer model.

Original languageEnglish (US)
Pages (from-to)2833-2853
Number of pages21
JournalMonthly Weather Review
Volume145
Issue number7
DOIs
StatePublished - Jul 1 2017

Fingerprint

data assimilation
radiative transfer
brightness temperature
convection

All Science Journal Classification (ASJC) codes

  • Atmospheric Science

Cite this

@article{737bdd9b3e5a4f6896676221293fa801,
title = "Correcting biased observation model error in data assimilation",
abstract = "While the formulation of most data assimilation schemes assumes an unbiased observation model error, in real applications model error with nontrivial biases is unavoidable. A practical example is errors in the radiative transfer model (which is used to assimilate satellite measurements) in the presence of clouds. Together with the dynamical model error, the result is that many (in fact 99{\%}) of the cloudy observed measurements are not being used although they may contain useful information. This paper presents a novel nonparametric Bayesian scheme that is able to learn the observation model error distribution and correct the bias in incoming observations. This scheme can be used in tandem with any data assimilation forecasting system. The proposed model error estimator uses nonparametric likelihood functions constructed with data-driven basis functions based on the theory of kernel embeddings of conditional distributions developed in the machine learning community. Numerically, positive results are shown with two examples. The first example is designed to produce a bimodality in the observation model error (typical of {"}cloudy{"} observations) by introducing obstructions to the observations that occur randomly in space and time. The second example, which is physically more realistic, is to assimilate cloudy satellite brightness temperature-like quantities, generated from a stochastic multicloud model for tropical convection and a simple radiative transfer model.",
author = "Tyrus Berry and John Harlim",
year = "2017",
month = "7",
day = "1",
doi = "10.1175/MWR-D-16-0428.1",
language = "English (US)",
volume = "145",
pages = "2833--2853",
journal = "Monthly Weather Review",
issn = "0027-0644",
publisher = "American Meteorological Society",
number = "7",

}

Correcting biased observation model error in data assimilation. / Berry, Tyrus; Harlim, John.

In: Monthly Weather Review, Vol. 145, No. 7, 01.07.2017, p. 2833-2853.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Correcting biased observation model error in data assimilation

AU - Berry, Tyrus

AU - Harlim, John

PY - 2017/7/1

Y1 - 2017/7/1

N2 - While the formulation of most data assimilation schemes assumes an unbiased observation model error, in real applications model error with nontrivial biases is unavoidable. A practical example is errors in the radiative transfer model (which is used to assimilate satellite measurements) in the presence of clouds. Together with the dynamical model error, the result is that many (in fact 99%) of the cloudy observed measurements are not being used although they may contain useful information. This paper presents a novel nonparametric Bayesian scheme that is able to learn the observation model error distribution and correct the bias in incoming observations. This scheme can be used in tandem with any data assimilation forecasting system. The proposed model error estimator uses nonparametric likelihood functions constructed with data-driven basis functions based on the theory of kernel embeddings of conditional distributions developed in the machine learning community. Numerically, positive results are shown with two examples. The first example is designed to produce a bimodality in the observation model error (typical of "cloudy" observations) by introducing obstructions to the observations that occur randomly in space and time. The second example, which is physically more realistic, is to assimilate cloudy satellite brightness temperature-like quantities, generated from a stochastic multicloud model for tropical convection and a simple radiative transfer model.

AB - While the formulation of most data assimilation schemes assumes an unbiased observation model error, in real applications model error with nontrivial biases is unavoidable. A practical example is errors in the radiative transfer model (which is used to assimilate satellite measurements) in the presence of clouds. Together with the dynamical model error, the result is that many (in fact 99%) of the cloudy observed measurements are not being used although they may contain useful information. This paper presents a novel nonparametric Bayesian scheme that is able to learn the observation model error distribution and correct the bias in incoming observations. This scheme can be used in tandem with any data assimilation forecasting system. The proposed model error estimator uses nonparametric likelihood functions constructed with data-driven basis functions based on the theory of kernel embeddings of conditional distributions developed in the machine learning community. Numerically, positive results are shown with two examples. The first example is designed to produce a bimodality in the observation model error (typical of "cloudy" observations) by introducing obstructions to the observations that occur randomly in space and time. The second example, which is physically more realistic, is to assimilate cloudy satellite brightness temperature-like quantities, generated from a stochastic multicloud model for tropical convection and a simple radiative transfer model.

UR - http://www.scopus.com/inward/record.url?scp=85021304929&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021304929&partnerID=8YFLogxK

U2 - 10.1175/MWR-D-16-0428.1

DO - 10.1175/MWR-D-16-0428.1

M3 - Article

AN - SCOPUS:85021304929

VL - 145

SP - 2833

EP - 2853

JO - Monthly Weather Review

JF - Monthly Weather Review

SN - 0027-0644

IS - 7

ER -