Toward ensemble assimilation of hyperspectral satellite observations with data compression and dimension reduction using principal component analysis

Yinghui Lu, Fuqing Zhang

Research output: Contribution to journalReview article

Abstract

Satellite-based hyperspectral radiometers usually have thousands of infrared channels that contain atmospheric state information with higher vertical resolution compared to observations from traditional sensors. However, the large numbers of channels can lead to computational burden in satellite data retrieval and assimilation. Furthermore, most of the channels are highly correlated and the pieces of independent information contained in the hyperspectral observations are usually much smaller than the number of channels. Principal component analysis (PCA) was used in this research to compress the observational information content contained in the Atmospheric Infrared Sounder (AIRS) channels to a few leading principal components (PCs). The corresponding PC scores were then assimilated into a PCA-based ensemble Kalman filter (EnKF) system. In this proof-of-concept study based on simulated observations, hyperspectral brightness temperatures were simulated using the atmospheric state vectors from convection-permitting ensemble simulations of Hurricane Harvey (2017) as input to the Community Radiative Transfer Model (CRTM). The PCs were derived from a preexisting training dataset of brightness temperatures calculated from convection-permitting simulation over a large domain in the Indian Ocean representing generic atmospheric conditions over tropical oceans. The EnKF increments from assimilating many individual measurements in the brightness temperature space were compared to the EnKF increments from assimilating significantly fewer numbers of leading PCs. Results showed that assimilating about 10–20 leading PCs could yield increments that were nearly indistinguishable to that from assimilating hyperspectral measurements from orders of magnitude larger number of hyperspectral channels.

Original languageEnglish (US)
Pages (from-to)3505-3518
Number of pages14
JournalMonthly Weather Review
Volume147
Issue number10
DOIs
StatePublished - Jan 1 2019

Fingerprint

principal component analysis
compression
Kalman filter
brightness temperature
convection
AIRS
radiometer
hurricane
simulation
radiative transfer
observation satellite
assimilation
satellite data
sensor
ocean

All Science Journal Classification (ASJC) codes

  • Atmospheric Science

Cite this

@article{eac7b3e3bcba431dad884d02822b98f6,
title = "Toward ensemble assimilation of hyperspectral satellite observations with data compression and dimension reduction using principal component analysis",
abstract = "Satellite-based hyperspectral radiometers usually have thousands of infrared channels that contain atmospheric state information with higher vertical resolution compared to observations from traditional sensors. However, the large numbers of channels can lead to computational burden in satellite data retrieval and assimilation. Furthermore, most of the channels are highly correlated and the pieces of independent information contained in the hyperspectral observations are usually much smaller than the number of channels. Principal component analysis (PCA) was used in this research to compress the observational information content contained in the Atmospheric Infrared Sounder (AIRS) channels to a few leading principal components (PCs). The corresponding PC scores were then assimilated into a PCA-based ensemble Kalman filter (EnKF) system. In this proof-of-concept study based on simulated observations, hyperspectral brightness temperatures were simulated using the atmospheric state vectors from convection-permitting ensemble simulations of Hurricane Harvey (2017) as input to the Community Radiative Transfer Model (CRTM). The PCs were derived from a preexisting training dataset of brightness temperatures calculated from convection-permitting simulation over a large domain in the Indian Ocean representing generic atmospheric conditions over tropical oceans. The EnKF increments from assimilating many individual measurements in the brightness temperature space were compared to the EnKF increments from assimilating significantly fewer numbers of leading PCs. Results showed that assimilating about 10–20 leading PCs could yield increments that were nearly indistinguishable to that from assimilating hyperspectral measurements from orders of magnitude larger number of hyperspectral channels.",
author = "Yinghui Lu and Fuqing Zhang",
year = "2019",
month = "1",
day = "1",
doi = "10.1175/MWR-D-18-0454.1",
language = "English (US)",
volume = "147",
pages = "3505--3518",
journal = "Monthly Weather Review",
issn = "0027-0644",
publisher = "American Meteorological Society",
number = "10",

}

Toward ensemble assimilation of hyperspectral satellite observations with data compression and dimension reduction using principal component analysis. / Lu, Yinghui; Zhang, Fuqing.

In: Monthly Weather Review, Vol. 147, No. 10, 01.01.2019, p. 3505-3518.

Research output: Contribution to journalReview article

TY - JOUR

T1 - Toward ensemble assimilation of hyperspectral satellite observations with data compression and dimension reduction using principal component analysis

AU - Lu, Yinghui

AU - Zhang, Fuqing

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Satellite-based hyperspectral radiometers usually have thousands of infrared channels that contain atmospheric state information with higher vertical resolution compared to observations from traditional sensors. However, the large numbers of channels can lead to computational burden in satellite data retrieval and assimilation. Furthermore, most of the channels are highly correlated and the pieces of independent information contained in the hyperspectral observations are usually much smaller than the number of channels. Principal component analysis (PCA) was used in this research to compress the observational information content contained in the Atmospheric Infrared Sounder (AIRS) channels to a few leading principal components (PCs). The corresponding PC scores were then assimilated into a PCA-based ensemble Kalman filter (EnKF) system. In this proof-of-concept study based on simulated observations, hyperspectral brightness temperatures were simulated using the atmospheric state vectors from convection-permitting ensemble simulations of Hurricane Harvey (2017) as input to the Community Radiative Transfer Model (CRTM). The PCs were derived from a preexisting training dataset of brightness temperatures calculated from convection-permitting simulation over a large domain in the Indian Ocean representing generic atmospheric conditions over tropical oceans. The EnKF increments from assimilating many individual measurements in the brightness temperature space were compared to the EnKF increments from assimilating significantly fewer numbers of leading PCs. Results showed that assimilating about 10–20 leading PCs could yield increments that were nearly indistinguishable to that from assimilating hyperspectral measurements from orders of magnitude larger number of hyperspectral channels.

AB - Satellite-based hyperspectral radiometers usually have thousands of infrared channels that contain atmospheric state information with higher vertical resolution compared to observations from traditional sensors. However, the large numbers of channels can lead to computational burden in satellite data retrieval and assimilation. Furthermore, most of the channels are highly correlated and the pieces of independent information contained in the hyperspectral observations are usually much smaller than the number of channels. Principal component analysis (PCA) was used in this research to compress the observational information content contained in the Atmospheric Infrared Sounder (AIRS) channels to a few leading principal components (PCs). The corresponding PC scores were then assimilated into a PCA-based ensemble Kalman filter (EnKF) system. In this proof-of-concept study based on simulated observations, hyperspectral brightness temperatures were simulated using the atmospheric state vectors from convection-permitting ensemble simulations of Hurricane Harvey (2017) as input to the Community Radiative Transfer Model (CRTM). The PCs were derived from a preexisting training dataset of brightness temperatures calculated from convection-permitting simulation over a large domain in the Indian Ocean representing generic atmospheric conditions over tropical oceans. The EnKF increments from assimilating many individual measurements in the brightness temperature space were compared to the EnKF increments from assimilating significantly fewer numbers of leading PCs. Results showed that assimilating about 10–20 leading PCs could yield increments that were nearly indistinguishable to that from assimilating hyperspectral measurements from orders of magnitude larger number of hyperspectral channels.

UR - http://www.scopus.com/inward/record.url?scp=85075570770&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075570770&partnerID=8YFLogxK

U2 - 10.1175/MWR-D-18-0454.1

DO - 10.1175/MWR-D-18-0454.1

M3 - Review article

AN - SCOPUS:85075570770

VL - 147

SP - 3505

EP - 3518

JO - Monthly Weather Review

JF - Monthly Weather Review

SN - 0027-0644

IS - 10

ER -