Decoding the precision of historical temperature observations

Andrew Rhines, Martin P. Tingley, Karen A. Mckinnon, Peter Huybers

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

Historical observations of temperature underpin our ability to monitor Earth's climate. We identify a pervasive issue in archived observations from surface stations, wherein the use of varying conventions for units and precision has led to distorted distributions of the data. Apart from the original precision being generally unknown, the majority of archived temperature data are found to be misaligned with the original measurements because of rounding on a Fahrenheit scale, conversion to Celsius, and re-rounding. Furthermore, we show that commonly used statistical methods including quantile regression are sensitive to the finite precision and to double-rounding of the data after unit conversion. To remedy these issues, we present a Hidden Markov Model that uses the differing frequencies of specific recorded values to recover the most likely original precision and units associated with each observation. This precision-decoding algorithm is used to infer the precision of the 644 million daily surface temperature observations in the Global Historical Climate Network database, providing more accurate values for the 63% of samples found to have been biased by double-rounding. The average absolute bias correction across the dataset is 0.018 °C, and the average inferred precision is 0.41 °C, even though data are archived at 0.1 °C precision. These results permit better inference of when record temperatures occurred, correction of rounding effects, and identification of inhomogeneities in surface temperature time series, amongst other applications. The precision-decoding algorithm is generally applicable to rounded observations-including surface pressure, humidity, precipitation, and other temperature data-thereby offering the potential to improve quality-control procedures for many datasets.

Original languageEnglish (US)
Pages (from-to)2923-2933
Number of pages11
JournalQuarterly Journal of the Royal Meteorological Society
Volume141
Issue number693
DOIs
StatePublished - Oct 1 2015

Fingerprint

temperature
surface temperature
climate
surface pressure
inhomogeneity
quality control
humidity
time series

All Science Journal Classification (ASJC) codes

  • Atmospheric Science

Cite this

Rhines, Andrew ; Tingley, Martin P. ; Mckinnon, Karen A. ; Huybers, Peter. / Decoding the precision of historical temperature observations. In: Quarterly Journal of the Royal Meteorological Society. 2015 ; Vol. 141, No. 693. pp. 2923-2933.
@article{ae7657d1091942559adbd038efb069e8,
title = "Decoding the precision of historical temperature observations",
abstract = "Historical observations of temperature underpin our ability to monitor Earth's climate. We identify a pervasive issue in archived observations from surface stations, wherein the use of varying conventions for units and precision has led to distorted distributions of the data. Apart from the original precision being generally unknown, the majority of archived temperature data are found to be misaligned with the original measurements because of rounding on a Fahrenheit scale, conversion to Celsius, and re-rounding. Furthermore, we show that commonly used statistical methods including quantile regression are sensitive to the finite precision and to double-rounding of the data after unit conversion. To remedy these issues, we present a Hidden Markov Model that uses the differing frequencies of specific recorded values to recover the most likely original precision and units associated with each observation. This precision-decoding algorithm is used to infer the precision of the 644 million daily surface temperature observations in the Global Historical Climate Network database, providing more accurate values for the 63{\%} of samples found to have been biased by double-rounding. The average absolute bias correction across the dataset is 0.018 °C, and the average inferred precision is 0.41 °C, even though data are archived at 0.1 °C precision. These results permit better inference of when record temperatures occurred, correction of rounding effects, and identification of inhomogeneities in surface temperature time series, amongst other applications. The precision-decoding algorithm is generally applicable to rounded observations-including surface pressure, humidity, precipitation, and other temperature data-thereby offering the potential to improve quality-control procedures for many datasets.",
author = "Andrew Rhines and Tingley, {Martin P.} and Mckinnon, {Karen A.} and Peter Huybers",
year = "2015",
month = "10",
day = "1",
doi = "10.1002/qj.2612",
language = "English (US)",
volume = "141",
pages = "2923--2933",
journal = "Quarterly Journal of the Royal Meteorological Society",
issn = "0035-9009",
publisher = "John Wiley and Sons Ltd",
number = "693",

}

Decoding the precision of historical temperature observations. / Rhines, Andrew; Tingley, Martin P.; Mckinnon, Karen A.; Huybers, Peter.

In: Quarterly Journal of the Royal Meteorological Society, Vol. 141, No. 693, 01.10.2015, p. 2923-2933.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Decoding the precision of historical temperature observations

AU - Rhines, Andrew

AU - Tingley, Martin P.

AU - Mckinnon, Karen A.

AU - Huybers, Peter

PY - 2015/10/1

Y1 - 2015/10/1

N2 - Historical observations of temperature underpin our ability to monitor Earth's climate. We identify a pervasive issue in archived observations from surface stations, wherein the use of varying conventions for units and precision has led to distorted distributions of the data. Apart from the original precision being generally unknown, the majority of archived temperature data are found to be misaligned with the original measurements because of rounding on a Fahrenheit scale, conversion to Celsius, and re-rounding. Furthermore, we show that commonly used statistical methods including quantile regression are sensitive to the finite precision and to double-rounding of the data after unit conversion. To remedy these issues, we present a Hidden Markov Model that uses the differing frequencies of specific recorded values to recover the most likely original precision and units associated with each observation. This precision-decoding algorithm is used to infer the precision of the 644 million daily surface temperature observations in the Global Historical Climate Network database, providing more accurate values for the 63% of samples found to have been biased by double-rounding. The average absolute bias correction across the dataset is 0.018 °C, and the average inferred precision is 0.41 °C, even though data are archived at 0.1 °C precision. These results permit better inference of when record temperatures occurred, correction of rounding effects, and identification of inhomogeneities in surface temperature time series, amongst other applications. The precision-decoding algorithm is generally applicable to rounded observations-including surface pressure, humidity, precipitation, and other temperature data-thereby offering the potential to improve quality-control procedures for many datasets.

AB - Historical observations of temperature underpin our ability to monitor Earth's climate. We identify a pervasive issue in archived observations from surface stations, wherein the use of varying conventions for units and precision has led to distorted distributions of the data. Apart from the original precision being generally unknown, the majority of archived temperature data are found to be misaligned with the original measurements because of rounding on a Fahrenheit scale, conversion to Celsius, and re-rounding. Furthermore, we show that commonly used statistical methods including quantile regression are sensitive to the finite precision and to double-rounding of the data after unit conversion. To remedy these issues, we present a Hidden Markov Model that uses the differing frequencies of specific recorded values to recover the most likely original precision and units associated with each observation. This precision-decoding algorithm is used to infer the precision of the 644 million daily surface temperature observations in the Global Historical Climate Network database, providing more accurate values for the 63% of samples found to have been biased by double-rounding. The average absolute bias correction across the dataset is 0.018 °C, and the average inferred precision is 0.41 °C, even though data are archived at 0.1 °C precision. These results permit better inference of when record temperatures occurred, correction of rounding effects, and identification of inhomogeneities in surface temperature time series, amongst other applications. The precision-decoding algorithm is generally applicable to rounded observations-including surface pressure, humidity, precipitation, and other temperature data-thereby offering the potential to improve quality-control procedures for many datasets.

UR - http://www.scopus.com/inward/record.url?scp=84952299914&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84952299914&partnerID=8YFLogxK

U2 - 10.1002/qj.2612

DO - 10.1002/qj.2612

M3 - Article

AN - SCOPUS:84952299914

VL - 141

SP - 2923

EP - 2933

JO - Quarterly Journal of the Royal Meteorological Society

JF - Quarterly Journal of the Royal Meteorological Society

SN - 0035-9009

IS - 693

ER -