The Cloud Hunter's problem: An automated decision algorithm to improve the productivity of scientific data collection in stochastic environments

Arthur A. Small, Jason B. Stefik, Johannes Verlinde, Nathaniel C. Johnson

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

A decision algorithm is presented that improves the productivity of data collection activities in stochastic environments. The algorithm was developed in the context of an aircraft field campaign organized to collect data in situ from boundary layer clouds. Required lead times implied that aircraft deployments had to be scheduled in advance, based on imperfect forecasts regarding the presence of conditions meeting specified requirements. Given an overall cap on the number of flights, daily fly/no-fly decisions were taken traditionally using a discussion-intensive process involving heuristic analysis of weather forecasts by a group of skilled human investigators. An alternative automated decision process uses self-organizing maps to convert weather forecasts into quantified probabilities of suitable conditions, together with a dynamic programming procedure to compute the opportunity costs of using up scarce flights from the limited budget. Applied to conditions prevailing during the 2009 Routine ARM Aerial Facility (AAF) Clouds with Low Optical Water Depths (CLOWD) Optical Radiative Observations (RACORO) campaign of the U.S. Department of Energy's Atmospheric Radiation Measurement Program, the algorithm shows a 21% increase in data yield and a 66% improvement in skill over the heuristic decision process used traditionally. The algorithmic approach promises to free up investigators' cognitive resources, reduce stress on flight crews, and increase productivity in a range of data collection applications.

Original languageEnglish (US)
Pages (from-to)2276-2289
Number of pages14
JournalMonthly Weather Review
Volume139
Issue number7
DOIs
StatePublished - Jul 1 2011

Fingerprint

productivity
flight
heuristics
aircraft
weather
water depth
boundary layer
decision
resource
cost
energy
forecast
decision process
budget
programme
radiation
analysis
in situ

All Science Journal Classification (ASJC) codes

  • Atmospheric Science

Cite this

@article{c9ac79fc4ae044e3a73438c125afd4ea,
title = "The Cloud Hunter's problem: An automated decision algorithm to improve the productivity of scientific data collection in stochastic environments",
abstract = "A decision algorithm is presented that improves the productivity of data collection activities in stochastic environments. The algorithm was developed in the context of an aircraft field campaign organized to collect data in situ from boundary layer clouds. Required lead times implied that aircraft deployments had to be scheduled in advance, based on imperfect forecasts regarding the presence of conditions meeting specified requirements. Given an overall cap on the number of flights, daily fly/no-fly decisions were taken traditionally using a discussion-intensive process involving heuristic analysis of weather forecasts by a group of skilled human investigators. An alternative automated decision process uses self-organizing maps to convert weather forecasts into quantified probabilities of suitable conditions, together with a dynamic programming procedure to compute the opportunity costs of using up scarce flights from the limited budget. Applied to conditions prevailing during the 2009 Routine ARM Aerial Facility (AAF) Clouds with Low Optical Water Depths (CLOWD) Optical Radiative Observations (RACORO) campaign of the U.S. Department of Energy's Atmospheric Radiation Measurement Program, the algorithm shows a 21{\%} increase in data yield and a 66{\%} improvement in skill over the heuristic decision process used traditionally. The algorithmic approach promises to free up investigators' cognitive resources, reduce stress on flight crews, and increase productivity in a range of data collection applications.",
author = "Small, {Arthur A.} and Stefik, {Jason B.} and Johannes Verlinde and Johnson, {Nathaniel C.}",
year = "2011",
month = "7",
day = "1",
doi = "10.1175/2010MWR3576.1",
language = "English (US)",
volume = "139",
pages = "2276--2289",
journal = "Monthly Weather Review",
issn = "0027-0644",
publisher = "American Meteorological Society",
number = "7",

}

The Cloud Hunter's problem : An automated decision algorithm to improve the productivity of scientific data collection in stochastic environments. / Small, Arthur A.; Stefik, Jason B.; Verlinde, Johannes; Johnson, Nathaniel C.

In: Monthly Weather Review, Vol. 139, No. 7, 01.07.2011, p. 2276-2289.

Research output: Contribution to journalArticle

TY - JOUR

T1 - The Cloud Hunter's problem

T2 - An automated decision algorithm to improve the productivity of scientific data collection in stochastic environments

AU - Small, Arthur A.

AU - Stefik, Jason B.

AU - Verlinde, Johannes

AU - Johnson, Nathaniel C.

PY - 2011/7/1

Y1 - 2011/7/1

N2 - A decision algorithm is presented that improves the productivity of data collection activities in stochastic environments. The algorithm was developed in the context of an aircraft field campaign organized to collect data in situ from boundary layer clouds. Required lead times implied that aircraft deployments had to be scheduled in advance, based on imperfect forecasts regarding the presence of conditions meeting specified requirements. Given an overall cap on the number of flights, daily fly/no-fly decisions were taken traditionally using a discussion-intensive process involving heuristic analysis of weather forecasts by a group of skilled human investigators. An alternative automated decision process uses self-organizing maps to convert weather forecasts into quantified probabilities of suitable conditions, together with a dynamic programming procedure to compute the opportunity costs of using up scarce flights from the limited budget. Applied to conditions prevailing during the 2009 Routine ARM Aerial Facility (AAF) Clouds with Low Optical Water Depths (CLOWD) Optical Radiative Observations (RACORO) campaign of the U.S. Department of Energy's Atmospheric Radiation Measurement Program, the algorithm shows a 21% increase in data yield and a 66% improvement in skill over the heuristic decision process used traditionally. The algorithmic approach promises to free up investigators' cognitive resources, reduce stress on flight crews, and increase productivity in a range of data collection applications.

AB - A decision algorithm is presented that improves the productivity of data collection activities in stochastic environments. The algorithm was developed in the context of an aircraft field campaign organized to collect data in situ from boundary layer clouds. Required lead times implied that aircraft deployments had to be scheduled in advance, based on imperfect forecasts regarding the presence of conditions meeting specified requirements. Given an overall cap on the number of flights, daily fly/no-fly decisions were taken traditionally using a discussion-intensive process involving heuristic analysis of weather forecasts by a group of skilled human investigators. An alternative automated decision process uses self-organizing maps to convert weather forecasts into quantified probabilities of suitable conditions, together with a dynamic programming procedure to compute the opportunity costs of using up scarce flights from the limited budget. Applied to conditions prevailing during the 2009 Routine ARM Aerial Facility (AAF) Clouds with Low Optical Water Depths (CLOWD) Optical Radiative Observations (RACORO) campaign of the U.S. Department of Energy's Atmospheric Radiation Measurement Program, the algorithm shows a 21% increase in data yield and a 66% improvement in skill over the heuristic decision process used traditionally. The algorithmic approach promises to free up investigators' cognitive resources, reduce stress on flight crews, and increase productivity in a range of data collection applications.

UR - http://www.scopus.com/inward/record.url?scp=79960715029&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79960715029&partnerID=8YFLogxK

U2 - 10.1175/2010MWR3576.1

DO - 10.1175/2010MWR3576.1

M3 - Article

AN - SCOPUS:79960715029

VL - 139

SP - 2276

EP - 2289

JO - Monthly Weather Review

JF - Monthly Weather Review

SN - 0027-0644

IS - 7

ER -