Automating data-model workflows at a Level-12 HUC scale in a distributed computing environment

Lorne Neil Leonard, Christopher J. Duffy

Research output: Contribution to conferencePaper

1 Citation (Scopus)

Abstract

The HydroTerre web services provide the Essential Terrestrial Variable (ETV) datasets to create common hydrological models anywhere in the continental United States (CONUS). Th ese services allow web users to download data for their own purposes in their own computing environment. The datasets are provided using standard Geographic Information System formats and the data transformation is dependent on the users' own needs, goals, and computing environment. In this article, we demonstrate the feasibility of automating data-transformation workflows for United States Geological Survey level-12 Hydrological Unit Codes (HUC-12) to be consumed in hydrological models. The Penn State Integrated Hydrological Model (PIHM) is demonstrated here, but the workflows serve as a template for other models to adapt and become new services. The focus of this article is the data transformation process, not the model results. We want to demonstrate that workflows empower modelers to create hydrological models rapidly anywhere in the CONUS, and to contribute to a dynamic resource that records provenance of HUC-12 models. To do this, an explanation is required of both the hardware and software architecture because the way in which they are coupled is critical for web service performance. A demonstration of the feasibility to automate data-model workflows for CONUS HUC-12 catchments is discussed with the emphasis on reproducibility by using data-model workflows and distributed computing resources.

Original languageEnglish (US)
Pages154-161
Number of pages8
StatePublished - Jan 1 2014
Event7th International Congress on Environmental Modelling and Software, iEMSs 2014 - San Diego, United States
Duration: Jun 15 2014Jun 19 2014

Other

Other7th International Congress on Environmental Modelling and Software, iEMSs 2014
CountryUnited States
CitySan Diego
Period6/15/146/19/14

Fingerprint

Distributed computer systems
Distributed Computing
Data Model
Work Flow
Data structures
Data Transformation
Web Services
Web services
Model
Resources
Provenance
Hardware Architecture
Geographic Information Systems
Computing
Reproducibility
Integrated Model
Software Architecture
Geological surveys
Demonstrate
Software architecture

All Science Journal Classification (ASJC) codes

  • Software
  • Environmental Engineering
  • Modeling and Simulation

Cite this

Leonard, L. N., & Duffy, C. J. (2014). Automating data-model workflows at a Level-12 HUC scale in a distributed computing environment. 154-161. Paper presented at 7th International Congress on Environmental Modelling and Software, iEMSs 2014, San Diego, United States.
Leonard, Lorne Neil ; Duffy, Christopher J. / Automating data-model workflows at a Level-12 HUC scale in a distributed computing environment. Paper presented at 7th International Congress on Environmental Modelling and Software, iEMSs 2014, San Diego, United States.8 p.
@conference{4fdb70ab4f2646c1bac57a29e84aeed2,
title = "Automating data-model workflows at a Level-12 HUC scale in a distributed computing environment",
abstract = "The HydroTerre web services provide the Essential Terrestrial Variable (ETV) datasets to create common hydrological models anywhere in the continental United States (CONUS). Th ese services allow web users to download data for their own purposes in their own computing environment. The datasets are provided using standard Geographic Information System formats and the data transformation is dependent on the users' own needs, goals, and computing environment. In this article, we demonstrate the feasibility of automating data-transformation workflows for United States Geological Survey level-12 Hydrological Unit Codes (HUC-12) to be consumed in hydrological models. The Penn State Integrated Hydrological Model (PIHM) is demonstrated here, but the workflows serve as a template for other models to adapt and become new services. The focus of this article is the data transformation process, not the model results. We want to demonstrate that workflows empower modelers to create hydrological models rapidly anywhere in the CONUS, and to contribute to a dynamic resource that records provenance of HUC-12 models. To do this, an explanation is required of both the hardware and software architecture because the way in which they are coupled is critical for web service performance. A demonstration of the feasibility to automate data-model workflows for CONUS HUC-12 catchments is discussed with the emphasis on reproducibility by using data-model workflows and distributed computing resources.",
author = "Leonard, {Lorne Neil} and Duffy, {Christopher J.}",
year = "2014",
month = "1",
day = "1",
language = "English (US)",
pages = "154--161",
note = "7th International Congress on Environmental Modelling and Software, iEMSs 2014 ; Conference date: 15-06-2014 Through 19-06-2014",

}

Leonard, LN & Duffy, CJ 2014, 'Automating data-model workflows at a Level-12 HUC scale in a distributed computing environment', Paper presented at 7th International Congress on Environmental Modelling and Software, iEMSs 2014, San Diego, United States, 6/15/14 - 6/19/14 pp. 154-161.

Automating data-model workflows at a Level-12 HUC scale in a distributed computing environment. / Leonard, Lorne Neil; Duffy, Christopher J.

2014. 154-161 Paper presented at 7th International Congress on Environmental Modelling and Software, iEMSs 2014, San Diego, United States.

Research output: Contribution to conferencePaper

TY - CONF

T1 - Automating data-model workflows at a Level-12 HUC scale in a distributed computing environment

AU - Leonard, Lorne Neil

AU - Duffy, Christopher J.

PY - 2014/1/1

Y1 - 2014/1/1

N2 - The HydroTerre web services provide the Essential Terrestrial Variable (ETV) datasets to create common hydrological models anywhere in the continental United States (CONUS). Th ese services allow web users to download data for their own purposes in their own computing environment. The datasets are provided using standard Geographic Information System formats and the data transformation is dependent on the users' own needs, goals, and computing environment. In this article, we demonstrate the feasibility of automating data-transformation workflows for United States Geological Survey level-12 Hydrological Unit Codes (HUC-12) to be consumed in hydrological models. The Penn State Integrated Hydrological Model (PIHM) is demonstrated here, but the workflows serve as a template for other models to adapt and become new services. The focus of this article is the data transformation process, not the model results. We want to demonstrate that workflows empower modelers to create hydrological models rapidly anywhere in the CONUS, and to contribute to a dynamic resource that records provenance of HUC-12 models. To do this, an explanation is required of both the hardware and software architecture because the way in which they are coupled is critical for web service performance. A demonstration of the feasibility to automate data-model workflows for CONUS HUC-12 catchments is discussed with the emphasis on reproducibility by using data-model workflows and distributed computing resources.

AB - The HydroTerre web services provide the Essential Terrestrial Variable (ETV) datasets to create common hydrological models anywhere in the continental United States (CONUS). Th ese services allow web users to download data for their own purposes in their own computing environment. The datasets are provided using standard Geographic Information System formats and the data transformation is dependent on the users' own needs, goals, and computing environment. In this article, we demonstrate the feasibility of automating data-transformation workflows for United States Geological Survey level-12 Hydrological Unit Codes (HUC-12) to be consumed in hydrological models. The Penn State Integrated Hydrological Model (PIHM) is demonstrated here, but the workflows serve as a template for other models to adapt and become new services. The focus of this article is the data transformation process, not the model results. We want to demonstrate that workflows empower modelers to create hydrological models rapidly anywhere in the CONUS, and to contribute to a dynamic resource that records provenance of HUC-12 models. To do this, an explanation is required of both the hardware and software architecture because the way in which they are coupled is critical for web service performance. A demonstration of the feasibility to automate data-model workflows for CONUS HUC-12 catchments is discussed with the emphasis on reproducibility by using data-model workflows and distributed computing resources.

UR - http://www.scopus.com/inward/record.url?scp=84911913757&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84911913757&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:84911913757

SP - 154

EP - 161

ER -

Leonard LN, Duffy CJ. Automating data-model workflows at a Level-12 HUC scale in a distributed computing environment. 2014. Paper presented at 7th International Congress on Environmental Modelling and Software, iEMSs 2014, San Diego, United States.