Tuning Heterogeneous Computing Platforms for Large-Scale Hydrology Data Management

Lorne Leonard, Kamesh Madduri, Christopher J. Duffy

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

HydroTerre is a research prototype platform developed at Penn State for the hydrology community. It provides access to aggregated scientific data sets that are useful for hydrological modeling and research. HydroTerre's frontend is a web service, and a user query can request creation of a data bundle whose size can vary from a few megabytes to 100's of gigabytes. In this article, we present software tuning and optimization strategies for various hardware configurations of the HydroTerre platform. Our goal is to minimize access time to a wide range of data bundle creation queries from users. We use automated schemes to estimate the computational work required for various queries, and identify the best-performing hardware/software configuration. We hope this study is instructive for researchers developing similar data management cyberinfrastructure in other science and engineering fields.

Original languageEnglish (US)
Article number7327220
Pages (from-to)2753-2765
Number of pages13
JournalIEEE Transactions on Parallel and Distributed Systems
Volume27
Issue number9
DOIs
StatePublished - Sep 1 2016

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics

Fingerprint Dive into the research topics of 'Tuning Heterogeneous Computing Platforms for Large-Scale Hydrology Data Management'. Together they form a unique fingerprint.

Cite this