TY - JOUR
T1 - Tuning Heterogeneous Computing Platforms for Large-Scale Hydrology Data Management
AU - Leonard, Lorne
AU - Madduri, Kamesh
AU - Duffy, Christopher J.
N1 - Funding Information:
This research was supported in part by the National Science Foundation through XSEDE resources provided by the XSEDE Science Gateways program (TG-EAR120019), NSF EarthCube (GEO-44417482), NSF INSPIRE (IIS-1344272), EPA (96305901), NOAA (NA10OAR4310166). This research was also supported in part by US National Science Foundation (NSF) grant ACI-1253881. The authors would like to acknowledge support from the Institute for CyberScience director Padma Raghavan and Penn State Institutes for Energy and the Environment director Tom Richard at The Pennsylvania State University.
Publisher Copyright:
© 1990-2012 IEEE.
PY - 2016/9/1
Y1 - 2016/9/1
N2 - HydroTerre is a research prototype platform developed at Penn State for the hydrology community. It provides access to aggregated scientific data sets that are useful for hydrological modeling and research. HydroTerre's frontend is a web service, and a user query can request creation of a data bundle whose size can vary from a few megabytes to 100's of gigabytes. In this article, we present software tuning and optimization strategies for various hardware configurations of the HydroTerre platform. Our goal is to minimize access time to a wide range of data bundle creation queries from users. We use automated schemes to estimate the computational work required for various queries, and identify the best-performing hardware/software configuration. We hope this study is instructive for researchers developing similar data management cyberinfrastructure in other science and engineering fields.
AB - HydroTerre is a research prototype platform developed at Penn State for the hydrology community. It provides access to aggregated scientific data sets that are useful for hydrological modeling and research. HydroTerre's frontend is a web service, and a user query can request creation of a data bundle whose size can vary from a few megabytes to 100's of gigabytes. In this article, we present software tuning and optimization strategies for various hardware configurations of the HydroTerre platform. Our goal is to minimize access time to a wide range of data bundle creation queries from users. We use automated schemes to estimate the computational work required for various queries, and identify the best-performing hardware/software configuration. We hope this study is instructive for researchers developing similar data management cyberinfrastructure in other science and engineering fields.
UR - http://www.scopus.com/inward/record.url?scp=84982170362&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84982170362&partnerID=8YFLogxK
U2 - 10.1109/TPDS.2015.2499741
DO - 10.1109/TPDS.2015.2499741
M3 - Article
AN - SCOPUS:84982170362
SN - 1045-9219
VL - 27
SP - 2753
EP - 2765
JO - IEEE Transactions on Parallel and Distributed Systems
JF - IEEE Transactions on Parallel and Distributed Systems
IS - 9
M1 - 7327220
ER -