Novel application development environment for large-scale scientific computations

X. Shen, W. Liao, A. Choudhary, G. Memik, M. Kandemir, S. More, G. Thiruvathukal, A. Singh

Research output: Contribution to conferencePaper

8 Scopus citations

Abstract

Effective high-level data management is becoming an important issue with more and more scientific applications manipulating huge amounts of secondary-storage and tertiary-storage data using parallel processors. A major problem facing the current solutions to this data management problem is that these solutions either require a deep understanding of specific data storage architectures and file layouts to obtain the best performance. In this paper, we discuss the design, implementation, and evaluation of a novel application development environment for scientific computations. This environment includes a number of components that make it easy for the programmers to code and run their applications without much programming effort, and at the same time, to harness the available computational and storage power on parallel architectures. Embarking on this ambitious goal, we first present a performance-oriented meta-data management system that governs data flow between storage devices and applications. Another component of our environment is a data analysis and visualization tool which has been integrated with the meta-data management system, storage subsystem, and user applications. We also present an automatic code generator component (ACG) to help users utilize the information in the meta-data management system when they are developing new applications. All these components are tied together using an integrated Java graphical user interface (IJ-GUI) through which the user can launch her applications, can query the meta-data management system to obtain accurate information about the datasets she is interested in and about the current state of the storage devices, and can carry out data analysis and visualization, all in a unified framework. Finally, we present performance numbers from our initial implementation. Our results demonstrate that our novel application development environment provides both ease-of-use and high performance for large-scale, I/O-intensive scientific applications.

Original languageEnglish (US)
Pages274-283
Number of pages10
StatePublished - Jan 1 2000
Event2000 International Conference on Supercomputing - Santa Fe, NM, USA
Duration: May 8 2000May 11 2000

Other

Other2000 International Conference on Supercomputing
CitySanta Fe, NM, USA
Period5/8/005/11/00

All Science Journal Classification (ASJC) codes

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Novel application development environment for large-scale scientific computations'. Together they form a unique fingerprint.

  • Cite this

    Shen, X., Liao, W., Choudhary, A., Memik, G., Kandemir, M., More, S., Thiruvathukal, G., & Singh, A. (2000). Novel application development environment for large-scale scientific computations. 274-283. Paper presented at 2000 International Conference on Supercomputing, Santa Fe, NM, USA, .