Global I/O optimizations for out-of-core computations

Mahmut Kandemir, Meena Kandaswamy, Alok Choudhary

Research output: Contribution to conferencePaperpeer-review

2 Scopus citations

Abstract

The use of parallel machines to solve large scale computational problems in science and engineering has increased considerably in recent times. Many of these problems have computational requirements which stretch the capabilities of even the fastest machine available today. In addition to requiring a great deal of computational power, these problems usually deal with large quantities of data up to a few terabytes. The main memory sizes of current parallel machines do not even come close to matching these requirements; hence data needs to be stored on disks and fetched during the execution of the program. Unfortunately, current optimizing compilers for parallel machines provide support only for in-core computations in which the data sets can fit into memory. This limitation severely affects the performance of programs which depend on disk resident data. Our previous research demonstrated that file layout optimizations are extremely important for optimizing such programs. In this paper we investigate solutions to the global I/O optimization problem for out-of-core computations. Since the general problem is NP-complete, we present fast heuristics that can result in near-optimal solutions for the programs encountered in practice. Preliminary results provide encouraging evidence that our algorithms can be successful in optimizing out-of-core programs.

Original languageEnglish (US)
Pages401-406
Number of pages6
StatePublished - Dec 1 1997
EventProceedings of the 1997 4th International Conference on High Performance Computing, HiPC - Bangalore, India
Duration: Dec 18 1997Dec 21 1997

Other

OtherProceedings of the 1997 4th International Conference on High Performance Computing, HiPC
CityBangalore, India
Period12/18/9712/21/97

All Science Journal Classification (ASJC) codes

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Global I/O optimizations for out-of-core computations'. Together they form a unique fingerprint.

Cite this