Locality optimization algorithms for compilation of out-of-core codes

M. Kandemir, A. Choudhary, J. Ramanujam, M. Kandaswamy

Research output: Contribution to journalArticle

7 Scopus citations

Abstract

The difficulty of handling out-of-core data limits the performance of supercomputers as well as the potential of parallel machines. Since writing an efficient out-of-core version of a program is a difficult tasks, and since virtual memory systems do not perform well on scientific computations, we believe that there is a clear need for a compiler-directed explicit I/O approach for out-of-core computations. In this paper, we present a compiler algorithm to optimize the locality of disk accesses in out-of-core codes by choosing a good combination of file layouts on disks and loop transformations. The transformations change the access order of array data. Our solution goes beyond the single-loop level (called global I/O optimization in this paper) for out-of-core computations. Since the general problem is NP-complete, we present efficient heuristics that produce near-optimal solutions for several programs encountered in practice. Experimental results obtained on an IBM SP-2 and Intel Paragon provide encouraging evidence that our approach is successful at optimizing programs which depend on disk-resident data in distributed-memory machines.

Original languageEnglish (US)
Pages (from-to)107-138
Number of pages32
JournalJournal of Information Science and Engineering
Volume14
Issue number1
StatePublished - Mar 1998

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Hardware and Architecture
  • Library and Information Sciences
  • Computational Theory and Mathematics

Fingerprint Dive into the research topics of 'Locality optimization algorithms for compilation of out-of-core codes'. Together they form a unique fingerprint.

  • Cite this