A collective I/O scheme based on compiler analysis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Current approaches to parallel I/O demand extensive user effort to obtain acceptable performance. This is in part due to difficulties in understanding the characteristics of a wide variety of I/O devices and in part due to inherent complexity of I/O software. While parallel I/O systems provide users with environments where persistent datasets can be shared between parallel processors, the ultimate performance of I/O-intensive codes depends largely on the relation between data access patterns and storage patterns of data in files and on disks. In cases where access patterns and storage patterns match, we can exploit parallel I/O hardware by allowing each processor to perform independent parallel I/O. To handle the cases in which data access patterns and storage patterns do not match, several I/O optimization techniques have been developed in recent years. Collective I/O is such an optimization technique that enables each processor to do I/O on behalf of other processors if doing so improves the overall performance. While it is generally accepted that collective I/O and its variants can bring impressive improvements as far as the I/O performance is concerned, it is difficult for the programmer to use collective I/O in an optimal manner. In this paper, we propose and evaluate a compiler-directed collective I/O approach which detects the opportunities for collective I/O and inserts the necessary I/O calls in the code automatically. An important characteristic of the approach is that instead of applying collective I/O indiscriminately, it uses collective I/O selectively, only in cases where independent parallel I/O would not be possible. We have implemented the necessary algorithms in a source-to-source translator and within a stand-alone tool. Our experimental results demonstrate that our compiler-directed collective I/O scheme performs very well on different setups built using nine applications from several scientific benchmarks.

Original languageEnglish (US)
Title of host publicationLanguages, Compilers, and Run-Time Systems for Scalable Computers - 5th International Workshop, LCR 2000, Selected Papers
EditorsSandhya Dwarkadas
PublisherSpringer Verlag
Pages1-15
Number of pages15
ISBN (Print)3540411852, 9783540411857
DOIs
StatePublished - Jan 1 2000
Event5th International Workshop on Languages, Compilers, and Run-Time Systems for Scalable Computers, LCR 2000 - Rochester, United States
Duration: May 25 2000May 27 2000

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume1915
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other5th International Workshop on Languages, Compilers, and Run-Time Systems for Scalable Computers, LCR 2000
CountryUnited States
CityRochester
Period5/25/005/27/00

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'A collective I/O scheme based on compiler analysis'. Together they form a unique fingerprint.

  • Cite this

    Kandemir, M. T. (2000). A collective I/O scheme based on compiler analysis. In S. Dwarkadas (Ed.), Languages, Compilers, and Run-Time Systems for Scalable Computers - 5th International Workshop, LCR 2000, Selected Papers (pp. 1-15). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 1915). Springer Verlag. https://doi.org/10.1007/3-540-40889-4_1