This paper describes optimization techniques for translating out-of-core programs written in a data parallel language to message passing node programs with explicit parallel I/O. We demonstrate that straightforward extension of in-core compilation techniques does not work well for out-of-core programs. We then describe how the compiler can optimize the code by (1) determining appropriate file layouts for out-of-core arrays, (2) permuting the loops in the nest(s) to allow efficient file access, and (3) partitioning the available node memory among references based on I/O cost estimation. Our experimental results indicate that these optimizations can reduce the amount of time spent in I/O by as much as an order of magnitude.
|Original language||English (US)|
|Number of pages||6|
|Journal||Proceedings of the International Parallel Processing Symposium, IPPS|
|State||Published - 1997|
All Science Journal Classification (ASJC) codes
- Hardware and Architecture