Automated tracing of I/O stack

Seong Jo Kim, Yuanrui Zhang, Seung Woo Son, Ramya Prabhakar, Mahmut Kandemir, Christina Patrick, Wei Keng Liao, Alok Choudhary

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

Efficient execution of parallel scientific applications requires high-performance storage systems designed to meet their I/O requirements. Most high-performance I/O intensive applications access multiple layers of the storage stack during their disk operations. A typical I/O request from these applications may include accesses to high-level libraries such as MPI I/O, executing on clustered parallel file systems like PVFS2, which are in turn supported by native file systems like Linux. In order to design and implement parallel applications that exercise this I/O stack, it is important to understand the flow of I/O calls through the entire storage system. Such understanding helps in identifying the potential performance and power bottlenecks in different layers of the storage hierarchy. To trace the execution of the I/O calls and to understand the complex interactions of multiple user-libraries and file systems, we propose an automatic code instrumentation technique, which enables us to collect detailed statistics of the I/O stack. Our proposed I/O tracing tool traces the flow of I/O calls across different layers of an I/O stack, and can be configured to work with different file systems and user-libraries. It also analyzes the collected information to generate output in terms of different user-specified metrics of interest.

Original languageEnglish (US)
Title of host publicationRecent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings
Pages72-81
Number of pages10
DOIs
StatePublished - Nov 12 2010
Event17th European MPI Users' Group Meeting, EuroMPI 2010 - Stuttgart, Germany
Duration: Sep 12 2010Sep 15 2010

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume6305 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other17th European MPI Users' Group Meeting, EuroMPI 2010
CountryGermany
CityStuttgart
Period9/12/109/15/10

Fingerprint

Tracing
File System
Storage System
HPSS
High Performance
Trace
Parallel File System
Parallel Applications
Multiple Access
Linux
Instrumentation
Exercise
Statistics
Entire
Metric
Output
Requirements
Interaction
Libraries

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Kim, S. J., Zhang, Y., Son, S. W., Prabhakar, R., Kandemir, M., Patrick, C., ... Choudhary, A. (2010). Automated tracing of I/O stack. In Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings (pp. 72-81). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6305 LNCS). https://doi.org/10.1007/978-3-642-15646-5_8
Kim, Seong Jo ; Zhang, Yuanrui ; Son, Seung Woo ; Prabhakar, Ramya ; Kandemir, Mahmut ; Patrick, Christina ; Liao, Wei Keng ; Choudhary, Alok. / Automated tracing of I/O stack. Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings. 2010. pp. 72-81 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{6828434903ab42068b09669d6f67a73c,
title = "Automated tracing of I/O stack",
abstract = "Efficient execution of parallel scientific applications requires high-performance storage systems designed to meet their I/O requirements. Most high-performance I/O intensive applications access multiple layers of the storage stack during their disk operations. A typical I/O request from these applications may include accesses to high-level libraries such as MPI I/O, executing on clustered parallel file systems like PVFS2, which are in turn supported by native file systems like Linux. In order to design and implement parallel applications that exercise this I/O stack, it is important to understand the flow of I/O calls through the entire storage system. Such understanding helps in identifying the potential performance and power bottlenecks in different layers of the storage hierarchy. To trace the execution of the I/O calls and to understand the complex interactions of multiple user-libraries and file systems, we propose an automatic code instrumentation technique, which enables us to collect detailed statistics of the I/O stack. Our proposed I/O tracing tool traces the flow of I/O calls across different layers of an I/O stack, and can be configured to work with different file systems and user-libraries. It also analyzes the collected information to generate output in terms of different user-specified metrics of interest.",
author = "Kim, {Seong Jo} and Yuanrui Zhang and Son, {Seung Woo} and Ramya Prabhakar and Mahmut Kandemir and Christina Patrick and Liao, {Wei Keng} and Alok Choudhary",
year = "2010",
month = "11",
day = "12",
doi = "10.1007/978-3-642-15646-5_8",
language = "English (US)",
isbn = "3642156452",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "72--81",
booktitle = "Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings",

}

Kim, SJ, Zhang, Y, Son, SW, Prabhakar, R, Kandemir, M, Patrick, C, Liao, WK & Choudhary, A 2010, Automated tracing of I/O stack. in Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6305 LNCS, pp. 72-81, 17th European MPI Users' Group Meeting, EuroMPI 2010, Stuttgart, Germany, 9/12/10. https://doi.org/10.1007/978-3-642-15646-5_8

Automated tracing of I/O stack. / Kim, Seong Jo; Zhang, Yuanrui; Son, Seung Woo; Prabhakar, Ramya; Kandemir, Mahmut; Patrick, Christina; Liao, Wei Keng; Choudhary, Alok.

Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings. 2010. p. 72-81 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6305 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Automated tracing of I/O stack

AU - Kim, Seong Jo

AU - Zhang, Yuanrui

AU - Son, Seung Woo

AU - Prabhakar, Ramya

AU - Kandemir, Mahmut

AU - Patrick, Christina

AU - Liao, Wei Keng

AU - Choudhary, Alok

PY - 2010/11/12

Y1 - 2010/11/12

N2 - Efficient execution of parallel scientific applications requires high-performance storage systems designed to meet their I/O requirements. Most high-performance I/O intensive applications access multiple layers of the storage stack during their disk operations. A typical I/O request from these applications may include accesses to high-level libraries such as MPI I/O, executing on clustered parallel file systems like PVFS2, which are in turn supported by native file systems like Linux. In order to design and implement parallel applications that exercise this I/O stack, it is important to understand the flow of I/O calls through the entire storage system. Such understanding helps in identifying the potential performance and power bottlenecks in different layers of the storage hierarchy. To trace the execution of the I/O calls and to understand the complex interactions of multiple user-libraries and file systems, we propose an automatic code instrumentation technique, which enables us to collect detailed statistics of the I/O stack. Our proposed I/O tracing tool traces the flow of I/O calls across different layers of an I/O stack, and can be configured to work with different file systems and user-libraries. It also analyzes the collected information to generate output in terms of different user-specified metrics of interest.

AB - Efficient execution of parallel scientific applications requires high-performance storage systems designed to meet their I/O requirements. Most high-performance I/O intensive applications access multiple layers of the storage stack during their disk operations. A typical I/O request from these applications may include accesses to high-level libraries such as MPI I/O, executing on clustered parallel file systems like PVFS2, which are in turn supported by native file systems like Linux. In order to design and implement parallel applications that exercise this I/O stack, it is important to understand the flow of I/O calls through the entire storage system. Such understanding helps in identifying the potential performance and power bottlenecks in different layers of the storage hierarchy. To trace the execution of the I/O calls and to understand the complex interactions of multiple user-libraries and file systems, we propose an automatic code instrumentation technique, which enables us to collect detailed statistics of the I/O stack. Our proposed I/O tracing tool traces the flow of I/O calls across different layers of an I/O stack, and can be configured to work with different file systems and user-libraries. It also analyzes the collected information to generate output in terms of different user-specified metrics of interest.

UR - http://www.scopus.com/inward/record.url?scp=78149245030&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78149245030&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-15646-5_8

DO - 10.1007/978-3-642-15646-5_8

M3 - Conference contribution

AN - SCOPUS:78149245030

SN - 3642156452

SN - 9783642156458

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 72

EP - 81

BT - Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings

ER -

Kim SJ, Zhang Y, Son SW, Prabhakar R, Kandemir M, Patrick C et al. Automated tracing of I/O stack. In Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings. 2010. p. 72-81. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-15646-5_8