On the performance of the POSIX I/O interface to PVFS

Murali Vilayannur, Robert B. Ross, Philip H. Carns, Rajeev Thakur, Anand Sivasubramaniam, Mahmut Kandemir

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

The ever-increasing gap in performance between CPU/memory technologies and the I/O subsystem (disks, I/O buses) in modern workstations has exacerbated the I/O bottlenecks inherent in applications that access large disk resident data sets. A common technique to alleviate the I/O bottlenecks on clusters of work-stations, is the use of parallel file systems. One such parallel file system is the Parallel Virtual File System (PVFS), which is a freely available tool to achieve high-performance I/O on Linux-based clusters. In this paper, we describe the performance and scalability of the UNIX I/O interface to PVFS. To illustrate the performance, we present experimental results using Bonnie++, a commonly used file system benchmark to test file system throughput; a synthetic parallel I/O application for calculating aggregate read and write bandwidths; and a synthetic benchmark which calculates the time taken to untar the Linux kernel source tree to measure performance of a large number of small file operations. We obtained aggregate read and write bandwidths as high as 550 MB/s with a Myrinet-based network and 160MB/S with fast Ethernet.

Original languageEnglish (US)
Title of host publicationProceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004
Pages332-339
Number of pages8
DOIs
StatePublished - Jul 12 2004
EventProceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004 - A Coruna, Spain
Duration: Feb 11 2004Feb 13 2004

Publication series

NameProceedings - Euromicro Conference on Parellel, Distribeted and Network-based Proceeding

Other

OtherProceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004
CountrySpain
CityA Coruna
Period2/11/042/13/04

Fingerprint

Interfaces (computer)
Bandwidth
UNIX
Ethernet
Program processors
Scalability
Throughput
Data storage equipment
Linux

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Cite this

Vilayannur, M., Ross, R. B., Carns, P. H., Thakur, R., Sivasubramaniam, A., & Kandemir, M. (2004). On the performance of the POSIX I/O interface to PVFS. In Proceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004 (pp. 332-339). (Proceedings - Euromicro Conference on Parellel, Distribeted and Network-based Proceeding). https://doi.org/10.1109/EMPDP.2004.1271463
Vilayannur, Murali ; Ross, Robert B. ; Carns, Philip H. ; Thakur, Rajeev ; Sivasubramaniam, Anand ; Kandemir, Mahmut. / On the performance of the POSIX I/O interface to PVFS. Proceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004. 2004. pp. 332-339 (Proceedings - Euromicro Conference on Parellel, Distribeted and Network-based Proceeding).
@inproceedings{e7220b6857114e3fab2b093fc9eb7495,
title = "On the performance of the POSIX I/O interface to PVFS",
abstract = "The ever-increasing gap in performance between CPU/memory technologies and the I/O subsystem (disks, I/O buses) in modern workstations has exacerbated the I/O bottlenecks inherent in applications that access large disk resident data sets. A common technique to alleviate the I/O bottlenecks on clusters of work-stations, is the use of parallel file systems. One such parallel file system is the Parallel Virtual File System (PVFS), which is a freely available tool to achieve high-performance I/O on Linux-based clusters. In this paper, we describe the performance and scalability of the UNIX I/O interface to PVFS. To illustrate the performance, we present experimental results using Bonnie++, a commonly used file system benchmark to test file system throughput; a synthetic parallel I/O application for calculating aggregate read and write bandwidths; and a synthetic benchmark which calculates the time taken to untar the Linux kernel source tree to measure performance of a large number of small file operations. We obtained aggregate read and write bandwidths as high as 550 MB/s with a Myrinet-based network and 160MB/S with fast Ethernet.",
author = "Murali Vilayannur and Ross, {Robert B.} and Carns, {Philip H.} and Rajeev Thakur and Anand Sivasubramaniam and Mahmut Kandemir",
year = "2004",
month = "7",
day = "12",
doi = "10.1109/EMPDP.2004.1271463",
language = "English (US)",
isbn = "0769520839",
series = "Proceedings - Euromicro Conference on Parellel, Distribeted and Network-based Proceeding",
pages = "332--339",
booktitle = "Proceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004",

}

Vilayannur, M, Ross, RB, Carns, PH, Thakur, R, Sivasubramaniam, A & Kandemir, M 2004, On the performance of the POSIX I/O interface to PVFS. in Proceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004. Proceedings - Euromicro Conference on Parellel, Distribeted and Network-based Proceeding, pp. 332-339, Proceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004, A Coruna, Spain, 2/11/04. https://doi.org/10.1109/EMPDP.2004.1271463

On the performance of the POSIX I/O interface to PVFS. / Vilayannur, Murali; Ross, Robert B.; Carns, Philip H.; Thakur, Rajeev; Sivasubramaniam, Anand; Kandemir, Mahmut.

Proceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004. 2004. p. 332-339 (Proceedings - Euromicro Conference on Parellel, Distribeted and Network-based Proceeding).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - On the performance of the POSIX I/O interface to PVFS

AU - Vilayannur, Murali

AU - Ross, Robert B.

AU - Carns, Philip H.

AU - Thakur, Rajeev

AU - Sivasubramaniam, Anand

AU - Kandemir, Mahmut

PY - 2004/7/12

Y1 - 2004/7/12

N2 - The ever-increasing gap in performance between CPU/memory technologies and the I/O subsystem (disks, I/O buses) in modern workstations has exacerbated the I/O bottlenecks inherent in applications that access large disk resident data sets. A common technique to alleviate the I/O bottlenecks on clusters of work-stations, is the use of parallel file systems. One such parallel file system is the Parallel Virtual File System (PVFS), which is a freely available tool to achieve high-performance I/O on Linux-based clusters. In this paper, we describe the performance and scalability of the UNIX I/O interface to PVFS. To illustrate the performance, we present experimental results using Bonnie++, a commonly used file system benchmark to test file system throughput; a synthetic parallel I/O application for calculating aggregate read and write bandwidths; and a synthetic benchmark which calculates the time taken to untar the Linux kernel source tree to measure performance of a large number of small file operations. We obtained aggregate read and write bandwidths as high as 550 MB/s with a Myrinet-based network and 160MB/S with fast Ethernet.

AB - The ever-increasing gap in performance between CPU/memory technologies and the I/O subsystem (disks, I/O buses) in modern workstations has exacerbated the I/O bottlenecks inherent in applications that access large disk resident data sets. A common technique to alleviate the I/O bottlenecks on clusters of work-stations, is the use of parallel file systems. One such parallel file system is the Parallel Virtual File System (PVFS), which is a freely available tool to achieve high-performance I/O on Linux-based clusters. In this paper, we describe the performance and scalability of the UNIX I/O interface to PVFS. To illustrate the performance, we present experimental results using Bonnie++, a commonly used file system benchmark to test file system throughput; a synthetic parallel I/O application for calculating aggregate read and write bandwidths; and a synthetic benchmark which calculates the time taken to untar the Linux kernel source tree to measure performance of a large number of small file operations. We obtained aggregate read and write bandwidths as high as 550 MB/s with a Myrinet-based network and 160MB/S with fast Ethernet.

UR - http://www.scopus.com/inward/record.url?scp=3042530916&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=3042530916&partnerID=8YFLogxK

U2 - 10.1109/EMPDP.2004.1271463

DO - 10.1109/EMPDP.2004.1271463

M3 - Conference contribution

SN - 0769520839

SN - 9780769520834

T3 - Proceedings - Euromicro Conference on Parellel, Distribeted and Network-based Proceeding

SP - 332

EP - 339

BT - Proceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004

ER -

Vilayannur M, Ross RB, Carns PH, Thakur R, Sivasubramaniam A, Kandemir M. On the performance of the POSIX I/O interface to PVFS. In Proceedings - 12th Euromicro Conference on Parallel, Distributed and Network-based Proceedings, PDP 2004. 2004. p. 332-339. (Proceedings - Euromicro Conference on Parellel, Distribeted and Network-based Proceeding). https://doi.org/10.1109/EMPDP.2004.1271463