Gang scheduling extensions for I/O intensive workloads

Yanyong Zhang, Antony Yang, Anand Sivasubramaniam, Jose Moreira

Research output: Chapter in Book/Report/Conference proceedingChapter

12 Scopus citations

Abstract

Scientific applications are becoming more complex and more I/O demanding than ever. For such applications, the system with dedicated I/O nodes does not provide enough scalability. Rather, a serverless approach is a viable alternative. However, with the serverless approach, a job's execution time is decided by whether it is co-located with the file blocks it needs. Gang scheduling (GS), which is widely used in supercomputing centers to schedule parallel jobs, is completely not aware of the application's spatial preferences. In this paper, we show that gang scheduling does not do a good job scheduling I/O intensive applications. We extend gang scheduling by adding different levels of I/O awareness, and propose three schemes. We show that all these three new schemes are better than gang scheduling for I/O intensive jobs. One of them, with the help of migration, outperforms the others significantly for all the workloads we look at.

Original languageEnglish (US)
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
EditorsDror Feitelson, Larry Rudolph, Uwe Schwiegelshohn
PublisherSpringer Verlag
Pages183-207
Number of pages25
ISBN (Print)9783540397274
DOIs
StatePublished - 2003

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2862
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Gang scheduling extensions for I/O intensive workloads'. Together they form a unique fingerprint.

  • Cite this

    Zhang, Y., Yang, A., Sivasubramaniam, A., & Moreira, J. (2003). Gang scheduling extensions for I/O intensive workloads. In D. Feitelson, L. Rudolph, & U. Schwiegelshohn (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 183-207). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2862). Springer Verlag. https://doi.org/10.1007/10968987_10