Coscheduling in Clusters: Is It a Viable Alternative?

Gyu Sang Choi, Jin Ha Kim, Deniz Ersoz, Andy B. Yoo, Chita R. Das

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

In this paper, we conduct an in-depth evaluation of a broad spectrum of scheduling alternatives for clusters. These include the widely used batch scheduling, local scheduling, gang scheduling, all prior communication-driven coscheduling algorithms (Dynamic Coscheduling (DCS), Spin Block (SB), Periodic Boost (PB), and Co-ordinated Coscheduling (CC)) and a newly proposed HYBRID coscheduling algorithm on a 16-node, Myrinet-connected Linux cluster. Performance and energy measurements using several NAS, LLNL and ANL benchmarks on the Linux cluster provide several interesting conclusions. First, although batch scheduling is currently used in most clusters, all blocking-based coscheduling techniques such as SB, CC and HYBRID and the gang scheduling can provide much better performance even in a dedicated cluster platform. Second, in contrast to some of the prior studies, we observe that blocking-based schemes like SB and HYBRID can provide better performance than spin-based techniques like PB on a Linux platform. Third, the proposed HYBRID scheduling provides the best performance-energy behavior and can be implemented on any cluster with little effort. All these results suggest that blocking-based coscheduling techniques are viable candidates to be used in clusters for significant performance-energy benefits.

Original languageEnglish (US)
Title of host publicationProceedings of the ACM/IEEE SC 2004 Conference
Subtitle of host publicationBridging Communities
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)0769521533, 9780769521534
DOIs
StatePublished - Jan 1 2004
Event2004 ACM/IEEE Conference on Supercomputing, SC 2004 - Pittsburgh, United States
Duration: Nov 6 2004Nov 12 2004

Publication series

NameProceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities

Other

Other2004 ACM/IEEE Conference on Supercomputing, SC 2004
CountryUnited States
CityPittsburgh
Period11/6/0411/12/04

Fingerprint

Scheduling
Spin dynamics
Electric power measurement
Communication
Linux

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Software

Cite this

Choi, G. S., Kim, J. H., Ersoz, D., Yoo, A. B., & Das, C. R. (2004). Coscheduling in Clusters: Is It a Viable Alternative? In Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities [1392946] (Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SC.2004.20
Choi, Gyu Sang ; Kim, Jin Ha ; Ersoz, Deniz ; Yoo, Andy B. ; Das, Chita R. / Coscheduling in Clusters : Is It a Viable Alternative?. Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities. Institute of Electrical and Electronics Engineers Inc., 2004. (Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities).
@inproceedings{3b796006ed67473e93f815a5992a056a,
title = "Coscheduling in Clusters: Is It a Viable Alternative?",
abstract = "In this paper, we conduct an in-depth evaluation of a broad spectrum of scheduling alternatives for clusters. These include the widely used batch scheduling, local scheduling, gang scheduling, all prior communication-driven coscheduling algorithms (Dynamic Coscheduling (DCS), Spin Block (SB), Periodic Boost (PB), and Co-ordinated Coscheduling (CC)) and a newly proposed HYBRID coscheduling algorithm on a 16-node, Myrinet-connected Linux cluster. Performance and energy measurements using several NAS, LLNL and ANL benchmarks on the Linux cluster provide several interesting conclusions. First, although batch scheduling is currently used in most clusters, all blocking-based coscheduling techniques such as SB, CC and HYBRID and the gang scheduling can provide much better performance even in a dedicated cluster platform. Second, in contrast to some of the prior studies, we observe that blocking-based schemes like SB and HYBRID can provide better performance than spin-based techniques like PB on a Linux platform. Third, the proposed HYBRID scheduling provides the best performance-energy behavior and can be implemented on any cluster with little effort. All these results suggest that blocking-based coscheduling techniques are viable candidates to be used in clusters for significant performance-energy benefits.",
author = "Choi, {Gyu Sang} and Kim, {Jin Ha} and Deniz Ersoz and Yoo, {Andy B.} and Das, {Chita R.}",
year = "2004",
month = "1",
day = "1",
doi = "10.1109/SC.2004.20",
language = "English (US)",
series = "Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "Proceedings of the ACM/IEEE SC 2004 Conference",
address = "United States",

}

Choi, GS, Kim, JH, Ersoz, D, Yoo, AB & Das, CR 2004, Coscheduling in Clusters: Is It a Viable Alternative? in Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities., 1392946, Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities, Institute of Electrical and Electronics Engineers Inc., 2004 ACM/IEEE Conference on Supercomputing, SC 2004, Pittsburgh, United States, 11/6/04. https://doi.org/10.1109/SC.2004.20

Coscheduling in Clusters : Is It a Viable Alternative? / Choi, Gyu Sang; Kim, Jin Ha; Ersoz, Deniz; Yoo, Andy B.; Das, Chita R.

Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities. Institute of Electrical and Electronics Engineers Inc., 2004. 1392946 (Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Coscheduling in Clusters

T2 - Is It a Viable Alternative?

AU - Choi, Gyu Sang

AU - Kim, Jin Ha

AU - Ersoz, Deniz

AU - Yoo, Andy B.

AU - Das, Chita R.

PY - 2004/1/1

Y1 - 2004/1/1

N2 - In this paper, we conduct an in-depth evaluation of a broad spectrum of scheduling alternatives for clusters. These include the widely used batch scheduling, local scheduling, gang scheduling, all prior communication-driven coscheduling algorithms (Dynamic Coscheduling (DCS), Spin Block (SB), Periodic Boost (PB), and Co-ordinated Coscheduling (CC)) and a newly proposed HYBRID coscheduling algorithm on a 16-node, Myrinet-connected Linux cluster. Performance and energy measurements using several NAS, LLNL and ANL benchmarks on the Linux cluster provide several interesting conclusions. First, although batch scheduling is currently used in most clusters, all blocking-based coscheduling techniques such as SB, CC and HYBRID and the gang scheduling can provide much better performance even in a dedicated cluster platform. Second, in contrast to some of the prior studies, we observe that blocking-based schemes like SB and HYBRID can provide better performance than spin-based techniques like PB on a Linux platform. Third, the proposed HYBRID scheduling provides the best performance-energy behavior and can be implemented on any cluster with little effort. All these results suggest that blocking-based coscheduling techniques are viable candidates to be used in clusters for significant performance-energy benefits.

AB - In this paper, we conduct an in-depth evaluation of a broad spectrum of scheduling alternatives for clusters. These include the widely used batch scheduling, local scheduling, gang scheduling, all prior communication-driven coscheduling algorithms (Dynamic Coscheduling (DCS), Spin Block (SB), Periodic Boost (PB), and Co-ordinated Coscheduling (CC)) and a newly proposed HYBRID coscheduling algorithm on a 16-node, Myrinet-connected Linux cluster. Performance and energy measurements using several NAS, LLNL and ANL benchmarks on the Linux cluster provide several interesting conclusions. First, although batch scheduling is currently used in most clusters, all blocking-based coscheduling techniques such as SB, CC and HYBRID and the gang scheduling can provide much better performance even in a dedicated cluster platform. Second, in contrast to some of the prior studies, we observe that blocking-based schemes like SB and HYBRID can provide better performance than spin-based techniques like PB on a Linux platform. Third, the proposed HYBRID scheduling provides the best performance-energy behavior and can be implemented on any cluster with little effort. All these results suggest that blocking-based coscheduling techniques are viable candidates to be used in clusters for significant performance-energy benefits.

UR - http://www.scopus.com/inward/record.url?scp=84934325514&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84934325514&partnerID=8YFLogxK

U2 - 10.1109/SC.2004.20

DO - 10.1109/SC.2004.20

M3 - Conference contribution

AN - SCOPUS:84934325514

T3 - Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities

BT - Proceedings of the ACM/IEEE SC 2004 Conference

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Choi GS, Kim JH, Ersoz D, Yoo AB, Das CR. Coscheduling in Clusters: Is It a Viable Alternative? In Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities. Institute of Electrical and Electronics Engineers Inc. 2004. 1392946. (Proceedings of the ACM/IEEE SC 2004 Conference: Bridging Communities). https://doi.org/10.1109/SC.2004.20