Performance comparison of coscheduling algorithms for non-dedicated clusters through a generic framework

Gyu Sang Choi, Saurabh Agarwal, Jin Ha Kim, Chitaranjan Das, Andy B. Yoo

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

In this paper, we address several key issues in designing coscheduling algorithms for clusters. First, we propose a generic framework for deploying coscheduling techniques by providing a reusable and dynamically loadable kernel module. Second, we implement several communication-driven coscheduling algorithms [dynamic coscheduling (DCS), spin block (SB) and periodic boost (PB)] on a 16- node Linux cluster using the above framework. Third, with exhaustive experimentation using mixed workloads, we observe that unlike PB, which provided the best performance on a Solaris platform, the SB scheme outperforms all other techniques on a Linux platform. Finally, we investigate the impact of several job placement strategies, multiprogramming level (MPL), communication intensity and CPU and I/O intensive jobs on the performance of these coscheduling schemes. The experimental results show that the blocking-based coscheduling scheme (SB) has better tolerance to system workload variation compared with the spin-based schemes (DCS and PB), and overall, the blocking-based coscheduling scheme seems a better choice for non-dedicated Linux clusters.

Original languageEnglish (US)
Pages (from-to)91-105
Number of pages15
JournalInternational Journal of High Performance Computing Applications
Volume21
Issue number1
DOIs
StatePublished - Jan 1 2007

Fingerprint

Performance Comparison
Linux
Multiprogramming
Spin dynamics
Communication
Workload
Program processors
Spin Dynamics
Dynamic Algorithms
Experimentation
Placement
Tolerance
Framework
kernel
Module
Experimental Results
Vertex of a graph

All Science Journal Classification (ASJC) codes

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture

Cite this

@article{bfc72e84e88943038fd72f88858cb393,
title = "Performance comparison of coscheduling algorithms for non-dedicated clusters through a generic framework",
abstract = "In this paper, we address several key issues in designing coscheduling algorithms for clusters. First, we propose a generic framework for deploying coscheduling techniques by providing a reusable and dynamically loadable kernel module. Second, we implement several communication-driven coscheduling algorithms [dynamic coscheduling (DCS), spin block (SB) and periodic boost (PB)] on a 16- node Linux cluster using the above framework. Third, with exhaustive experimentation using mixed workloads, we observe that unlike PB, which provided the best performance on a Solaris platform, the SB scheme outperforms all other techniques on a Linux platform. Finally, we investigate the impact of several job placement strategies, multiprogramming level (MPL), communication intensity and CPU and I/O intensive jobs on the performance of these coscheduling schemes. The experimental results show that the blocking-based coscheduling scheme (SB) has better tolerance to system workload variation compared with the spin-based schemes (DCS and PB), and overall, the blocking-based coscheduling scheme seems a better choice for non-dedicated Linux clusters.",
author = "Choi, {Gyu Sang} and Saurabh Agarwal and Kim, {Jin Ha} and Chitaranjan Das and Yoo, {Andy B.}",
year = "2007",
month = "1",
day = "1",
doi = "10.1177/1094342006074868",
language = "English (US)",
volume = "21",
pages = "91--105",
journal = "International Journal of High Performance Computing Applications",
issn = "1094-3420",
publisher = "SAGE Publications Inc.",
number = "1",

}

Performance comparison of coscheduling algorithms for non-dedicated clusters through a generic framework. / Choi, Gyu Sang; Agarwal, Saurabh; Kim, Jin Ha; Das, Chitaranjan; Yoo, Andy B.

In: International Journal of High Performance Computing Applications, Vol. 21, No. 1, 01.01.2007, p. 91-105.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Performance comparison of coscheduling algorithms for non-dedicated clusters through a generic framework

AU - Choi, Gyu Sang

AU - Agarwal, Saurabh

AU - Kim, Jin Ha

AU - Das, Chitaranjan

AU - Yoo, Andy B.

PY - 2007/1/1

Y1 - 2007/1/1

N2 - In this paper, we address several key issues in designing coscheduling algorithms for clusters. First, we propose a generic framework for deploying coscheduling techniques by providing a reusable and dynamically loadable kernel module. Second, we implement several communication-driven coscheduling algorithms [dynamic coscheduling (DCS), spin block (SB) and periodic boost (PB)] on a 16- node Linux cluster using the above framework. Third, with exhaustive experimentation using mixed workloads, we observe that unlike PB, which provided the best performance on a Solaris platform, the SB scheme outperforms all other techniques on a Linux platform. Finally, we investigate the impact of several job placement strategies, multiprogramming level (MPL), communication intensity and CPU and I/O intensive jobs on the performance of these coscheduling schemes. The experimental results show that the blocking-based coscheduling scheme (SB) has better tolerance to system workload variation compared with the spin-based schemes (DCS and PB), and overall, the blocking-based coscheduling scheme seems a better choice for non-dedicated Linux clusters.

AB - In this paper, we address several key issues in designing coscheduling algorithms for clusters. First, we propose a generic framework for deploying coscheduling techniques by providing a reusable and dynamically loadable kernel module. Second, we implement several communication-driven coscheduling algorithms [dynamic coscheduling (DCS), spin block (SB) and periodic boost (PB)] on a 16- node Linux cluster using the above framework. Third, with exhaustive experimentation using mixed workloads, we observe that unlike PB, which provided the best performance on a Solaris platform, the SB scheme outperforms all other techniques on a Linux platform. Finally, we investigate the impact of several job placement strategies, multiprogramming level (MPL), communication intensity and CPU and I/O intensive jobs on the performance of these coscheduling schemes. The experimental results show that the blocking-based coscheduling scheme (SB) has better tolerance to system workload variation compared with the spin-based schemes (DCS and PB), and overall, the blocking-based coscheduling scheme seems a better choice for non-dedicated Linux clusters.

UR - http://www.scopus.com/inward/record.url?scp=33846661427&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33846661427&partnerID=8YFLogxK

U2 - 10.1177/1094342006074868

DO - 10.1177/1094342006074868

M3 - Article

AN - SCOPUS:33846661427

VL - 21

SP - 91

EP - 105

JO - International Journal of High Performance Computing Applications

JF - International Journal of High Performance Computing Applications

SN - 1094-3420

IS - 1

ER -