D-factor: A quantitative model of application slow-down in multi-resource shared systems

Seung Hwan Lim, Jae Seok Huh, Youngjae Kim, Galen M. Shipman, Chita R. Das

Research output: Chapter in Book/Report/Conference proceedingConference contribution

24 Citations (Scopus)

Abstract

Scheduling multiple jobs onto a platform enhances system utilization by sharing resources. The benefits from higher resource utilization include reduced cost to construct, operate, and maintain a system, which often include energy consumption. Maximizing these benefits, while satisfying performance limits, comes at a price - resource contention among jobs increases job completion time. In this paper, we analyze slow-downs of jobs due to contention for multiple resources in a system; referred to as dilation factor. We observe that multiple-resource contention creates non-linear dilation factors of jobs. From this observation, we establish a general quantitative model for dilation factors of jobs in multi-resource systems. A job is characterized by a vector-valued loading statistics and dilation factors of a job set are given by a quadratic function of their loading vectors. We demonstrate how to systematically characterize a job, maintain the data structure to calculate the dilation factor (loading matrix), and calculate the dilation factor of each job. We validated the accuracy of the model with multiple processes running on a native Linux server, virtualized servers, and with multiple MapReduce workloads co-scheduled in a cluster. Evaluation with measured data shows that the D-factor model has an error margin of less than 16%. We also show that the model can be integrated with an existing on-line scheduler to minimize the makespan of workloads.

Original languageEnglish (US)
Title of host publicationSIGMETRICS/Performance 2012 - Proceedings of the 2012 ACM SIGMETRICS/Performance, Joint International Conference on Measurement and Modeling of Computer Systems
Pages271-282
Number of pages12
Edition1 SPEC. ISS.
DOIs
StatePublished - Aug 13 2012
Event12th Joint International Conference on Measurement and Modeling of Computer Systems, ACM SIGMETRICS/Performance 2012 - London, United Kingdom
Duration: Jun 11 2012Jun 15 2012

Publication series

NamePerformance Evaluation Review
Number1 SPEC. ISS.
Volume40
ISSN (Print)0163-5999

Other

Other12th Joint International Conference on Measurement and Modeling of Computer Systems, ACM SIGMETRICS/Performance 2012
CountryUnited Kingdom
CityLondon
Period6/11/126/15/12

Fingerprint

Servers
Data structures
Energy utilization
Scheduling
Statistics
Costs
Linux

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

Lim, S. H., Huh, J. S., Kim, Y., Shipman, G. M., & Das, C. R. (2012). D-factor: A quantitative model of application slow-down in multi-resource shared systems. In SIGMETRICS/Performance 2012 - Proceedings of the 2012 ACM SIGMETRICS/Performance, Joint International Conference on Measurement and Modeling of Computer Systems (1 SPEC. ISS. ed., pp. 271-282). (Performance Evaluation Review; Vol. 40, No. 1 SPEC. ISS.). https://doi.org/10.1145/2254756.2254790
Lim, Seung Hwan ; Huh, Jae Seok ; Kim, Youngjae ; Shipman, Galen M. ; Das, Chita R. / D-factor : A quantitative model of application slow-down in multi-resource shared systems. SIGMETRICS/Performance 2012 - Proceedings of the 2012 ACM SIGMETRICS/Performance, Joint International Conference on Measurement and Modeling of Computer Systems. 1 SPEC. ISS. ed. 2012. pp. 271-282 (Performance Evaluation Review; 1 SPEC. ISS.).
@inproceedings{8b74e40797474f84a437cfacf997d727,
title = "D-factor: A quantitative model of application slow-down in multi-resource shared systems",
abstract = "Scheduling multiple jobs onto a platform enhances system utilization by sharing resources. The benefits from higher resource utilization include reduced cost to construct, operate, and maintain a system, which often include energy consumption. Maximizing these benefits, while satisfying performance limits, comes at a price - resource contention among jobs increases job completion time. In this paper, we analyze slow-downs of jobs due to contention for multiple resources in a system; referred to as dilation factor. We observe that multiple-resource contention creates non-linear dilation factors of jobs. From this observation, we establish a general quantitative model for dilation factors of jobs in multi-resource systems. A job is characterized by a vector-valued loading statistics and dilation factors of a job set are given by a quadratic function of their loading vectors. We demonstrate how to systematically characterize a job, maintain the data structure to calculate the dilation factor (loading matrix), and calculate the dilation factor of each job. We validated the accuracy of the model with multiple processes running on a native Linux server, virtualized servers, and with multiple MapReduce workloads co-scheduled in a cluster. Evaluation with measured data shows that the D-factor model has an error margin of less than 16{\%}. We also show that the model can be integrated with an existing on-line scheduler to minimize the makespan of workloads.",
author = "Lim, {Seung Hwan} and Huh, {Jae Seok} and Youngjae Kim and Shipman, {Galen M.} and Das, {Chita R.}",
year = "2012",
month = "8",
day = "13",
doi = "10.1145/2254756.2254790",
language = "English (US)",
isbn = "9781450310970",
series = "Performance Evaluation Review",
number = "1 SPEC. ISS.",
pages = "271--282",
booktitle = "SIGMETRICS/Performance 2012 - Proceedings of the 2012 ACM SIGMETRICS/Performance, Joint International Conference on Measurement and Modeling of Computer Systems",
edition = "1 SPEC. ISS.",

}

Lim, SH, Huh, JS, Kim, Y, Shipman, GM & Das, CR 2012, D-factor: A quantitative model of application slow-down in multi-resource shared systems. in SIGMETRICS/Performance 2012 - Proceedings of the 2012 ACM SIGMETRICS/Performance, Joint International Conference on Measurement and Modeling of Computer Systems. 1 SPEC. ISS. edn, Performance Evaluation Review, no. 1 SPEC. ISS., vol. 40, pp. 271-282, 12th Joint International Conference on Measurement and Modeling of Computer Systems, ACM SIGMETRICS/Performance 2012, London, United Kingdom, 6/11/12. https://doi.org/10.1145/2254756.2254790

D-factor : A quantitative model of application slow-down in multi-resource shared systems. / Lim, Seung Hwan; Huh, Jae Seok; Kim, Youngjae; Shipman, Galen M.; Das, Chita R.

SIGMETRICS/Performance 2012 - Proceedings of the 2012 ACM SIGMETRICS/Performance, Joint International Conference on Measurement and Modeling of Computer Systems. 1 SPEC. ISS. ed. 2012. p. 271-282 (Performance Evaluation Review; Vol. 40, No. 1 SPEC. ISS.).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - D-factor

T2 - A quantitative model of application slow-down in multi-resource shared systems

AU - Lim, Seung Hwan

AU - Huh, Jae Seok

AU - Kim, Youngjae

AU - Shipman, Galen M.

AU - Das, Chita R.

PY - 2012/8/13

Y1 - 2012/8/13

N2 - Scheduling multiple jobs onto a platform enhances system utilization by sharing resources. The benefits from higher resource utilization include reduced cost to construct, operate, and maintain a system, which often include energy consumption. Maximizing these benefits, while satisfying performance limits, comes at a price - resource contention among jobs increases job completion time. In this paper, we analyze slow-downs of jobs due to contention for multiple resources in a system; referred to as dilation factor. We observe that multiple-resource contention creates non-linear dilation factors of jobs. From this observation, we establish a general quantitative model for dilation factors of jobs in multi-resource systems. A job is characterized by a vector-valued loading statistics and dilation factors of a job set are given by a quadratic function of their loading vectors. We demonstrate how to systematically characterize a job, maintain the data structure to calculate the dilation factor (loading matrix), and calculate the dilation factor of each job. We validated the accuracy of the model with multiple processes running on a native Linux server, virtualized servers, and with multiple MapReduce workloads co-scheduled in a cluster. Evaluation with measured data shows that the D-factor model has an error margin of less than 16%. We also show that the model can be integrated with an existing on-line scheduler to minimize the makespan of workloads.

AB - Scheduling multiple jobs onto a platform enhances system utilization by sharing resources. The benefits from higher resource utilization include reduced cost to construct, operate, and maintain a system, which often include energy consumption. Maximizing these benefits, while satisfying performance limits, comes at a price - resource contention among jobs increases job completion time. In this paper, we analyze slow-downs of jobs due to contention for multiple resources in a system; referred to as dilation factor. We observe that multiple-resource contention creates non-linear dilation factors of jobs. From this observation, we establish a general quantitative model for dilation factors of jobs in multi-resource systems. A job is characterized by a vector-valued loading statistics and dilation factors of a job set are given by a quadratic function of their loading vectors. We demonstrate how to systematically characterize a job, maintain the data structure to calculate the dilation factor (loading matrix), and calculate the dilation factor of each job. We validated the accuracy of the model with multiple processes running on a native Linux server, virtualized servers, and with multiple MapReduce workloads co-scheduled in a cluster. Evaluation with measured data shows that the D-factor model has an error margin of less than 16%. We also show that the model can be integrated with an existing on-line scheduler to minimize the makespan of workloads.

UR - http://www.scopus.com/inward/record.url?scp=84864680980&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84864680980&partnerID=8YFLogxK

U2 - 10.1145/2254756.2254790

DO - 10.1145/2254756.2254790

M3 - Conference contribution

AN - SCOPUS:84864680980

SN - 9781450310970

T3 - Performance Evaluation Review

SP - 271

EP - 282

BT - SIGMETRICS/Performance 2012 - Proceedings of the 2012 ACM SIGMETRICS/Performance, Joint International Conference on Measurement and Modeling of Computer Systems

ER -

Lim SH, Huh JS, Kim Y, Shipman GM, Das CR. D-factor: A quantitative model of application slow-down in multi-resource shared systems. In SIGMETRICS/Performance 2012 - Proceedings of the 2012 ACM SIGMETRICS/Performance, Joint International Conference on Measurement and Modeling of Computer Systems. 1 SPEC. ISS. ed. 2012. p. 271-282. (Performance Evaluation Review; 1 SPEC. ISS.). https://doi.org/10.1145/2254756.2254790