Towards performance modeling as a service by exploiting resource diversity in the public cloud

Mark Meredith, Bhuvan Urgaonkar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Cloud computing platforms such as Amazon EC2, Google Computing Engine, and Microsoft Azure offer dozens of virtual machine (VM) types with a wide range of resource capacity vs. price trade-offs, requiring a customer to consider numerous resource configurations when evaluating service needs. This report investigates the possibility of using the diversity of VM types to predict the performance of new VM types using black box modeling. The performance model used is a multiple linear regression of the average server response time, server load (throughput in requests per second), the number of CPU cores, and the memory in the procured VM. For three commonly used database servers-Redis (key-value stores), Apache Cassandra (NoSQL) and MySQL-the model accuracy increases for larger sets of VMs. E.g., for Redis, the measure of model efficacy improves from 0.4-0.5 with 2 VM types for training and 0.7 for 3 VM types to 0.8 for 4 VM types. These results suggest further interesting research challenges, such as the possibility of automating the process of calibrating performance models using diverse resource types on a public cloud leading to "performance modeling as a service."

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016
EditorsIan Foster, Nimish Radia, Ian Foster
PublisherIEEE Computer Society
Pages204-211
Number of pages8
ISBN (Electronic)9781509026197
DOIs
StatePublished - Jan 17 2017
Event9th International Conference on Cloud Computing, CLOUD 2016 - San Francisco, United States
Duration: Jun 27 2016Jul 2 2016

Publication series

NameIEEE International Conference on Cloud Computing, CLOUD
ISSN (Print)2159-6182
ISSN (Electronic)2159-6190

Other

Other9th International Conference on Cloud Computing, CLOUD 2016
CountryUnited States
CitySan Francisco
Period6/27/167/2/16

Fingerprint

Servers
Response time (computer systems)
Cloud computing
Virtual machine
Linear regression
Program processors
Throughput
Engines
Data storage equipment

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Information Systems
  • Software

Cite this

Meredith, M., & Urgaonkar, B. (2017). Towards performance modeling as a service by exploiting resource diversity in the public cloud. In I. Foster, N. Radia, & I. Foster (Eds.), Proceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016 (pp. 204-211). [7820273] (IEEE International Conference on Cloud Computing, CLOUD). IEEE Computer Society. https://doi.org/10.1109/CLOUD.2016.35
Meredith, Mark ; Urgaonkar, Bhuvan. / Towards performance modeling as a service by exploiting resource diversity in the public cloud. Proceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016. editor / Ian Foster ; Nimish Radia ; Ian Foster. IEEE Computer Society, 2017. pp. 204-211 (IEEE International Conference on Cloud Computing, CLOUD).
@inproceedings{8c30702c9bfc46a68f8382d7a55033c5,
title = "Towards performance modeling as a service by exploiting resource diversity in the public cloud",
abstract = "Cloud computing platforms such as Amazon EC2, Google Computing Engine, and Microsoft Azure offer dozens of virtual machine (VM) types with a wide range of resource capacity vs. price trade-offs, requiring a customer to consider numerous resource configurations when evaluating service needs. This report investigates the possibility of using the diversity of VM types to predict the performance of new VM types using black box modeling. The performance model used is a multiple linear regression of the average server response time, server load (throughput in requests per second), the number of CPU cores, and the memory in the procured VM. For three commonly used database servers-Redis (key-value stores), Apache Cassandra (NoSQL) and MySQL-the model accuracy increases for larger sets of VMs. E.g., for Redis, the measure of model efficacy improves from 0.4-0.5 with 2 VM types for training and 0.7 for 3 VM types to 0.8 for 4 VM types. These results suggest further interesting research challenges, such as the possibility of automating the process of calibrating performance models using diverse resource types on a public cloud leading to {"}performance modeling as a service.{"}",
author = "Mark Meredith and Bhuvan Urgaonkar",
year = "2017",
month = "1",
day = "17",
doi = "10.1109/CLOUD.2016.35",
language = "English (US)",
series = "IEEE International Conference on Cloud Computing, CLOUD",
publisher = "IEEE Computer Society",
pages = "204--211",
editor = "Ian Foster and Nimish Radia and Ian Foster",
booktitle = "Proceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016",
address = "United States",

}

Meredith, M & Urgaonkar, B 2017, Towards performance modeling as a service by exploiting resource diversity in the public cloud. in I Foster, N Radia & I Foster (eds), Proceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016., 7820273, IEEE International Conference on Cloud Computing, CLOUD, IEEE Computer Society, pp. 204-211, 9th International Conference on Cloud Computing, CLOUD 2016, San Francisco, United States, 6/27/16. https://doi.org/10.1109/CLOUD.2016.35

Towards performance modeling as a service by exploiting resource diversity in the public cloud. / Meredith, Mark; Urgaonkar, Bhuvan.

Proceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016. ed. / Ian Foster; Nimish Radia; Ian Foster. IEEE Computer Society, 2017. p. 204-211 7820273 (IEEE International Conference on Cloud Computing, CLOUD).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Towards performance modeling as a service by exploiting resource diversity in the public cloud

AU - Meredith, Mark

AU - Urgaonkar, Bhuvan

PY - 2017/1/17

Y1 - 2017/1/17

N2 - Cloud computing platforms such as Amazon EC2, Google Computing Engine, and Microsoft Azure offer dozens of virtual machine (VM) types with a wide range of resource capacity vs. price trade-offs, requiring a customer to consider numerous resource configurations when evaluating service needs. This report investigates the possibility of using the diversity of VM types to predict the performance of new VM types using black box modeling. The performance model used is a multiple linear regression of the average server response time, server load (throughput in requests per second), the number of CPU cores, and the memory in the procured VM. For three commonly used database servers-Redis (key-value stores), Apache Cassandra (NoSQL) and MySQL-the model accuracy increases for larger sets of VMs. E.g., for Redis, the measure of model efficacy improves from 0.4-0.5 with 2 VM types for training and 0.7 for 3 VM types to 0.8 for 4 VM types. These results suggest further interesting research challenges, such as the possibility of automating the process of calibrating performance models using diverse resource types on a public cloud leading to "performance modeling as a service."

AB - Cloud computing platforms such as Amazon EC2, Google Computing Engine, and Microsoft Azure offer dozens of virtual machine (VM) types with a wide range of resource capacity vs. price trade-offs, requiring a customer to consider numerous resource configurations when evaluating service needs. This report investigates the possibility of using the diversity of VM types to predict the performance of new VM types using black box modeling. The performance model used is a multiple linear regression of the average server response time, server load (throughput in requests per second), the number of CPU cores, and the memory in the procured VM. For three commonly used database servers-Redis (key-value stores), Apache Cassandra (NoSQL) and MySQL-the model accuracy increases for larger sets of VMs. E.g., for Redis, the measure of model efficacy improves from 0.4-0.5 with 2 VM types for training and 0.7 for 3 VM types to 0.8 for 4 VM types. These results suggest further interesting research challenges, such as the possibility of automating the process of calibrating performance models using diverse resource types on a public cloud leading to "performance modeling as a service."

UR - http://www.scopus.com/inward/record.url?scp=85014240305&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014240305&partnerID=8YFLogxK

U2 - 10.1109/CLOUD.2016.35

DO - 10.1109/CLOUD.2016.35

M3 - Conference contribution

AN - SCOPUS:85014240305

T3 - IEEE International Conference on Cloud Computing, CLOUD

SP - 204

EP - 211

BT - Proceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016

A2 - Foster, Ian

A2 - Radia, Nimish

A2 - Foster, Ian

PB - IEEE Computer Society

ER -

Meredith M, Urgaonkar B. Towards performance modeling as a service by exploiting resource diversity in the public cloud. In Foster I, Radia N, Foster I, editors, Proceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016. IEEE Computer Society. 2017. p. 204-211. 7820273. (IEEE International Conference on Cloud Computing, CLOUD). https://doi.org/10.1109/CLOUD.2016.35