SNC-meister: Admitting more tenants with tail latency SLOs

Timothy Zhu, Daniel S. Berger, Mor Harchol-Balter

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Citations (Scopus)

Abstract

Meeting tail latency Service Level Objectives (SLOs) in shared cloud networks is both important and challenging. One primary challenge is determining limits on the multitenancy such that SLOs are met. Doing so involves estimating latency, which is difficult, especially when tenants exhibit bursty behavior as is common in production environments. Nevertheless, recent papers in the past two years (Silo, QJump, and PriorityMeister) show techniques for calculating latency based on a branch of mathematical modeling called Deterministic Network Calculus (DNC). The DNC theory is designed for adversarial worst-case conditions, which is sometimes necessary, but is often overly conservative. Typical tenants do not require strict worst-case guarantees, but are only looking for SLOs at lower percentiles (e.g., 99th, 99.9th). This paper describes SNC-Meister, a new admission control system for tail latency SLOs. SNC-Meister improves upon the state-of-the-art DNC-based systems by using a new theory, Stochastic Network Calculus (SNC), which is designed for tail latency percentiles. Focusing on tail latency percentiles, rather than the adversarial worst-case DNC latency, allows SNC-Meister to pack together many more tenants: in experiments with production traces, SNC-Meister supports 75% more tenants than the state-of-the-art.

Original languageEnglish (US)
Title of host publicationProceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016
EditorsYanlei Diao, Marcos K. Aguilera, Brian Cooper, Yanlei Diao
PublisherAssociation for Computing Machinery, Inc
Pages374-387
Number of pages14
ISBN (Electronic)9781450345255
DOIs
StatePublished - Oct 5 2016
Event7th ACM Symposium on Cloud Computing, SoCC 2016 - Santa Clara, United States
Duration: Oct 5 2016Oct 7 2016

Publication series

NameProceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016

Other

Other7th ACM Symposium on Cloud Computing, SoCC 2016
CountryUnited States
CitySanta Clara
Period10/5/1610/7/16

Fingerprint

Access control
Control systems
Experiments

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Computational Theory and Mathematics

Cite this

Zhu, T., Berger, D. S., & Harchol-Balter, M. (2016). SNC-meister: Admitting more tenants with tail latency SLOs. In Y. Diao, M. K. Aguilera, B. Cooper, & Y. Diao (Eds.), Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016 (pp. 374-387). (Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016). Association for Computing Machinery, Inc. https://doi.org/10.1145/2987550.2987585
Zhu, Timothy ; Berger, Daniel S. ; Harchol-Balter, Mor. / SNC-meister : Admitting more tenants with tail latency SLOs. Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016. editor / Yanlei Diao ; Marcos K. Aguilera ; Brian Cooper ; Yanlei Diao. Association for Computing Machinery, Inc, 2016. pp. 374-387 (Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016).
@inproceedings{430d7315407a4ca39a6007cced857fe5,
title = "SNC-meister: Admitting more tenants with tail latency SLOs",
abstract = "Meeting tail latency Service Level Objectives (SLOs) in shared cloud networks is both important and challenging. One primary challenge is determining limits on the multitenancy such that SLOs are met. Doing so involves estimating latency, which is difficult, especially when tenants exhibit bursty behavior as is common in production environments. Nevertheless, recent papers in the past two years (Silo, QJump, and PriorityMeister) show techniques for calculating latency based on a branch of mathematical modeling called Deterministic Network Calculus (DNC). The DNC theory is designed for adversarial worst-case conditions, which is sometimes necessary, but is often overly conservative. Typical tenants do not require strict worst-case guarantees, but are only looking for SLOs at lower percentiles (e.g., 99th, 99.9th). This paper describes SNC-Meister, a new admission control system for tail latency SLOs. SNC-Meister improves upon the state-of-the-art DNC-based systems by using a new theory, Stochastic Network Calculus (SNC), which is designed for tail latency percentiles. Focusing on tail latency percentiles, rather than the adversarial worst-case DNC latency, allows SNC-Meister to pack together many more tenants: in experiments with production traces, SNC-Meister supports 75{\%} more tenants than the state-of-the-art.",
author = "Timothy Zhu and Berger, {Daniel S.} and Mor Harchol-Balter",
year = "2016",
month = "10",
day = "5",
doi = "10.1145/2987550.2987585",
language = "English (US)",
series = "Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016",
publisher = "Association for Computing Machinery, Inc",
pages = "374--387",
editor = "Yanlei Diao and Aguilera, {Marcos K.} and Brian Cooper and Yanlei Diao",
booktitle = "Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016",

}

Zhu, T, Berger, DS & Harchol-Balter, M 2016, SNC-meister: Admitting more tenants with tail latency SLOs. in Y Diao, MK Aguilera, B Cooper & Y Diao (eds), Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016. Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016, Association for Computing Machinery, Inc, pp. 374-387, 7th ACM Symposium on Cloud Computing, SoCC 2016, Santa Clara, United States, 10/5/16. https://doi.org/10.1145/2987550.2987585

SNC-meister : Admitting more tenants with tail latency SLOs. / Zhu, Timothy; Berger, Daniel S.; Harchol-Balter, Mor.

Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016. ed. / Yanlei Diao; Marcos K. Aguilera; Brian Cooper; Yanlei Diao. Association for Computing Machinery, Inc, 2016. p. 374-387 (Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - SNC-meister

T2 - Admitting more tenants with tail latency SLOs

AU - Zhu, Timothy

AU - Berger, Daniel S.

AU - Harchol-Balter, Mor

PY - 2016/10/5

Y1 - 2016/10/5

N2 - Meeting tail latency Service Level Objectives (SLOs) in shared cloud networks is both important and challenging. One primary challenge is determining limits on the multitenancy such that SLOs are met. Doing so involves estimating latency, which is difficult, especially when tenants exhibit bursty behavior as is common in production environments. Nevertheless, recent papers in the past two years (Silo, QJump, and PriorityMeister) show techniques for calculating latency based on a branch of mathematical modeling called Deterministic Network Calculus (DNC). The DNC theory is designed for adversarial worst-case conditions, which is sometimes necessary, but is often overly conservative. Typical tenants do not require strict worst-case guarantees, but are only looking for SLOs at lower percentiles (e.g., 99th, 99.9th). This paper describes SNC-Meister, a new admission control system for tail latency SLOs. SNC-Meister improves upon the state-of-the-art DNC-based systems by using a new theory, Stochastic Network Calculus (SNC), which is designed for tail latency percentiles. Focusing on tail latency percentiles, rather than the adversarial worst-case DNC latency, allows SNC-Meister to pack together many more tenants: in experiments with production traces, SNC-Meister supports 75% more tenants than the state-of-the-art.

AB - Meeting tail latency Service Level Objectives (SLOs) in shared cloud networks is both important and challenging. One primary challenge is determining limits on the multitenancy such that SLOs are met. Doing so involves estimating latency, which is difficult, especially when tenants exhibit bursty behavior as is common in production environments. Nevertheless, recent papers in the past two years (Silo, QJump, and PriorityMeister) show techniques for calculating latency based on a branch of mathematical modeling called Deterministic Network Calculus (DNC). The DNC theory is designed for adversarial worst-case conditions, which is sometimes necessary, but is often overly conservative. Typical tenants do not require strict worst-case guarantees, but are only looking for SLOs at lower percentiles (e.g., 99th, 99.9th). This paper describes SNC-Meister, a new admission control system for tail latency SLOs. SNC-Meister improves upon the state-of-the-art DNC-based systems by using a new theory, Stochastic Network Calculus (SNC), which is designed for tail latency percentiles. Focusing on tail latency percentiles, rather than the adversarial worst-case DNC latency, allows SNC-Meister to pack together many more tenants: in experiments with production traces, SNC-Meister supports 75% more tenants than the state-of-the-art.

UR - http://www.scopus.com/inward/record.url?scp=84995554091&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84995554091&partnerID=8YFLogxK

U2 - 10.1145/2987550.2987585

DO - 10.1145/2987550.2987585

M3 - Conference contribution

AN - SCOPUS:84995554091

T3 - Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016

SP - 374

EP - 387

BT - Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016

A2 - Diao, Yanlei

A2 - Aguilera, Marcos K.

A2 - Cooper, Brian

A2 - Diao, Yanlei

PB - Association for Computing Machinery, Inc

ER -

Zhu T, Berger DS, Harchol-Balter M. SNC-meister: Admitting more tenants with tail latency SLOs. In Diao Y, Aguilera MK, Cooper B, Diao Y, editors, Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016. Association for Computing Machinery, Inc. 2016. p. 374-387. (Proceedings of the 7th ACM Symposium on Cloud Computing, SoCC 2016). https://doi.org/10.1145/2987550.2987585