The Fast and the Frugal: Tail Latency Aware Provisioning for Coping with Load Variations

Adithya Kumar, Iyswarya Narayanan, Timothy Zhu, Anand Sivasubramaniam

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Small and medium sized enterprises use the cloud for running online, user-facing, tail latency sensitive applications with well-defined fixed monthly budgets. For these applications, adequate system capacity must be provisioned to extract maximal performance despite the challenges of uncertainties in load and request-sizes. In this paper, we address the problem of capacity provisioning under fixed budget constraints with the goal of minimizing tail latency. To tackle this problem, we propose building systems using a heterogeneous mix of low latency expensive resources and cheap resources that provide high throughput per dollar. As load changes through the day, we use more faster resources to reduce tail latency during low load periods and more cheaper resources to handle the high load periods. To achieve these tail latency benefits, we introduce novel heterogeneity-aware scheduling and autoscaling algorithms that are designed for minimizing tail latency. Using software prototypes and by running experiments on the public cloud, we show that our approach can outperform existing capacity provisioning systems by reducing the tail latency by as much as 45% under fixed-budget settings.

Original languageEnglish (US)
Title of host publicationThe Web Conference 2020 - Proceedings of the World Wide Web Conference, WWW 2020
PublisherAssociation for Computing Machinery, Inc
Pages314-326
Number of pages13
ISBN (Electronic)9781450370233
DOIs
StatePublished - Apr 20 2020
Event29th International World Wide Web Conference, WWW 2020 - Taipei, Taiwan, Province of China
Duration: Apr 20 2020Apr 24 2020

Publication series

NameThe Web Conference 2020 - Proceedings of the World Wide Web Conference, WWW 2020

Conference

Conference29th International World Wide Web Conference, WWW 2020
CountryTaiwan, Province of China
CityTaipei
Period4/20/204/24/20

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software

Fingerprint Dive into the research topics of 'The Fast and the Frugal: Tail Latency Aware Provisioning for Coping with Load Variations'. Together they form a unique fingerprint.

Cite this