Network scheduling aware task placement in datacenters

Ali Munir, Ting He, Ramya Raghavendra, Franck Le, Alex X. Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

To improve the performance of data-intensive applications, existing datacenter schedulers optimize either the placement of tasks or the scheduling of network flows. The task scheduler strives to place tasks close to their input data (i.e., maximize data locality) to minimize network traffic, while assuming fair sharing of the network. The network scheduler strives to finish flows as quickly as possible based on their sources and destinations determined by the task scheduler, while the scheduling is based on flow properties (e.g., size, deadline, and correlation) and not bound to fair sharing. Inconsistent assumptions of the two schedulers can compromise the overall application performance. In this paper, we propose NEAT, a task scheduling framework that leverages information from the underlying network scheduler to make task placement decisions. The core of NEAT is a task completion time predictor that estimates the completion tijme of a task under given network condition and a given network scheduling policy. NEAT leverages the predicted task completion times to minimize the average completion time of active tasks. Evaluation using ns2 simulations and real-testbed shows that NEAT improves application performance by up to 3.7x for the suboptimal network scheduling policies and up to 30% for the optimal network scheduling policy.

Original languageEnglish (US)
Title of host publicationCoNEXT 2016 - Proceedings of the 12th International Conference on Emerging Networking EXperiments and Technologies
PublisherAssociation for Computing Machinery, Inc
Pages221-235
Number of pages15
ISBN (Electronic)9781450342926
DOIs
StatePublished - Dec 6 2016
Event12th ACM Conference on Emerging Networking Experiments and Technologies, ACM CoNEXT 2016 - Irvine, United States
Duration: Dec 12 2016Dec 15 2016

Publication series

NameCoNEXT 2016 - Proceedings of the 12th International Conference on Emerging Networking EXperiments and Technologies

Other

Other12th ACM Conference on Emerging Networking Experiments and Technologies, ACM CoNEXT 2016
CountryUnited States
CityIrvine
Period12/12/1612/15/16

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Network scheduling aware task placement in datacenters'. Together they form a unique fingerprint.

  • Cite this

    Munir, A., He, T., Raghavendra, R., Le, F., & Liu, A. X. (2016). Network scheduling aware task placement in datacenters. In CoNEXT 2016 - Proceedings of the 12th International Conference on Emerging Networking EXperiments and Technologies (pp. 221-235). (CoNEXT 2016 - Proceedings of the 12th International Conference on Emerging Networking EXperiments and Technologies). Association for Computing Machinery, Inc. https://doi.org/10.1145/2999572.2999588