Quickly finding near-optimal storage designs

Eric Anderson, Susan Spence, Ram Swaminathan, Mahesh Kallahalla, Qian Wang

Research output: Contribution to journalArticle

56 Citations (Scopus)

Abstract

Despite the importance of storage in enterprise computer systems, there are few adequate tools to design and configure a storage system to meet application data requirements efficiently. Storage system design involves choosing the disk arrays to use, setting the configuration options on those arrays, and determining an efficient mapping of application data onto the configured system. This is a complex process because of the multitude of disk array configuration options, and the need to take into account both capacity and potentially contending I/O performance demands when placing the data. Thus, both existing tools and administrators using rules of thumb often generate designs that are of poor quality. This article presents the Disk Array Designer (DAD), which is a tool that can be used both to guide administrators in their design decisions and to automate the design process. DAD uses a generalized best-fit bin packing heuristic with randomization and backtracking to search efficiently through the huge number of possible design choices. It makes decisions using device models that estimate storage system performance. We evaluate DAD's designs based on traces from a variety of database, filesystem, and e-mail workloads. We show that DAD can handle the difficult task of configuring midrange and high-end disk arrays, even with complex real-world workloads. We also show that DAD quickly generates near-optimal storage system designs, improving in both speed and quality over previous tools.

Original languageEnglish (US)
Pages (from-to)337-374
Number of pages38
JournalACM Transactions on Computer Systems
Volume23
Issue number4
DOIs
StatePublished - Dec 1 2005

Fingerprint

Systems analysis
Bins
Computer systems
Industry

All Science Journal Classification (ASJC) codes

  • Computer Science(all)

Cite this

Anderson, Eric ; Spence, Susan ; Swaminathan, Ram ; Kallahalla, Mahesh ; Wang, Qian. / Quickly finding near-optimal storage designs. In: ACM Transactions on Computer Systems. 2005 ; Vol. 23, No. 4. pp. 337-374.
@article{4aba26d87d2747c094601af22b954df8,
title = "Quickly finding near-optimal storage designs",
abstract = "Despite the importance of storage in enterprise computer systems, there are few adequate tools to design and configure a storage system to meet application data requirements efficiently. Storage system design involves choosing the disk arrays to use, setting the configuration options on those arrays, and determining an efficient mapping of application data onto the configured system. This is a complex process because of the multitude of disk array configuration options, and the need to take into account both capacity and potentially contending I/O performance demands when placing the data. Thus, both existing tools and administrators using rules of thumb often generate designs that are of poor quality. This article presents the Disk Array Designer (DAD), which is a tool that can be used both to guide administrators in their design decisions and to automate the design process. DAD uses a generalized best-fit bin packing heuristic with randomization and backtracking to search efficiently through the huge number of possible design choices. It makes decisions using device models that estimate storage system performance. We evaluate DAD's designs based on traces from a variety of database, filesystem, and e-mail workloads. We show that DAD can handle the difficult task of configuring midrange and high-end disk arrays, even with complex real-world workloads. We also show that DAD quickly generates near-optimal storage system designs, improving in both speed and quality over previous tools.",
author = "Eric Anderson and Susan Spence and Ram Swaminathan and Mahesh Kallahalla and Qian Wang",
year = "2005",
month = "12",
day = "1",
doi = "10.1145/1113574.1113575",
language = "English (US)",
volume = "23",
pages = "337--374",
journal = "ACM Transactions on Computer Systems",
issn = "0734-2071",
publisher = "Association for Computing Machinery (ACM)",
number = "4",

}

Anderson, E, Spence, S, Swaminathan, R, Kallahalla, M & Wang, Q 2005, 'Quickly finding near-optimal storage designs', ACM Transactions on Computer Systems, vol. 23, no. 4, pp. 337-374. https://doi.org/10.1145/1113574.1113575

Quickly finding near-optimal storage designs. / Anderson, Eric; Spence, Susan; Swaminathan, Ram; Kallahalla, Mahesh; Wang, Qian.

In: ACM Transactions on Computer Systems, Vol. 23, No. 4, 01.12.2005, p. 337-374.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Quickly finding near-optimal storage designs

AU - Anderson, Eric

AU - Spence, Susan

AU - Swaminathan, Ram

AU - Kallahalla, Mahesh

AU - Wang, Qian

PY - 2005/12/1

Y1 - 2005/12/1

N2 - Despite the importance of storage in enterprise computer systems, there are few adequate tools to design and configure a storage system to meet application data requirements efficiently. Storage system design involves choosing the disk arrays to use, setting the configuration options on those arrays, and determining an efficient mapping of application data onto the configured system. This is a complex process because of the multitude of disk array configuration options, and the need to take into account both capacity and potentially contending I/O performance demands when placing the data. Thus, both existing tools and administrators using rules of thumb often generate designs that are of poor quality. This article presents the Disk Array Designer (DAD), which is a tool that can be used both to guide administrators in their design decisions and to automate the design process. DAD uses a generalized best-fit bin packing heuristic with randomization and backtracking to search efficiently through the huge number of possible design choices. It makes decisions using device models that estimate storage system performance. We evaluate DAD's designs based on traces from a variety of database, filesystem, and e-mail workloads. We show that DAD can handle the difficult task of configuring midrange and high-end disk arrays, even with complex real-world workloads. We also show that DAD quickly generates near-optimal storage system designs, improving in both speed and quality over previous tools.

AB - Despite the importance of storage in enterprise computer systems, there are few adequate tools to design and configure a storage system to meet application data requirements efficiently. Storage system design involves choosing the disk arrays to use, setting the configuration options on those arrays, and determining an efficient mapping of application data onto the configured system. This is a complex process because of the multitude of disk array configuration options, and the need to take into account both capacity and potentially contending I/O performance demands when placing the data. Thus, both existing tools and administrators using rules of thumb often generate designs that are of poor quality. This article presents the Disk Array Designer (DAD), which is a tool that can be used both to guide administrators in their design decisions and to automate the design process. DAD uses a generalized best-fit bin packing heuristic with randomization and backtracking to search efficiently through the huge number of possible design choices. It makes decisions using device models that estimate storage system performance. We evaluate DAD's designs based on traces from a variety of database, filesystem, and e-mail workloads. We show that DAD can handle the difficult task of configuring midrange and high-end disk arrays, even with complex real-world workloads. We also show that DAD quickly generates near-optimal storage system designs, improving in both speed and quality over previous tools.

UR - http://www.scopus.com/inward/record.url?scp=33745205292&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33745205292&partnerID=8YFLogxK

U2 - 10.1145/1113574.1113575

DO - 10.1145/1113574.1113575

M3 - Article

AN - SCOPUS:33745205292

VL - 23

SP - 337

EP - 374

JO - ACM Transactions on Computer Systems

JF - ACM Transactions on Computer Systems

SN - 0734-2071

IS - 4

ER -