TY - JOUR
T1 - SolarDB
T2 - Toward a shared-everything database on distributed log-structured storage
AU - Zhu, Tao
AU - Zhao, Zhuoyue
AU - Li, Feifei
AU - Qian, Weining
AU - Zhou, Aoying
AU - Xie, Dong
AU - Stutsman, Ryan
AU - Li, Haining
AU - Hu, Huiqi
N1 - Funding Information:
This article is an extended version of the paper entitled Solar: Towards a Shared-Everything Database on Distributed Log-Structured Storage, published in USENIX ATC’18. T. Zhu, W. Qian, and A. Zhou were supported by the 863 Program (2015AA015307), National Key R&D Plan Project (2018YFB1003303), and NSFC (61432006 and 61332006). F. Li, Z. Zhao, and D. Xie were supported in part by NSF grants 1619287 and 1443046. F. Li was also supported in part by NSFC grant 61729202. R. Stutsman was supported in part by NSF grant CNS-1750558. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. Authors’ addresses: T. Zhu, W. Qian, and A. Zhou, East China Normal University, 3663 Zhongshan N. Rd, Shanghai, China 200241; emails: zhutao@stu.ecnu.edu.cn, {wnqian, ayzhou}@sei.ecnu.edu.cn; Z. Zhao, F. Li, D. Xie, and R. Stutsman, University of Utah, 50 Central Campus Dr, Salt Lake City, UT, USA, 84112; emails: {zyzhao, lifeifei, dongx, stutsman}@cs.utah.edu; H. Li and H. Hu, Bank of Communications, 188 Yincheng Middle Rd, Shanghai, China, 200090; emails: {lihn, hqhu}@bankcomm.com. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2019 Association for Computing Machinery. 1553-3077/2019/06-ART11 $15.00 https://doi.org/10.1145/3318158
Funding Information:
T. Zhu, W. Qian, and A. Zhou were supported by the 863 Program (2015AA015307), National Key R&D Plan Project (2018YFB1003303), and NSFC (61432006 and 61332006). F. Li, Z. Zhao, and D. Xie were supported in part by NSF grants 1619287 and 1443046. F. Li was also supported in part by NSFC grant 61729202. R. Stutsman was supported in part by NSF grant CNS-1750558. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Publisher Copyright:
© 2019 Association for Computing Machinery.
PY - 2019/6
Y1 - 2019/6
N2 - Efficient transaction processing over large databases is a key requirement for many mission-critical applications. Although modern databases have achieved good performance through horizontal partitioning, their performance deteriorates when cross-partition distributed transactions have to be executed. This article presents SolarDB, a distributed relational database system that has been successfully tested at a large commercial bank. The key features of SolarDB include (1) a shared-everything architecture based on a twolayer log-structured merge-tree; (2) a new concurrency control algorithm that works with the log-structured storage, which ensures efficient and non-blocking transaction processing even when the storage layer is compacting data among nodes in the background; and (3) find-grained data access to effectively minimize and balance network communication within the cluster. According to our empirical evaluations on TPC-C, Smallbank, and a real-world workload, SolarDB outperforms the existing shared-nothing systems by up to 50x when there are close to or more than 5% distributed transactions.
AB - Efficient transaction processing over large databases is a key requirement for many mission-critical applications. Although modern databases have achieved good performance through horizontal partitioning, their performance deteriorates when cross-partition distributed transactions have to be executed. This article presents SolarDB, a distributed relational database system that has been successfully tested at a large commercial bank. The key features of SolarDB include (1) a shared-everything architecture based on a twolayer log-structured merge-tree; (2) a new concurrency control algorithm that works with the log-structured storage, which ensures efficient and non-blocking transaction processing even when the storage layer is compacting data among nodes in the background; and (3) find-grained data access to effectively minimize and balance network communication within the cluster. According to our empirical evaluations on TPC-C, Smallbank, and a real-world workload, SolarDB outperforms the existing shared-nothing systems by up to 50x when there are close to or more than 5% distributed transactions.
UR - http://www.scopus.com/inward/record.url?scp=85074823571&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85074823571&partnerID=8YFLogxK
U2 - 10.1145/3318158
DO - 10.1145/3318158
M3 - Article
AN - SCOPUS:85074823571
SN - 1553-3077
VL - 15
JO - ACM Transactions on Storage
JF - ACM Transactions on Storage
IS - 2
M1 - 2
ER -