TY - GEN
T1 - SHOWAR
T2 - 12th Annual ACM Symposium on Cloud Computing, SoCC 2021
AU - Baarzi, Ataollah Fatahi
AU - Kesidis, George
N1 - Funding Information:
Finally, we plan to improve the SHOWAR’s affinity and anti-affinity rule generators. Currently we determine affinity pairwise between microservices using simple empirical resource-utilization correlation coefficients. We can in the future explore, for example, the impact on affinities of other statistics such as cross-correlations between different types of resources, and explore different types of scheduling mechanisms that can exploit such “raw" statistical information directly toward more efficient resource utilization [19]. 7 Conclusion In summary, we propose SHOWAR a framework consisting of a vertical autoscaler, a horizontal autoscaler and a scheduling affinity rule generator for the microservices. SHOWAR’s vertical autoscaler embraces the empirical variance in the historical resource usage to find the optimal size and reduce the resource usage slack (the difference between allocated resource and actual resource usage). For horizontal scaling SHOWAR uses ideas from control theory along with kernel level performance metrics (i.e. eBPF runq latency) to perform accurate horizontal scaling actions. In particular uses proportional–integral–derivative controller (PID controller) as a stateful controller to control the number of replicas for each microservices. The vertical and horizontal autoscalers in SHOWAR work in tandem to improve the resource utilization while maintaining a good performance. Additionally, once the size for each microservices is found, SHOWAR bridges the gap between optimal resource allocation and scheduling by generating affinity hints for the task scheduler to further improve performance. Our empirical experiments using a variety of microservice applications and real-world workloads show that, compared to state of the art autoscaling systems, on average SHOWAR improves the resource allocation by up to 22% (and hence saving in costs) while improving the 99th percentile end-to-end user request latency by 20%. Acknowledgments We thank Mohammad Shahrad for his valuable feedback on this work. We also thank the anonymous reviewers and our shepherd, Jana Giceva, for helping us improve the paper. This work was supported in part by NSF CCF grant 2028929 and NSF CNS grant 2122155.
Publisher Copyright:
© 2021 Association for Computing Machinery.
PY - 2021/11/1
Y1 - 2021/11/1
N2 - Microservices architecture have been widely adopted in designing distributed cloud applications where the application is decoupled into multiple small components (i.e. "microservices"). One of the challenges in deploying microservices is finding the optimal amount of resources (i.e. size) and the number of instances (i.e. replicas) for each microservice in order to maintain a good performance as well as prevent resource wastage and under-utilization which is not cost-effective. This paper presents SHOWAR, a framework that configures the resources by determining the number of replicas (horizontal scaling) and the amount of CPU and Memory for each microservice (vertical scaling). For vertical scaling, SHOWAR uses empirical variance in the historical resource usage to find the optimal size and mitigate resource wastage. For horizontal scaling, SHOWAR uses basic ideas from control theory along with kernel level performance metrics. Additionally, once the size for each microservice is found, SHOWAR bridges the gap between optimal resource allocation and scheduling by generating affinity rules (i.e. hints) for the scheduler to further improve the performance. Our experiments, using a variety of microservice applications and real-world workloads, show that, compared to the state-of-the-art autoscaling and scheduling systems, SHOWAR on average improves the resource allocation by up to 22% while improving the 99th percentile end-to-end user request latency by 20%.
AB - Microservices architecture have been widely adopted in designing distributed cloud applications where the application is decoupled into multiple small components (i.e. "microservices"). One of the challenges in deploying microservices is finding the optimal amount of resources (i.e. size) and the number of instances (i.e. replicas) for each microservice in order to maintain a good performance as well as prevent resource wastage and under-utilization which is not cost-effective. This paper presents SHOWAR, a framework that configures the resources by determining the number of replicas (horizontal scaling) and the amount of CPU and Memory for each microservice (vertical scaling). For vertical scaling, SHOWAR uses empirical variance in the historical resource usage to find the optimal size and mitigate resource wastage. For horizontal scaling, SHOWAR uses basic ideas from control theory along with kernel level performance metrics. Additionally, once the size for each microservice is found, SHOWAR bridges the gap between optimal resource allocation and scheduling by generating affinity rules (i.e. hints) for the scheduler to further improve the performance. Our experiments, using a variety of microservice applications and real-world workloads, show that, compared to the state-of-the-art autoscaling and scheduling systems, SHOWAR on average improves the resource allocation by up to 22% while improving the 99th percentile end-to-end user request latency by 20%.
UR - http://www.scopus.com/inward/record.url?scp=85119298171&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85119298171&partnerID=8YFLogxK
U2 - 10.1145/3472883.3486999
DO - 10.1145/3472883.3486999
M3 - Conference contribution
AN - SCOPUS:85119298171
T3 - SoCC 2021 - Proceedings of the 2021 ACM Symposium on Cloud Computing
SP - 427
EP - 441
BT - SoCC 2021 - Proceedings of the 2021 ACM Symposium on Cloud Computing
PB - Association for Computing Machinery, Inc
Y2 - 1 November 2021 through 4 November 2021
ER -