TY - GEN
T1 - Characterizing AI Model Inference Applications Running in the SGX Environment
AU - Jing, Shixiong
AU - Bao, Qinkun
AU - Wang, Pei
AU - Tang, Xulong
AU - Wu, Dinghao
N1 - Funding Information:
ACKNOWLEDGEMENT The authors sincerely thank all the reviewers for their constructive feedback and suggestions. This work is supported in part by NSF grant #2011146 and startup funding from the University of Pittsburgh.
Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Intel Software Guard Extensions (SGX) is a set of extensions built into Intel CPUs for the trusted computation. It creates a hardware-assisted secure container, within which programs are protected from data leakage and data manipulations by privileged software and hypervisors. With the trend that more and more machine learning based programs are moving to cloud computing, SGX can be used in cloud-based Machine Learning applications to protect user data from malicious privileged programs.However, applications running in SGX suffer from several overheads, including frequent context switching, memory page encryption/decryption, and memory page swapping, which significantly degrade the execution efficiency. In this paper, we aim to i) comprehensively explore the execution of general AI applications running on SGX, ii) systematically characterize the data reuses at both page granularity and cacheline granularity, and iii) provide optimization insights for efficient deployment of machine learning based applications on SGX. To the best of our knowledge, our work is the first to study machine learning applications on SGX and explore the potential of data reuses to reduce the runtime overheads in SGX.
AB - Intel Software Guard Extensions (SGX) is a set of extensions built into Intel CPUs for the trusted computation. It creates a hardware-assisted secure container, within which programs are protected from data leakage and data manipulations by privileged software and hypervisors. With the trend that more and more machine learning based programs are moving to cloud computing, SGX can be used in cloud-based Machine Learning applications to protect user data from malicious privileged programs.However, applications running in SGX suffer from several overheads, including frequent context switching, memory page encryption/decryption, and memory page swapping, which significantly degrade the execution efficiency. In this paper, we aim to i) comprehensively explore the execution of general AI applications running on SGX, ii) systematically characterize the data reuses at both page granularity and cacheline granularity, and iii) provide optimization insights for efficient deployment of machine learning based applications on SGX. To the best of our knowledge, our work is the first to study machine learning applications on SGX and explore the potential of data reuses to reduce the runtime overheads in SGX.
UR - http://www.scopus.com/inward/record.url?scp=85123167745&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85123167745&partnerID=8YFLogxK
U2 - 10.1109/NAS51552.2021.9605445
DO - 10.1109/NAS51552.2021.9605445
M3 - Conference contribution
AN - SCOPUS:85123167745
T3 - 2021 IEEE International Conference on Networking, Architecture and Storage, NAS 2021 - Proceedings
BT - 2021 IEEE International Conference on Networking, Architecture and Storage, NAS 2021 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 15th IEEE International Conference on Networking, Architecture and Storage, NAS 2021
Y2 - 24 October 2021 through 26 October 2021
ER -