Déjà View: Spatio-Temporal Compute Reuse for' Energy-Efficient 360° VR Video Streaming

Shulin Zhao, Haibo Zhang, Sandeepa Bhuyan, Cyan Subhra Mishra, Ziyu Ying, Mahmut T. Kandemir, Anand Sivasubramaniam, Chita R. Das

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The emergence of virtual reality (VR) and augmented reality (AR) has revolutionized our lives by enabling a 360° artificial sensory stimulation across diverse domains, including, but not limited to, sports, media, healthcare, and gaming. Unlike the conventional planar video processing, where memory access is the main bottleneck, in 360° VR videos the compute is the primary bottleneck and contributes to more than 50% energy consumption in battery-operated VR headsets. Thus, improving the computational efficiency of the video processing pipeline in a VR is critical. While prior efforts have attempted to address this problem through acceleration using a GPU or FPGA, none of them has analyzed the 360° VR pipeline to examine if there is any scope to optimize the computation with known techniques such as memoization. Thus, in this paper, we analyze the VR computation pipeline and observe that there is significant scope to skip computations by leveraging the temporal and spatial locality in head orientation and eye correlations, respectively, resulting in computation reduction and energy efficiency. The proposed Déjà View design takes advantage of temporal reuse by memoizing head orientation and spatial reuse by establishing a relationship between left and right eye projection, and can be implemented either on a GPU or an FPGA. We propose both software modifications for existing compute pipeline and microarchitectural additions for further enhancement. We evaluate our design by implementing the software enhancements on an NVIDIA Jetson TX2 GPU board and our microarchitectural additions on a Xilinx Zynq-7000 FPGA model using five video workloads. Experimental results show that Déjà View can provide 34% computation reduction and 17% energy saving, compared to the state-of-the-art design.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture, ISCA 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages241-253
Number of pages13
ISBN (Electronic)9781728146614
DOIs
StatePublished - May 2020
Event47th ACM/IEEE Annual International Symposium on Computer Architecture, ISCA 2020 - Virtual, Online, Spain
Duration: May 30 2020Jun 3 2020

Publication series

NameProceedings - International Symposium on Computer Architecture
Volume2020-May
ISSN (Print)1063-6897

Conference

Conference47th ACM/IEEE Annual International Symposium on Computer Architecture, ISCA 2020
CountrySpain
CityVirtual, Online
Period5/30/206/3/20

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Déjà View: Spatio-Temporal Compute Reuse for' Energy-Efficient 360° VR Video Streaming'. Together they form a unique fingerprint.

Cite this