To move or not to move? Page migration for irregular applications in over-subscribed GPU memory systems with DynaMap

Chia Hao Chang, Adithya Kumar, Anand Sivasubramaniam

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper focuses on the severe page thrashing problem that can arise when running large irregular memory access applications on limited GPU memory systems. Such memory over-subscription causes very poor performance in the currently on demand (eager) or page-group granularity access-counter based (lazy) page migration mechanisms found in NVIDIA's UVM drivers. Our detailed analysis of these executions reveals a very novel insight: rather than duplicate the responsibility of catering to both temporal and spatial locality in both GPU caches and its memory, it is better for the former to simply cater to the temporal aspect, and the latter to the spatial aspect, thereby saving precious memory system capacities. Based on this, we build an adaptive page migration scheme, called DynaMap, that (i) uses a compiler pass to instrument off-the-shelf CUDA UVM applications for spatial utilization tracking, (ii) dynamically sets a spatial utilization threshold to determine migration based on memory pressure and access characteristics, and (iii) enhances the current NVIDIA UVM driver to dynamically migrate the page (from the host memory to the GPU) based on the threshold. Using 7 irregular applications from public benchmark suites, we implement DynaMap on a real system with different over-subscription ratios to show speedups as much as 2.5X (34% on the average) over state-of-the-art UVM implementations.

Original languageEnglish (US)
Title of host publicationSYSTOR 2021 - Proceedings of the 14th ACM International Conference on Systems and Storage
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450383981
DOIs
StatePublished - Jun 14 2021
Event14th ACM International Conference on Systems and Storage, SYSTOR 2021 - Virtual, Online, Israel
Duration: Jun 14 2021Jun 16 2021

Publication series

NameSYSTOR 2021 - Proceedings of the 14th ACM International Conference on Systems and Storage

Conference

Conference14th ACM International Conference on Systems and Storage, SYSTOR 2021
Country/TerritoryIsrael
CityVirtual, Online
Period6/14/216/16/21

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • Computer Science Applications
  • Hardware and Architecture
  • Software

Cite this