Gyrokinetic toroidal simulations on leading multi- and manycore HPC systems

Kamesh Madduri, Khaled Z. Ibrahim, Samuel Williams, Eun Jin Im, Stephane Ethier, John Shalf, Leonid Oliker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

28 Scopus citations

Abstract

The gyrokinetic Particle-in-Cell (PIC) method is a critical computational tool enabling petascale fusion simulation re-search. In this work, we present novel multi- and manycore-centric optimizations to enhance performance of GTC, a PIC-based production code for studying plasma microtur-bulence in tokamak devices. Our optimizations encompass all six GTC sub-routines and include multi-level particle and grid decompositions designed to improve multi-node parallel scaling, particle binning for improved load balance, GPU acceleration of key subroutines, and memory-centric optimizations to improve single-node scaling and reduce memory utilization. The new hybrid MPI-OpenMP and MPI-OpenMP-CUDA GTC versions achieve up to a 2× speedup over the production Fortran code on four parallel systems - clusters based on the AMD Magny-Cours, Intel Nehalem-EP, IBM BlueGene/P, and NVIDIA Fermi architectures. Finally, strong scaling experiments provide insight into parallel scalability, memory utilization, and programmability trade-offs for large-scale gyrokinetic PIC simulations, while attaining a 1.6× speedup on 49,152 XE6 cores.

Original languageEnglish (US)
Title of host publicationProceedings of 2011 SC - International Conference for High Performance Computing, Networking, Storage and Analysis
DOIs
StatePublished - Dec 14 2011
Event2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC11 - Seattle, WA, United States
Duration: Nov 12 2011Nov 18 2011

Publication series

NameProceedings of 2011 SC - International Conference for High Performance Computing, Networking, Storage and Analysis

Other

Other2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC11
CountryUnited States
CitySeattle, WA
Period11/12/1111/18/11

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications

Cite this

Madduri, K., Ibrahim, K. Z., Williams, S., Im, E. J., Ethier, S., Shalf, J., & Oliker, L. (2011). Gyrokinetic toroidal simulations on leading multi- and manycore HPC systems. In Proceedings of 2011 SC - International Conference for High Performance Computing, Networking, Storage and Analysis [23] (Proceedings of 2011 SC - International Conference for High Performance Computing, Networking, Storage and Analysis). https://doi.org/10.1145/2063384.2063415