GPU acceleration of the particle-in-cell code OSIRIS with maximal memory utilization

Fully relativistic particle-in-cell (PIC) simulations are crucial in advancing knowledge of plasma physics. Modern state-of-the-art GPU supercomputers offer the potential to perform PIC simulations of unprecedented scale, but require robust, performant, and feature-rich codes to support users. We have addressed this demand by porting the PIC code OSIRIS to GPUs. Our approach strikes a balance between preserving the existing CPU codebase and achieving high performance by accelerating only the particle pushing and sorting routines. Within this design framework we maximize performance by enabling high device memory utilization through tile-based dynamic load balancing and a pre-allocated, manually-managed memory pool for particle data. Additionally, the memory pool eliminates any cost of resizing particle buffers and enables complete fault tolerance during particle communication. We present detailed benchmarks for performance on Perlmutter, and demonstrate that GPU acceleration of only particle routines can yield high performance in a PIC code. Our approach may serve as a model for implementation of GPU acceleration on other mature PIC codes as they transition to full GPU support.