Compressible multiphase flow simulation at near-exascale via a scalable GPU implementation

Today's, and likely tomorrow's, exascale computers get the lion's share of their computational capabilities from GPUs. We present a method for simulating compressible multiphase flows that efficiently leverages GPU accelerators in the face of otherwise low arithmetic-intensity operations. The simulation method is based on high-order accurate finite-volume reconstructions that perform near the compute roofline of the latest accelerators. Offloading is handled via OpenACC and remote direct memory access (RDMA) keeps network costs low. The implementation is in MFC (mflowcode.github.io), an open-source Fortran-based solver. We observe about a 500-times speedup for an A100 GPU over a single modern Intel CPU core. These numbers correspond to between a 50- and 100-times node-level speed-up on current leadership class computers. Within 3% of ideal weak scaling is observed on OLCF Summit (we test on up to about 13,000 GPUs). We close with a discussion of how the algorithms perform on other architectures, including ARM and POWER-based systems, and what that entails for CFD simulation on future heterogeneous supercomputers.