We describe NekRS, which is an open source spectral element thermal/fluids code designed for exascale platforms. NekRS supports both incompressible and low-Mach formulations, ALE-based moving meshes, uRANS, and numerous boundary conditions of relevance to engineering flows. Second- and third-order timesteppers are provided, using either standard (CFL-limited) semi-implicit methods or characteristics-based subcycling for advection. A variety of scalable multilevel preconditioners for the pressure Poisson problem have been implemented to ensure optimal performance across a large application space. Several simulation results are presented, including turbulent flow in the full core of a pebble bed reactor that features 352,000 spherical pebbles in an annular domain. The mesh features 98.8 million elements of order N=8, for a total of n=50.5 billion gridpoints. Using 27648 NVIDIA V100s on the OLCF supercomputer, Summit, the wall-clock time for this case is 0.25 seconds per step, corresponding to 6 hours for an entire flow-through time.
–
Publication: Paul Fischer, Stefan Kerkemeier, Misun Min, Yu-Hsiang Lan, Malachi Phillips, Thilina Rath- nayake, Elia Merzari, Ananias Tomboulides, Ali Karakus, Noel Chalmers, and Tim Warburton. NekRS, a GPU-accelerated spectral element Navier-Stokes solver (under review).<br><br>Paul Fischer, Elia Merzari, Misun Min, Stefan Kerkemeier, Yu-Hsiang Lan, Malachi Phillips, Thilina Rathnayake, April Novak, Derek Gaston, Noel Chalmers, and Tim Warburton. Highly optimized full-core reactor simulations on Summit. (2021).<br><br>Yu-Hsiang Lan, Paul Fischer, Elia Merzari, and Misun Min. All-hex meshing strategies for densely packed spheres. In The 29th International Meshing Roundtable, 2021.
