Parallel performance of iterative Poisson Solvers for Uniform and Structured Adaptive Grids on a Cray XT5 (Kraken)

The Poisson equation solution is of great importance in areas as diverse as Gravitation and Electrostatics to fractional step methods for viscous incompressible flows. For uniform grids fast direct methods based on FFT are usually adopted, which scale well to thousands of processors. For block-wise structured adaptive grids, scaling depends on the amount of communication and load balancing inherent on the solution method. In the present study we report scaling tests on uniform and two-level grids of increasing size. A direct solver based on trigonometric transforms is used on the uniform grid cases, and three different iterative solvers are used on the adaptive cases. In particular, two multigrid algorithms and a bi-conjugate gradient stabilized algorithm, (BiPCGSTAB), preconditioned by one sweep of the Multigrid (no corrections applied in preconditioned) are utilized. All solvers are implemented within the Flash software infrastructure. The computations are performed on the Kraken, Cray XT5 supercomputer employing up to 40,000 cores. It is seen that interprocessor communication required by the data redistribution operations, and the base level fast solution has an important effect on the decline in parallel scaling.