APS Logo

High-Performance Single-Particle Imaging Reconstruction on Pre-Exascale Computing Platforms

ORAL · Invited

Abstract

Single-particle imaging (SPI) reconstruction is the analysis of X-ray diffraction patterns from a light source directed at biological molecules. SPI uses intense X-ray free electron laser (FEL) pulses to obtain a continuous diffraction pattern from single molecules in a serial manner--one molecule at a time The analysis is computationally intensive, requiring diffraction patterns to be sorted into conformations and oriented in 3D for model building and refinement. SPI is also data-intensive, requiring potentially hundreds of thousands of diffraction patterns in order to produce a high-resolution result.

The ExaFEL project is an Exascale Computing Project with the goal of performing SPI reconstruction in quasi-real time where the analysis keeps up with experimental data rates. Upgraded detectors and light sources such as the Linac Coherent Light Source (LCLS) are delivering SPI experimental data at ever higher velocities and larger volumes.

This talk will overview the capabilities of SpiniFEL, the high-performance analytics code for SPI reconstruction developed by the ExaFEL team. SpiniFEL implements the multi-tiered iterative phasing (M-TIP) algorithm for SPI reconstruction. SpiniFEL has been accelerated via GPU offloading as well as through parallel and distributed programming models. It can run using the Message Passing Interface (MPI) or via the task-based Legion Runtime System. SpiniFEL runs on the pre-exascale machines Summit at the Oak Ridge Leadership Computing Facility (OLCF) and Perlmutter at the National Energy Research Scientific Computing Center (NERSC).

We will present results of SpiniFEL runs on pre-exascale computing platforms and outline how SpiniFEL SPI analysis fits into the larger vision of an inter-facility workflow that moves data from acquisition at an experimental facility to computation at a supercomputing center, so scientists can quickly determine structures and produce high quality results efficiently using scarce experimental resources.

Publication: Shih, Yu-hsuan, Garrett Wright, Joakim And'en, Johannes Blaschke and Alex H. Barnett. "cuFINUFFT: a load-balanced GPU library for general-purpose nonuniform FFTs." 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) (2021): 688-697.<br><br>Kommera, Pranay Reddy, Vinay Ramakrishnaiah, Christine Sweeney, Jeffrey Donatelli, and Petrus H. Zwart. GPU-accelerated multitiered iterative phasing algorithm for fluctuation X-ray scattering. J. Appl. Cryst. (2021). 54, 1179–1188.<br><br>Chang, Hsing-Yin, Elliott Slaughter, Seema Mirchandaney, Jeffrey Donatelli, and Chun Hong Yoon. Scaling and Acceleration of Three-Dimensional Structure Determination for Single-Particle Imaging Experiments with SpiniFEL. The 4th Annual<br>Parallel Applications Workshop, Alternatives To MPI+X. November 2021.

Presenters

  • Christine Sweeney

    Los Alamos National Laboratory

Authors

  • Christine Sweeney

    Los Alamos National Laboratory

  • Christine Sweeney

    Los Alamos National Laboratory

  • Johannes Blaschke

    Lawrence Berkeley National Laboratory

  • Hsing-Yin Chang

    SLAC National Accelerator Laboratory

  • Jeffrey Donatelli

    Lawrence Berkeley National Laboratory

  • Antoine Dujardin

    SLAC National Accelerator Laboratory

  • Seema Mirchandaney

    SLAC National Accelerator Laboratory

  • Ariana Peck

    SLAC National Accelerator Laboratory

  • Amedeo Perazzo

    SLAC National Accelerator Laboratory

  • Vinay Ramakrishnaiah

    Los Alamos National Laboratory

  • Elliott Slaughter

    SLAC National Accelerator Laboratory

  • Monarin Uervirojnangkoorn

    SLAC National Accelerator Laboratory

  • Chun Hong Yoon

    SLAC National Accelerator Laboratory

  • Petrus H Zwart

    Lawrence Berkeley National Laboratory