Deep Learning for In-situ Compression of Large CFD Simulations

The ExaWind project seeks to develop blade-resolved LES simulations of wind turbines for next-generation exascale computing architectures. Such simulations are expected to generate data with over a billion degrees of freedom and upwards of a million time steps, requiring significant computational resources to be dedicated to data storage, visualization, and analysis. In many cases, performing these tasks on the full dataset is intractable, prompting the need for in-situ data compression. The singular value decomposition (SVD) is the standard matrix compression approach; however, the linear nature of the low-rank approximation limits its ability to reconstruct highly nonlinear turbulent flow data. In this work, we explore deep learning methods for in-situ data compression, specifically a deep convolution autoencoder network that that maps 3D turbulent fields to a low-dimensional latent space. We compare the autoencoder against single-pass randomized SVD approaches in lossy restart studies where simulations are check-pointed and restarted. Our results show that an autoencoder trained on canonical turbulent flows can be applied to unseen configurations and is competitive with a single-pass SVD in terms of compression ratio, error, and computational cost.