Data Compression for Turbulence Databases Using Spatio-Temporal Sub-Sampling and Local Re-simulation

The size of datasets from numerical simulations of turbulent flows has been growing roughly at the rate expected from Moore's law. However, storing the rapidly growing amounts of numerical simulation data is very challenging. Motivated by specific data and accuracy requirements for building numerical databases of turbulent flows, data compression using spatio-temporal sub-sampling and local re-simulation is proposed. The DNS data are aggressively subsampled in space and time during the simulation, and a local re-simulation is to be performed later when the data are needed. Numerical re-simulation experiments for decaying isotropic turbulence based on sub-sampled data are undertaken. The results and error analyses are used to establish parameter choices for sufficiently accurate sub-sampling and sub-domain re-simulation. It is found that the numerical algorithms of the re-simulation have to match the original simulation exactly to reproduce error-free results: any mismatch between the two is found to lead to surprisingly large errors. By carefully controlling the re-simulation parameters, re-simulation errors close to single-precision machine accuracy can be achieved under stringent conditions.