Integrating the spectral analyses of neural networks and climate physics for stable, explainable, and generalizable models

The atmospheric and oceanic turbulent circulations involve a variety of nonlinearly interacting physical processes spanning a broad range of spatial and temporal scales. To make simulations of these turbulent flows computationally tractable, processes with scales smaller than the typical grid size of weather/climate models have to be parameterized. Recently, there has been substantial interest (and progress) in using deep learning techniques to develop data-driven subgrid-scale (SGS) parameterizations for the climate system. Another approach that is rapidly gaining popularity is to learn the entire spatio-temporal variability of the climate system from data, i.e., developing fully data-driven forecast models or emulators. For either of these approaches to be useful and reliable in practice, a number of major challenges have to be addressed. These include 1) instabilities or unphysical drifts, 2) learning in the small-data regime, 3) interpretability, and 4) extrapolation to different parameters. Using several setups of 2D turbulence, two-layer quasi-geostrophic turbulence, Rayleigh-Benard convection, and ERA5 reanalysis, we introduce methods to address (1)-(4). The key aspect of some of these methods is combining the spectral analyses of deep neural networks and turbulence/nonlinear physics, as well as leveraging recent advances in theory and applications of deep learning. We will show how these spectral analyses shed light on the inner workings of the deep neural networks and connect them to the underlying physics, providing a general framework for interpreting and understanding deep neural networks when applied to nonlinear dynamical systems such as the climate system.