Interpretable transfer learning: Applications to climate change modeling

Recent studies have found promising results using machine learning (ML) techniques such as convolutional neural networks (CNNs) to improve climate models, e.g., by better representing subgrid-scale processes. However, applying NN-enhanced models to a different climate system, for example with a different radiative forcing, can lead to inaccurate and even unstable simulations. This is because NNs and similar techniques cannot be expected to work accurately outside their training manifold, i.e, they often do not extrapolate. Transfer learning (TL), which involves re-training some layers with a small amount of new data, offers a solution to this and a few recent studies have found promising results in simple test cases. However, the general understanding of TL, mainly from applications involving static images, does not apply to climate modeling. Here, we present a framework to guide TL for applications involving climate modeling. The framework is based on the spectral analysis of the new data and inputs, weights, and outputs of the re-trained layers in TL. Using 2D turbulence as the test case, we show how this framework connects with the physics of the flow and some of the recent advances in the ML community on the training of NNs.