Why are the data-driven surrogates of multi-scale dynamical systems long-term unstable?

Today's state-of-the-art computational algorithms that model and predict the states of these dynamical systems numerically solve discretized versions of the partial differential equations that govern these systems. While such approaches have yielded tremendous success, especially in large-scale scientific problems e.g., fluid dynamics, weather and climate modeling, etc., they come at an enormous computational cost. Hence, recent efforts in building data-driven surrogates for high-dimensional dynamical systems for forecasting applications have received much attention and garnered noticeable success. These autoregressive data-driven models yield significantly competitive short-term forecasting results (as compared to traditional numerical models) at a fraction of the computational cost of numerical models. However, these data-driven models do not remain stable when time-integrated for a long time. Such a long time-integration is often essential for gathering insights into the statistics of dynamical systems e.g., extreme events. While many studies have reported this instability, especially for data-driven models of turbulent flows, a causal mechanism for this instability is not clear. Most efforts to obtain stability are ad-hoc and empirical. In this work, we present a causal mechanism for this instability observed in data-driven models of turbulent flows through the lens of a phenomenon called “spectral bias”. Furthermore, we provide a rigorous solution to improve the stability of these data-driven models.