Jacobian Eigenvalue Analysis for the Stability of Neural Autoregressive Models of Chaotic Dynamic Systems

Neural network-based autoregressive methods for predicting complex multi-scale chaotic dynamical systems suffer from instability. Over long timeframes, compounding errors eventually result in a complete divergence from the training distribution. The lack of explainable dynamics inside the network itself makes it difficult to determine how quickly these autoregressive models would become unstable without exhaustive testing. Previous works have demonstrated that this divergence is caused by cascading errors that start in the small scale modes and propagate into the large scale modes, which result in eventual instability. The timeframe at which this process happens is currently not determinable without running to model until said divergence. Spectral analysis of the Jacobian of the model, with a focus on its largest eigenvalue in particular, gives a quantifiable metric to determine how quickly the model can become unstable. This work demonstrates this fact by analyzing the instabilities of multiple network architectures, and multiple numerical integration methods, on a canonical chaotic dynamical system.