Robust Reservoir Computing for the Prediction of Chaotic Systems

Reservoir computing with Echo State Networks (ESNs) is an accurate machine learning technique to predict the evolution of chaotic dynamical systems. ESNs have been applied for the prediction of extreme events in turbulent channel flow and learning ergodic averages in thermoacoustics. These studies indicate that ESNs are an accurate tool for the prediction of chaotic dynamics, but they are sensitive to the ``tuning'' of the hyperparameters. In this work, we assess and improve the robustness of existing architectures. First, we find that the commonly used strategy to determine the hyperparameters lacks robustness. Secondly, we propose a validation strategy-the Recycle Validation-to improve robustness. Thirdly, we modify folds selection in existing validation strategies. We call this variant \emph{chaotic}, given its roots in the properties of the underlying signal. Both methods are versatile and can be readily applied to Recurrent Neural Network architectures. We test the robust ESNs on different datasets obtained from 3D ODE systems, including a reduced order model of Rayleigh-B\'{e}nard convection. In all testcases, the robust ESN outperforms the traditional ESN. This work opens up new possibilities for robustly employing reservoir computing to higher-dimensional fluid dynamics.