Machine learning of sequential data for non-intrusive reduced-order models

We study implementations of machine learning strategies for time-series data for non-intrusive reduced-order models of non-linear partial differential equations. Our reduced space is obtained with an $L_2$-optimal proper orthogonal decomposition (POD) with subsequent truncation. We then study the performance of these techniques for systems that require closure due to insufficient resolution of all of the energy in the system. Accurate non-linear dynamics in POD space are learned through recurrent neural networks and neural ordinary differential equations which utilize history, analogous to the Mori-Zwanzig formalism, to retain the effects of the unresolved modes. We also detail the use of attention to maintain the precision of learning for long-term prediction horizons and conclude by discussing distributed hyperparameter search strategies using asynchronous model-based Bayesian optimization.