Opening the black box: Improving knowledge-free machine learning with knowledge-based models

In recent years, machine learning methods such as "deep learning" have proven enormously successful for tasks such as image classification, voice recognition, and more. Despite their effectiveness for big-data classification problems, these methods have had limited success for time series prediction, especially for complex systems like those we see in weather, solar activity, and even brain dynamics. In this talk, I will discuss how a Reservoir Computer (RC) - a special kind of artificial neural network that offers a "universal" dynamical system - can draw on its own internal complex dynamics in order to forecast systems like the weather, far beyond the time horizon of other methods. Like many other machine learning architectures, the RC provides a knowledge-free approach because it builds forecasts purely from past measurements without any specific knowledge of the system dynamics. By building a new approach that judiciously combines the knowledge-free prediction of the RC with a knowledge-based model, we demonstrate a further, dramatic, improvement in forecasting complex systems. This hybrid approach can given us new insights into the weaknesses of our knowledge-based models and also reveal limitations in our machine learning system, guiding improvements in both knowledge-free and knowledge-based prediction techniques.