Towards interpretable and reliable machines learning physics

Machine learning (ML) models are known for their black-box construction, i.e., they usually hinder any insight into the reasoning behind their predictions. This opaqueness increases with the model complexity and expressivity and is inherent to neural networks. As a result, neither we can fully trust their predictions (lack of reliability) nor learn what the ML model learned (lack of interpretability). Both qualities are crucial, especially if we want to apply ML to novel problems with unknown solutions. Instead of limiting ourselves to simple interpretable models, we can look at the neural network through the lens of its Hessian which encodes valuable information on the model. We show how the Hessian-based methods can extract a concept of similarity between input data learned by a network, estimate the uncertainty of model predictions, or judge whether a network makes an extrapolating guess instead of a data-based decision. We present these gains on the example of convolutional neural networks learning quantum phases both in the numerically simulated and experimental data.