Unsupervised feature extraction in simple physical models through mutual information maximization

When studying systems with many degrees of freedom, a typical problem is to find the correct low-dimensional variables to describe them on a higher level of abstraction. However, sometimes it is not clear how to choose meaningful quantities. By defining relevant features as low dimensional variables that preserve the largest mutual information with the original coordinates of the system, we set up an unsupervised learning technique to automatically extract those features. A variational bound allows to estimate mutual information through deep neural networks. We show example applications to statistical mechanics and classical dynamics.