Phase transition prediction and entropy estimation with restricted Boltzmann machines

The restricted Boltzmann machine (RBM) is an energy-based model consisting of a visible and a hidden layer. Previous studies applying the RBM to the Ising model demonstrated its ability of learning the Boltzmann distribution and reconstructing thermal quantities. However, how the RBM extracts physical information and captures the phase transition without extra human knowledge are not fully explored. We train RBMs on 2d and 3d Ising model with a system size much larger than those used before and carefully examine the mechanism of RBM learning. By analyzing machine learning parameters and functions, such weight matrix, hidden layer embedding, visible energy, and pseudo-likelihood, we find several characteristics for the phase transition. We verify that the hidden layer contains an approximately equal number of positive and negative units without special spatial correlation. We also prove that the pseudo-likelihood can be used to estimate the (physical) entropy.