Statistical Mechanics of GFlowNets

Sampling from high-dimensional distributions has become a central problem across the sciences. However, estimation of the corresponding partition function is plagued by the curse of dimensionality, highlighting the need for new techniques. When the distribution in question exhibits underlying structure, the sampling mechanism may be more easily learned from data, suggesting the use of AI-based approaches. Recent work along these lines has proposed a novel framework for generating samples compositionally, termed "Generative Flow Networks" (GFlowNets). Connections between GFlowNets and maximum entropy reinforcement learning (RL) have been established in recent work, however, several open questions remain. We address these questions based on the mapping between non-equilibrium statistical mechanics (NESM) and RL, gaining novel insights into the construction of GFlowNets. These insights lead us to propose a new algorithm for learning the flow function, derive results on reward shaping in GFlowNets, and develop an approach to solve the zero temperature (ground state) sampling problem.