Distributional maximum entropy models

Maximum entropy methods build statistical physics models that connect to experimental data on complex, non-equilibrium systems. In particular, models that match the local means and pairwise correlations have been successful in describing networks of real neurons, flocks of birds, sequences in families of proteins, and patterns of gene expression. As experiments provide access to larger and larger systems, however, the full correlation matrix becomes impossible to sample. To address this issue, we can constrain low-order moments of the distributions of means and correlations across variables, rather than their individual values. We implement this "distributional maximum entropy" [Phys Rev Lett 113, 117204 (2014)] approach to match moments of the means and correlations simultaneously. With a small number of moments the system is described by a mean field theory, and solutions organize into distinct blocks, reminiscent of spin-glass behavior. We show that this can be applied to real data on patterns of gene expression or neural activity; numerical solution of the mean field equations is delicate but possible. This framework provides a scalable solution for high-dimensional data, and opens a path to constructing phase diagrams for biological systems in which statistics of the measured activity provide natural axes.