Free energy and phase diagram of modern Hopfield network with fourth order interactions

The Hopfield network is a spin glass model of neurons connected via long-range interactions, which was developed in 1982 as a model for associative memory. Amit et al. found the free energy of this model using replica theory and used it to find its phase diagram and estimate its capacity for memory storage, which was found to scale with its system size N as 0.14N. In recent days, this model has been extended to include higher order interactions, which significantly increases the memory capacity of the network for the same number of neurons. We have estimated the replica symmetric partition function and free energy of the modern Hopfield network with quartic interactions, and used it to find its equation of state and phases and estimate the memory capacity of this model. We find that the relevant parameter in this case is α₄ = K/N³, where N is the system size and K is the number of memories, and thus the capacity in this case scales as N³, which agrees with Krotov and Hopfield's estimates which were made using the probability of retrieval error.