Accuracy and capacity of modern Hopfield networks with synaptic noise

The Hopfield network, proposed in 1982, uses a network of binary neurons with long-range pairwise interactions to describe associative memory. In 2016, Krotov and Hopfield extended this model to include higher order interactions (corresponding to terms of higher than quadratic order in the Hamiltonian), leading to models which can retrieve a much larger number of memories for the same number of neurons. We have analyzed the accuracy of memory retrieval and estimated the capacity of modern Hopfield networks with n-neuron interactions (for any n ≥ 2) in presence of noise as well as models where the interactions are randomly deleted or clipped to all have the same magnitude, and found that the capacity of the network scales with the system size N as Nⁿ⁻¹ in all cases, even though it is reduced in presence of noise compared to the networks with Hebbian interactions. For n = 2, our results agree with those found previously by McEliece and Sompolinsky. We compare our results with numerical simulations and find them to be in agreement.