Emergence of memory manifolds

The ability to store continuous variables in the state of a biological system (e.g. a neural network) is critical for many behaviours. Most models for implementing such a memory manifold require hand-crafted symmetries in the interactions or precise fine-tuning of parameters. We present a general principle that we refer to as frozen stabilisation, which allows a family of neural networks to self-organise to a critical state exhibiting memory manifolds without parameter fine-tuning or symmetries. The principle works by momentarily freezing/slowing the dynamics of a subpopulation, thereby creating a static background input which serves to stabilise the remaining population. These memory manifolds exhibit a true continuum of memory states and can be used as general purpose integrators for inputs aligned with the manifold. Perturbations off the manifold relax back to equilibrium with a heavy-tailed distribution of timescales. The response to inputs over a wide range of timescales is an essentially collective property of the system. Moreover, frozen stabilisation allows robust memory manifolds in small networks, which is relevant to the puzzle of implementing continuous attractors with a small number of neurons in light of recent experimental discoveries.