Associative memory and basin structure in multi-state recurrent neural networks

Traditional neural network models of associative memories, such as the prestigious Hopfield network, were primarily used to store and retrieve static patterns. We extend this paradigm by studying the multi-state recurrent neural network framework (RNN) capable of storing and retrieving complex dynamical attractors, under two common recall scenarios in neuropsychology: location-addressable and content-addressable. We use reservoir computing and long short-term memory (LSTM) networks as examples of RNNs.

We demonstrate that, for location-addressable retrieval, a single RNN can memorize a large number of periodic and chaotic attractors, each retrievable with a specific index value. We articulate control strategies to achieve successful switching among the attractors, unveil the mechanism behind failed switching, and uncover various scaling behaviors between the number of stored attractors and the network size.

For content-addressable retrieval, we exploit multistability with cue signals, where the stored attractors coexist in the high-dimensional hidden space of the RNN. As the length of the cue signal increases through a critical value, a high success rate can be achieved. We find surprisingly complex basin structures in such a multi-state neural network, with novel features never observed before. These structures also exhibit some characteristics similar to those seen in other highly multistable systems, suggesting the generality of these features across nonlinear dynamical systems under certain conditions.

Our work provides important insights into associative memory in neural networks, specifically in developing long-term memories for complex dynamical behaviors and understanding the mechanisms behind memory retrieval success and failure. Additionally, we offer new insights into the basin structure of highly multi-state dynamical systems.