Derivation of Neural Circuits from the Similarity Matching Principle

Our brains analyze high-dimensional datasets streamed by our sensory organs in multiple stages. Sensory cortices, for example, perform tasks like dimensionality reduction, sparse feature discovery and clustering. To model these tasks we pursue an approach analogous to use of action principles in physics and propose a new family of objective functions based on the principle of similarity matching. From these objective functions we derive online distributed algorithms that can be implemented by biological neural networks resembling cortical circuits. Our networks can adapt to changes in the number of latent dimensions or the number of clusters in the input dataset. Furthermore, we formulate minimax optimization problems from which we derive online algorithms with two classes of neurons identified with principal neurons and interneurons in biological circuits. In addition to bearing resemblance to biological circuits, our algorithms are competitive for Big Data applications.