An objective function for Hebbian self-limiting synaptic plasticity rules

Objective functions, formulated in terms of information theoretical measures with respect to the input and output probability distributions, provide a useful framework for the formulation of guiding principles for information processing systems, such as neural networks. In the present work, a guiding principle for neural plasticity is formulated in terms of an objective function expressed as the Fisher information with respect to an operator that we denote as the synaptic flux \footnote{Echeveste $\&$ Gros, \textbf{Front. Robot. AI} 1, 2014}. By minimization of this objective function, we obtain Hebbian self-limiting synaptic plasticity rules, avoiding unbounded weight growth. Furthermore, we show how the rules are selective to directions of maximal negative excess kurtosis, making them suitable for independent component analysis. As an application, the non-linear bars problem \footnote{F\"{o}ldiak, \textbf{Biol. Cybern.} 64: 165–170, 1990} is studied, in which each neuron is presented with a non-linear superposition of horizontal and vertical bars. We show that, under the here presented rules, the neurons are able to find the independent components of the input.