Structured Neural Codes Enable Sample Efficient Learning Through Code-Task Alignment

Learning from a limited number of experiences requires suitable inductive biases. To identify how inductive biases are implemented in and shaped by neural codes, we study sample-efficient learning of arbitrary stimulus-response maps from arbitrary neural population codes with biologically-plausible readouts. Using the replica method from the physics of disordered systems, we develop an analytical theory that predicts the average case generalization error of the readout as a function of the number of observed examples. Our theory illustrates in a mathematically precise way how the structure of population codes shapes inductive bias and generalization performance during learning. We further illustrate how a match between the code and the task is crucial for sample-efficient learning. We show applications of our theory to visual cortex recordings and to reservoir computing with RNNs.