The Importance of Experimental Controls: Evidence that Task-Dependent Invariances Define Functional Specialization in Cortical Hierarchy

The recent surge in collaboration between neuro/cognitive scientists and machine learning researchers has generated a large body of work that uses artificial neural nets (ANNs) to model brain function. This work is often focused on strong positive correlations between the output of an ANN layer and observed activations in a brain region in response to a sensory stimulus. But the absence of adequate controls often limits our ability to conclude that mechanisms in the brain are similar to those in the ANN (or any other statistical pattern analyzer). Data and tools like those publicly available at BrainScore.org can be used to provide controls in the form of alternative models that yield similar correlations with functional brain data. Interestingly, these BrainScore comparisons show that models with widely-varying architectures can score similarly well. We argue that rather than suggesting architectural analogies between ANN and brain, activity correlation emerges inevitably in any hierarchical network trained for a given task, based on the invariances required to perform that task. Furthermore, neuron specialization will occur in multi-task ANNs, as determined by the invariances required for each task.