Understanding vision through the lens of prediction

Prediction is necessary for long-term planning and decision-making, but prediction is also essential to the short-term calculations necessary to overcome the sensory and motor delays present in all neural systems. In order to interact appropriately with a changing environment, the brain must respond not only to the current state of sensory inputs but to rapid predictions of these inputs' future state. To test whether the visual system performs optimal predictive compression and computation, we compute the past and future stimulus information in populations of retinal ganglion cells, the output cells of the retina, in salamanders and rats. By controlling the motion statistics of the input stimulus presented to the retina, a moving bar with inertia making a random walk in space, we can derive the optimal tradeoff between compressing information about the past stimulus while retaining as much information as possible about the future stimulus. By changing parameters in the equation of motion for the bar, we can explore qualitatively different motion prediction problems. We show that retinal ganglion cells sit near this optimum for some motion types but not others, and compare these results between the two sampled species. Taking the next step towards exploring the predictive capacity of neural systems, we characterize the ensemble of spatiotemporal correlations present in the natural environment. To do so, we construct and analyze a database of natural motion videos. We have made high-speed, high-pixel-depth recordings of natural scenes and have preliminary data quantifying the space-time power spectra and the local motion content of these scenes.