Deep Learning Theory Beyond the Kernel Limit

Deep learning has emerged as a successful paradigm for solving challenging machine learning and computational problems across a variety of domains. However, theoretical understanding of the training and generalization of modern deep learning methods lags behind current practice. I will give an overview of our recent results in this domain, including a new theory that we derived by applying dynamical field theory to deep learning dynamics. This theory gives insight into internal representations learned by the network under different learning rules.