Statistical Physics Meets Machine Learning I

Presentations

• Scaling Laws in Deep Neural Networks: Insights from Statistical Mechanics and Exactly Solvable Models

  ORAL · Invited

  March 6, 2023, 6:00 PM – March 6, 2023, 6:36 PM

  Publication: None

  Presenters

  • Yasaman Bahri

    Google LLC

  Authors

  • Yasaman Bahri

    Google LLC

  View abstract →

• Results from a Mapping Between Reinforcement Learning and Non-Equilibrium Statistical Mechanics

  ORAL

  March 6, 2023, 6:36 PM – March 6, 2023, 6:48 PM

  Publication: "Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learning": J. Adamczyk, A. Arriojas, S. Tiomkin, R. V. Kulkarni; under review at AAAI-23<br><br>"Analytical framework for maximum entropy reinforcement learning using large deviation theory": A. Arriojas, J. Adamczyk, S. Tiomkin, R. V. Kulkarni; under review at Physical Review Research

  Presenters

  • Jacob Adamczyk

    University of Massachusetts Boston

  Authors

  • Jacob Adamczyk

    University of Massachusetts Boston

  • Argenis Arriojas Maldonado

    University of Massachusetts Boston

  • Stas Tiomkin

    San Jose State University

  • Rahul V Kulkarni

    University of Massachusetts Boston

  View abstract →

• The Onset of Variance-Limited Behavior for Neural Networks at Finite Width and Sample Size.

  ORAL

  March 6, 2023, 6:48 PM – March 6, 2023, 7:00 PM

  Publication: Planned paper "The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes".

  Presenters

  • Alexander B Atanasov

    Harvard University

  Authors

  • Alexander B Atanasov

    Harvard University

  • Cengiz Pehlevan

    Harvard University

  • Blake Bordelon

    Harvard University

  • Sabarish Sainathan

    Harvard University

  View abstract →

• Feature learning and overfitting in neural networks

  ORAL

  March 6, 2023, 7:00 PM – March 6, 2023, 7:12 PM

  Publication: arxiv: 2206.12314<br>Accepted to NeurIPS 2022

  Presenters

  • Francesco Cagnetta

    Ecole Polytechnique Federale de Lausanne

  Authors

  • Francesco Cagnetta

    Ecole Polytechnique Federale de Lausanne

  View abstract →

• Flatter, Faster; Scaling Momentum for Optimal Speedup of SGD

  ORAL

  March 6, 2023, 7:12 PM – March 6, 2023, 7:24 PM

  Presenters

  • Aditya Cowsik

    Stanford University

  Authors

  • Aditya Cowsik

    Stanford University

  • Tankut U Can

    Institute for Advanced Study, The School of Natural Sciences at the Institute for Advanced Study at Princeton

  • Paolo Glorioso

    Stanford University

  View abstract →

• Statistical Mechanics of Infinitely-Wide Convolutional Networks

  ORAL

  March 6, 2023, 7:24 PM – March 6, 2023, 7:36 PM

  Publication: [1] Alessandro Favero, Francesco Cagnetta, and Matthieu Wyart. "Locality defeats the curse of dimensionality in convolutional teacher-student scenarios." Advances in Neural Information Processing Systems 34 (2021): 9456-9467.<br>[2] Francesco Cagnetta, Alessandro Favero, and Matthieu Wyart. "What can be learnt with wide convolutional neural networks?" arXiv preprint arXiv:2208.01003 (2022).

  Presenters

  • Alessandro Favero

    Ecole Polytechnique Federale de Lausanne

  Authors

  • Alessandro Favero

    Ecole Polytechnique Federale de Lausanne

  • Francesco Cagnetta

    Ecole Polytechnique Federale de Lausanne

  • Matthieu Wyart

    Ecole Polytechnique Federale de Lausanne

  View abstract →

• Phase diagram of training dynamics in deep neural networks: effect of learning rate, depth, and width

  ORAL

  March 6, 2023, 7:36 PM – March 6, 2023, 7:48 PM

  Presenters

  • Dayal Singh Kalra

    University of Maryland, College Park

  Authors

  • Dayal Singh Kalra

    University of Maryland, College Park

  • Maissam Barkeshli

    Joint Quantum Institute, NIST/University of Maryland, College Park, University of Maryland College Park, University of Maryland, College Park

  View abstract →

• Generative probabilistic matrix model of data with different low-dimensional latent structures

  ORAL

  March 6, 2023, 7:48 PM – March 6, 2023, 8:00 PM

  Presenters

  • Philipp Fleig

    Max Planck Institute for Medical Research

  Authors

  • Philipp Fleig

    Max Planck Institute for Medical Research

  • Ilya M Nemenman

    Emory, Emory University

  View abstract →

• The Evolution of the Fisher Information Matrix During Deep Neural Network Training

  ORAL

  March 6, 2023, 8:00 PM – March 6, 2023, 8:12 PM

  Presenters

  • Chase W Goddard

    Princeton University

  Authors

  • Chase W Goddard

    Princeton University

  • David J Schwab

    The Graduate Center, CUNY

  View abstract →

• When does Dual Dimensionality Reduction perform better than Single Dimensionality Reduction?

  ORAL

  March 6, 2023, 8:12 PM – March 6, 2023, 8:24 PM

  Presenters

  • Eslam Abdelaleem

    Emory University

  Authors

  • Eslam Abdelaleem

    Emory University

  • K. Michael Martini

    Emory University

  • Ahmed H Roman

    Emory University

  • Ilya M Nemenman

    Emory, Emory University

  View abstract →

• Physics-Informed featurization of spectral functions

  ORAL

  March 6, 2023, 8:24 PM – March 6, 2023, 8:36 PM

  Presenters

  • Shubhang Goswami

    University of Illinois at Urbana-Champaign

  Authors

  • Shubhang Goswami

    University of Illinois at Urbana-Champaign

  • Kipton M Barros

    Los Alamos Natl Lab, Theoretical Division and CNLS, Los Alamos National Laboratory

  • Matthew R Carbone

    Brookhaven National Laboratory, Computational Science Initiative, Brookhaven National Laboratory

  View abstract →

• Efficient Modelling of Ge₁₅Te₈₅ using Active Learning Methods

  ORAL

  March 6, 2023, 8:36 PM – March 6, 2023, 8:48 PM

  Presenters

  • Thomas Arbaugh

    Wesleyan University

  Authors

  • Thomas Arbaugh

    Wesleyan University

  • Francis W Starr

    Wesleyan Univ

  View abstract →

• A simple model for Grokking modular arithmetic

  ORAL

  March 6, 2023, 8:48 PM – March 6, 2023, 9:00 PM

  Presenters

  • Andrey Gromov

    University of Maryland, College Park

  Authors

  • Andrey Gromov

    University of Maryland, College Park

  View abstract →