Optimizing Fermionic Neural Networks with Decision Geometry

Recently, Variational Monte Carlo (VMC) solutions to the quantum many-body problem have experienced tremendous progress thanks to the use of neural network quantum states [1, 2, 3]. While more and more sophisticated ansätze have been designed to tackle a wide variety of many-body problems, little progress has been made on their optimization process. In this talk, we will revisit the Kronecker Factored Approximate Curvature (KFAC), one of the main optimizers used for the most challenging many-body systems [2, 4]. After exposing how KFAC is fundamentally unfit for VMC with Fermionic Neural Networks (FNNs), I will discuss the design of a novel optimization strategy based on decision geometry [5]. As a test bench, we consider a FNN modelling polarized fermions interacting in an harmonic trap. Preliminary results will be reported, showing how this new optimizer outperforms KFAC in terms of stability, accuracy and speed of convergence. Beyond VMC, the versatility of this approach suggests that decision geometry could provide a solid foundation for accelerating a broad class of machine learning problems.

[1] G. Carleo and M. Troyer. Science, 355(6325):602–606, 2017.

[2] D. Pfau, J. S. Spencer, A. G. Matthews, and W. M. C. Foulkes. Phys. Rev. Res., 2(3):033429, 2020.

[3] A. Lovato, C. Adams, G. Carleo, and N. Rocco. Phys. Rev. Res., 4(4):043178, 2022.

[4] J. Martens and R. Grosse. In International conference on machine learning, pages 2408–2417. PMLR, 2015.

[5] A. P. Dawid. Ann. Inst. Stat. Math., 59:77–93, 2007.