Noisy and biased learning dynamics of physical systems

Noise and bias are inevitable in most biological and physical processes. In the context of learning, they lead to perturbed learning rules and solutions that differ from the ideal case. In this work, we experimentally and theoretically explore how noise and bias affect physical learning dynamics in periodic training setups.

We train Contrastive Local Learning Networks of resistors to learn two tasks periodically. In ideal conditions, this process converges to an optimal solution for both tasks. However, in the presence of noise and bias, learning does not converge to a fixed point but instead leads to limit cycles in the space of learning degrees of freedom. We characterize these limit cycles by their size and cost error, which follow scaling laws based on the training period. We rationalize these findings with a statistical mechanics description of the learning dynamics. Our results reveal a complex interplay between the geometry of the solution space (linked to task complexity), bias, and noise, leading to distinct learning phases in terms of the training period. Notably, we show that in certain cases, noise and bias can be leveraged to find lower-error solutions.