The effect of learning on information content in learning machines

Recent work has demonstrated the ability to train mechanical, flow, and electrical networks to perform desired tasks using local learning rules. The ability to perform a task is a collective property of the network that arises from the interactions among the edges. Here we use information measures to study the learning process, starting with lossless compression of the network’s response. A flow network on a square lattice makes a perfect candidate for this study for its simplicity and trainability. We train ensembles of networks to deliver specified pressures at specified output nodes in response to specified pressures at input nodes, and observe the evolution of the information content during training, and its relation to system size and task complexity.