The information bottleneck powered by deep learning to illuminate micro to macro relationships in complex systems

Deep learning is quickly becoming a ubiquitous tool for studying relationships between microscopic details and macroscopic behavior in complex systems. After a predictive model is successfully trained, however, the route to further science is often unclear as insight remains locked inside the metaphorical black box. Here we show how to leverage the information bottleneck (IB) to provide crucial interpretability and illuminate the rich interplay between details at the smallest scales of complex systems. Grounded in information theory, IB is about the fundamental tradeoff between frugality of detail and fidelity of representation when relating one variable to another. By varying the relative importance of the terms in the IB tradeoff, the nature of a relationship between variables is laid bare as information is sorted by order of relevance. As a practical case study, we apply a variational formulation of IB to simulated glasses and find the most relevant combinations of markers of local structure for determining future rearrangement. We study the sharp transitions which can occur in the IB and cast their significance in terms of physical features. Finally, we probe beyond the comparative relevance of variables by examining how variables are compressed along the IB tradeoff.