How machine learning and an information theory perspective can enrich research on complex systems

One of the first steps toward understanding a complex system with a multitude of interacting elements is to identify the most informative measurements that can be made. What are the detectable precursors to rearrangement in a disordered solid, or the most informative tests to perform on a hospital patient for predicting eventual outcome? While the search is often navigated heuristically based on an intuitive notion of information, in this work we ground the process in information theory and then employ machine learning and data to the same end. Specifically, we use machine learning to optimize a variant of the Information Bottleneck that allocates information across multiple measurements. In doing so, we map out the information in a complex system as a powerful means to deeper understanding.