Automated experiment in microscopy towards discovering new physics and new materials

In this presentation, I will discuss recent progress in machine learning applications in electron microscopy, ranging from feature extraction, learning generative physical models, and to physics discovery via active learning. The applications of classical deep learning methods in streaming image analysis are strongly affected by the out of distribution drift effects, and the approaches to minimize these to enable real time feature finding are discussed. I further illustrate discovery of structure-property relationships on the example of plasmonic structures. Finally, I illustrate transition from post-experiment data analysis to active learning process. Here, the strategies based on simple Gaussian Processes often tend to produce sub-optimal results due to the lack of prior knowledge and very simplified (via learned kernel function) representation of spatial complexity of the system. Comparatively, deep kernel learning (DKL) methods allow to realize both the exploration of complex systems towards the discovery of structure-property relationship, and enable automated experiment targeting physics (rather than simple spatial feature) discovery. The latter is illustrated via experimental discovery of the edge plasmons in STEM/EELS and 4D STEM exploration of twisted bilayer structures.