Artificial structures using reinforcement learning based matter manipulation

Matter manipulation using the scanning tunneling microscope (STM) enables the fabrication of artificial structures that host exotic quantum states. However, the sensitive nature of the STM tip limits exploration of diverse configurations and construction of property-specific artificial lattices. Here we demonstrate a reinforcement learning (RL) based framework, integrated with automated workflows, to create artificial structures by manipulating carbon-monoxide (CO) molecules on a copper surface using the STM tip. The automated workflow involves a combination of molecule detection and manipulation. We use deep learning-based object detection to locate the position of the CO molecules, and linear assignment to allocate the molecules to designated target positions. For the manipulation, we initially execute molecule maneuvering procedures based on random sampling of the parameters- bias, setpoint and the tip speed. This dataset is formulated into an action trajectory, used to train the RL agent. The model is subsequently deployed on the STM for fine-tuning during the creation of artificial structures. Our approach incorporates additional techniques such as data augmentation, drift compensation, and precise instrument controls to realize artificial quantum structures.