Autonomous Nanocars based on Reinforcement Learning

At the world’s first nanocar race at CEMES-CNRS, in France, participants had to direct a nanocar across a “racetrack” [1]. In order to control their nanocar, they had to move it using the tip of a STM, albeit without making direct contact with the nanocar.

The physics that govern the molecule’s movement and rotation is complex and involves the interaction between the molecule and the tip as well as the molecule and the substrate [2]. Thus, it requires some expertise from humans to manoeuvre the nanocar and predict the outcome of a performed action.

Here, we show how an artificial intelligence (AI) based on reinforcement learning (RL) can be implemented to manipulate single molecules. The AI is implemented in the form of an off-policy RL algorithm, known as the Q-Learning. Being off-policy enables the AI to learn without the necessity of a physical model. In a prime example, the AI manoeuvres the nanocar with a success rate of 89%.

Our results can be the basis for more sophisticated techniques of molecular manipulations which allow identification and relocation of single molecules at will, building the basis for future bottom-up constructions of nanotechnology.


[1] G. Rapenne et al., Nature Rev. Mater. 2, 17040 (2017)
[2] G. J. Simpson et al., Nature Nanotech. 12, 604 (2017)