Independent Control of Microrobots using Q-learning framework

Automated manipulation of multiple independently controlled Janus microrobots along with efficient and fast motion planning can potentially lead to a breakthrough in the formation of functional microstructures, both engineering and biological, in a scalable and reliable manner. In this talk, we present the first foundational step towards realizing this objective by developing a Q-learning-based controller which approximately optimizes the trajectories of the microrobots through rewards and discounts while avoiding obstacles, such as randomly dispersed micro-objects, or already formed microstructures. The self-learning microrobots can (1) recover a previously known propulsion policy, (2) identify a more effective policy, and (3) adapt the propulsion policy based on their interactions with the surrounding media. We show simulation experiments for independent motion and steering control for groups of one, two, and three microrobots, actuated using electromagnetic coils setup, to demonstrate the feasibility and effectiveness of our method.