Robotic Implementation of Online Deep Reinforcement Learning for Autonomous Underwater Navigation

In many robotic applications such as ocean surveying, robots must navigate autonomously in the presence of background flow fields using onboard sensors. Here, we investigate the application of deep reinforcement learning (RL) to discover efficient navigation policies in both simulated and physical environments. Inspired by the wide variety of flow-based navigation techniques found in nature, we compare flow sensing strategies for navigating in a 2D, unsteady simulated flow field, and find that velocity sensors yield highly successful and robust navigation policies. To investigate the real-world feasibility of this deep RL approach, we developed a palm-sized robotic swimmer that can learn online and autonomously. The deep neural network that controls the robot's actions is trained onboard using a high-speed microcontroller. Equipped with sensors, the robot is tasked with learning how to navigate in a 6'x6'x18' water tank.