Autonomous navigation of simulated swimmers using deep reinforcement learning

Underwater vehicles usually have to rely on a combination of multiple types of navigation sensors such as SONAR for localization and object detection. We explore the feasibility of using hydrodynamic sensors as an alternative system for underwater navigation. This study uses reinforcement learning coupled with 2-dimensional numerical simulations of self propelled swimmers. The artificial swimmers with mechanosensory inputs similar to the lateral line in biological fish are trained using deep reinforcement learning to optimally perform various actions such as obstacle avoidance and autonomous navigation towards a target. The swimmers trained in this manner can then be used to navigate more reliably in previously unmapped areas. By comparing the behavior exhibited by the swimmers in different flows such as uniform flow and Karman vortex fields, we analyze the different optimal strategies for navigation in each scenario. The results can then be used to develop control strategies for unmanned robots and underwater robotic swarms.