Navigation of interacting swimmers using Multi-agent Reinforcement Learning

It is known that aquatic organisms like fish use the velocity fields generated by the wakes of obstacles or other swimmers located upstream to reduce their energy expenditure. In this work, we explore the hydrodynamic benefits of group swimming using two-dimensional simulations of artificial self-propelled swimmers, coupled with multi-agent reinforcement learning. These swimmers utilize a sensory input system that allows them to detect the velocity field and pressure on the surface of their body, which is similar to the lateral line sensing system in biological fish. Deep reinforcement learning is used as a tool to discover optimal swimming patterns at the group level as well as the individual level, as a response to different objectives and flow fields. This can be useful in distinguishing various swimming patterns and their role in achieving higher speed or efficiency, which are desirable objectives in different scenarios. The adaptations in response to changes in the surrounding flow field will be examined by training the swimmers in stationary flow, as well as using uniform inflow. These conditions are representative of conditions encountered by fish in lakes and oceans (stationary flow), as well as during long-distance migration and in rivers (uniform flow). The physical mechanisms revealed can be helpful for developing optimal strategies for efficient collective navigation and coordination of autonomous underwater vehicles.