Learning underwater navigation using egocentric sensory cues

Navigation in the presence of background flows is an essential yet challenging task for autonomous underwater vehicles. In contrast, fish naturally accomplish such tasks by exploiting ambient flow features. Two major difficulties exist in solving this problem: First, the vehicle has access to flow information only in its immediate surroundings, which discounts an optimization approach over the entire flow field. Second, flow information and target position are available in the vehicle frame of reference. Recent studies proved that reinforcement learning is an effective tool for flow navigation but training a control policy directly with CFD data is very costly in time and memory. Here, we design a reduced-order von Kármán vortex street that resembles a drag wake and we train a swimmer to reach a given target against strong background flows by exploiting the wake. The swimmer uses only local sensory information to perform a continuous reorientation. To compare, we also train the swimmer in the CFD wake and test the two trained policies in each other's environment. We found that while egocentric sensory input poses an obstacle to navigation compared to lab frame data, increasing the number of sensors and proper configuration of the sensors significantly improve the success rate.