Reinforcement Learning for a Bio-Inspired Vehicle with Undulating Fin Propulsion.

Undulating fins provide natural swimmers with fascinating locomotion capabilities. However, the use of undulating fin propulsion for underwater vessel to perform specific maneuvers is non-trivial. Currently, researchers implement ad-hoc fin kinematics such as sinusoidal traveling waves due to ease of implementation, or use kinematics based on live animals. In the proposed work, we integrate reinforcement learning with simulations and underwater robotic experiments to determine optimal undulating fin kinematics for a variety of swimming performance. A six degree-of-freedom numerical model is used to simulate the motion of a vessel with an undulating fin. The swimmer is modeled as a rigid body with a single fin running along the length of the body. A reinforcement learning algorithm was developed to determine the optimal kinematics of the fin for basic locomotion, including straight swimming and turning. We find that a model-free self-learning approach can be used to generate more complex actuation combination for improved performance. The simulation results are compared to the performance of a bio-inspired vehicle with an undulating fin propulsion. The results indicate that the use of reinforcement learning could be particularly useful for unsupervised decision-making, especially in the presence of unpredictable disturbances or flow conditions.