Elimination of velocity deficit behind a cylinder using reinforcement learning

In this study, we present an active flow control strategy through deep reinforcement learning (DRL) for eliminating the velocity deficit in the wake of a circular cylinder. A group of windward-suction-leeward-blowing actuators are adopted. Their individual velocities are automatically adjusted by the DRL agent through feedback signals from a downstream sensor array. Simulations are conducted using a GPU-accelerated Lattice Boltzmann solver with multi-block mesh partition. The high-dimensionality and non-linearity features of this problem make it challenging to explicitly determining the control strategy. By adopting the DRL, the agent can learn from the time sequences of the sensors, actuators and a specified reward function through trials and errors, and finally converge and determine the optimal control strategy. Results show that the well-trained control strategy can eliminate 99.7{\%} of the velocity deficit. The policy is further tranferred for fluid-structure interaction situations, and results based on four representative cases show that the transferred control strategy is robust and can effectively eliminate the velocity deficit by around 96{\%}. Overall, the current study offers an innovative view that could potentially help underwater vehicles achieve low detectability.