Drag Reduction in Flows Past 2D and 3D Circular Cylinders Through Reinforcement Learning

Drag reduction in bluff body flows is crucial for efficient aero- and hydrodynamic design, impacting power consumption and emissions. While passive control methods involve surface modifications, active methods employ surface actuators like tangential actuators, plasma, or mass transpiration. In this study, we identify drag reduction mechanisms in flows past 2D and 3D cylinders controlled by surface actuators using deep reinforcement learning. We investigate flows at Reynolds numbers of 1000, 2000, 4000 (3D), and 8000 (2D). The learning agents are trained in planar flows at various Reynolds numbers, considering actuation energy.

The discovered actuation policies demonstrate remarkable generalization capabilities, enabling open-loop control for Reynolds numbers outside the training range in 2D flows. Surprisingly, the 2D controls and induced drag reduction mechanisms, such as delayed separation and vorticity generation, transfer to three-dimensional cylinder flows. The study unveils intriguing trade-offs between drag reduction and energy input, with aggressive actuation leading to significant decreases in flow separation angle and up to 35\% drag reduction. Conversely, a more conservative approach strategically turns actuators on and off, achieving drag reduction with limited resources.

To alleviate the computational cost associated with stochastic search, we employ a parallel implementation of V-RACER with Remember and Forget for Experience Replay (ReF-ER) in Korali (an open-source optimization and Reinforcement Learning framework), enabling parallel training with direct numerical simulations at a manageable cost. Furthermore, it underscores the cost-effectiveness of surrogate models, such as 2D simulations, for training models applicable to real-world 3D flows at moderately high Reynolds numbers. These findings hold significant implications for drag reduction in practical fluid dynamic applications.