Distributed Actuation of Turbulent Flow Around a Cylinder using Deep Reinforcement Learning

The turbulent wake behind a cylinder in crossflow exhibits large-scale unsteadiness which is highly sensitive to perturbations in the freestream. The ability to effectively modulate the wake can benefit various performance metrics such as reduced drag, noise suppression, and mixing enhancement. Prior work on flow control around cylinders has focused on utilizing a variety of actuation methods such as steady suction and blowing, cylinder rotation, acoustic excitation, electromagnetic forcing, synthetic jets, and various other approaches. Although these actuation methods have succeeded in effectively reducing drag and lift forces by suppressing vortex shedding, traditional control methods usually rely on linearization approaches, which can limit their effectiveness in fully-developed turbulent flows. This study presents a data-driven approach to modulating large- and small-scale coherent structures by coupling Large Eddy Simulations (LES) in fully developed turbulent flow (10⁴ < Re < 10⁵) with an autonomous technique called Deep Reinforcement Learning (RL). An RL agent is trained to perturb the flow in real-time using a coordinated array of actuators distributed over the surface of the cylinder. The RL algorithm dynamically alters the actuation of 4 independent spanwise surface actuators to produce local sources of wall vorticity, thereby modulating the coupling between large- and small-scale coherent structures downstream.