Reinforcement learning for turbulent drag reduction of realistic road vehicles with dynamic flaps

The complex wake dynamics behind road vehicles is a dominant contributor to aerodynamic drag. Using Reinforcement Learning (RL) and digital environments (Direct Numerical Simulations), we demonstrate that dynamic rear flaps can fully stabilize the vortex shedding instability in laminar regimes. Next, we extend our study to heavy road vehicle models using RL in a wind tunnel environment. A real-time, time-critical control loop was established to enable online interactions between RL controllers and the flow environment. In this setup, the RL-trained controller receives instant pressure feedback from the truck's surface and outputs control signals as motor angles. The rear pitching flap, motorized by servo motors, acts as the control surface. The primary challenges in this experiment are controlling the highly turbulent wake of the vehicle and addressing partial observability due to the limited surface-mounted pressure sensors and signal delays. To overcome these challenges, we incorporate memory-based neural networks and efficient RL algorithms, demonstrating a significant reduction in instability and showcasing the effectiveness of our approach.