Deep reinforcement learning of airfoil pitch control in freestream disturbances

Freestream disturbances or sudden transverse motions (plunging) can drastically change the aerodynamic loads on an airfoil and are an essential aspect of the development of agile or small-scale flight vehicles. When the magnitude of the relative transverse velocity is of the same order as the forward flight speed, the strong non-linear dependence of the circulatory lift on the flow state prevents us from applying classic (linear) control techniques to adequately track a reference lift trajectory during a rapid succession of such maneuvers or disturbances. For these problems, data-driven control, such as deep reinforcement learning, could be a better-suited alternative to classic control, as it has already been successfully applied to the active flow control of plasma actuators and synthetic jets on airfoils. This work investigates the applicability of deep reinforcement learning for the pitch control of an airfoil to minimize lift variations while tracking a reference lift during random transverse freestream disturbances. The environment is simulated using a two-dimensional high-fidelity Navier-Stokes solver, and different learning algorithms are compared to find a continuous-action policy that selects an angular acceleration input based on limited information about the state of the airfoil and the wake. The associated challenges of partial observability and sample efficiency are discussed in detail.