Transformer-based reinforcement learning for aerodynamic lift regulation in gust sequences

We propose a transformer-based reinforcement learning (RL) framework to learn an effective control strategy for regulating aerodynamic lift of a flat-plate airfoil in gust sequences via pitch control. The transformer addresses the challenge of partial observability from limited surface pressure sensors. We show that the training can be accelerated if we have a pre-obtained control policy as a warm start, which can be (i) learned from proportional control (P control) strategy designed in the same flow configuration by pretraining and (ii) learned from training the RL agent in a simplified flow configuration by transfer learning. We demonstrate that the RL control strategy outperforms the best P control, with the performance gap widening as the number of gusts increases, and the strategy learned in an environment with a small number of successive gusts is effectively generalized to an environment with an arbitrarily long sequence of gusts. We investigate the impact of pivot configuration, revealing that quarter-chord pitching provides superior lift regulation with lower control effort compared to mid-chord pitching. This advantage is attributed to the dominant added-mass contribution accessible via quarter-chord pitching through a lift decomposition. These results demonstrate the generalizability of the transformer-based RL framework, offering a promising approach to tackle more computationally demanding flow control challenges, when combined with the proposed acceleration techniques.