HydroGym: A Reinforcement Learning Control Framework for Fluid Dynamics

We propose HydroGym, a framework for reinforcement learning control of fluid flows. Over the past years, Reinforcement Learning has proven itself as a highly effective control paradigm in complex environments ranging from robotics to protein folding, building on a foundation of scalable reinforcement learning frameworks and standardized benchmark problems. Progress in the application of reinforcement learning to flow control has in contrast been challenged by the scarcity of such frameworks, and benchmarks. To this end, we present HydroGym, a new solver-independent reinforcement learning framework for flow control, which enables the seamless scaling of flow control reinforcement learning environments with state of the art online, offline, and differentiable reinforcement learning. We present these various online, offline, and differentiable reinforcement learning results on a set of four canonical fluid flow environments showing the ease-of-use, scalability, and ease of extensibility to new flow control environments.