Connecting value functions to flow map operators for path planning in flow fields

Path planning through unsteady fluid flow fields is a challenging problem with important applications in robotics, navigation, and sensor deployment. It is possible to formulate this task as an optimization problem which involves solving for value functions over unsteady flow fields. Value functions play a fundamental role at the heart of all of optimal control, including recent formulations in reinforcement learning and model predictive control. However, the highly nonlinear and multi-scale nature of fluid flow fields often make it challenging to solve for, and interpret value functions. In this work, we establish a strong connection between the fluid coherent structures and material separatrices, which can be quantified through the finite time Lyapunov exponent (FTLE) field, and value functions computed over the domain. In particular, we show that FTLE ridges tend to demarcate sharp transitions in the value function, illuminating regions where active transport may be achieved at a much lower cost. Further, we explore mathematical connections between the particle flow map operator underlying the FTLE field and solutions of the Hamilton Jacobi Bellman equation, which is central to optimal control and reinforcement learning. We demonstrate this approach on simple unsteady flow fields to develop intuition. These findings will lay the theoretical foundation for principled performance analyses of motion planning policies and deployment strategies for intelligent mobile sensors in fluid flows.