Ramp-down Trajectory Optimization with Robustness to Physics Uncertainty with Reinforcement Learning Enabled by Scientific Machine Learning

The safe termination of plasmas is critical for upcoming burning plasma devices, both for routine device operations, but also for disruption avoidance and mitigation, where it is desirable to rapidly decrease the plasma current and stored energy to ameliorate the consequences of a disruption. In this work, we report on experiments at TCV where we successfully exit a high beta, high Greenwald fraction scenario before terminating the plasma at a low current. The exit trajectory is designed by a reinforcement learning (RL) algorithm which determines the optimal trajectories for: 1) plasma current, 2) elongation, 3) minor radius, and 4) neutral beam injection while avoiding user prescribed limits on: greenwald fraction, poloidal beta, safety factor, and internal inductance. The RL training environment is a hybrid physics and machine learning model that is trained using techniques developed for training neural differential equations [1]. The introduction of physics structure into a machine learning model enables generalization with a relatively modest dataset size of <300 shots, making this approach potentially relevant for upcoming devices which will generate several hundred shots of data through the commissioning phase. A key innovation is the demonstration of the ability to train a single trajectory to achieve the desired outcome despite uncertainty in the plasma dynamics by parallelizing the training environment on GPU.

[1] Chen, Ricky TQ, et al. "Neural ordinary differential equations." Advances in neural information processing systems 31 (2018).