Efficient and Robust Training Strategies for Physics and Equality Constrained Artificial Neural Networks

Deep neural networks trained on governing physical laws have shown significant promise in solving forward and inverse problems. However, several issues remain challenging for developing models that are trustworthy and produce physically feasible predictions. A common approach in formulating a physics-informed objective function is to aggregate a weighted sum of the residual form of a governing partial differential equation (PDE) and its boundary conditions. The weights that balance the interplay between each objective term are problem specific and not known a-priori. In previous work, we have demonstrated that the formulation of the objective function as a constrained optimization problem is critically significant and proposed physics and equality constrained artificial neural networks (PECANNs) to successfully learn the solution of PDEs for a variety of problems. In PECANNs, we employ the Augmented Lagrangian method (ALM) to enforce equality constraints on the PDE loss. Previously, we gradually updated the penalty parameter until a maximum safeguarding value was reached for all constraints. However, finding an optimal strategy to update the penalty parameter as well as setting a proper safeguarding penalty parameter remained a challenge. In this work, we propose a novel strategy to adaptively learn a penalty parameter for every constraint without setting a safeguarding penalty parameter. Additionally, we refine the formulation of the constrained optimization problem to enable mini-batch training and reduce its computer memory footprint for large-scale PDE problems. We apply our method to several challenging benchmark problems and demonstrate marked improvement over existing methods.