Towards Digital Design at the Exascale: Advances in Bayesian Optimization with Neural Networks

The expense and consequence of many scientific and engineering applications, such as inertial confinement fusion (ICF), necessitate the use of digital design, whereby promising configurations are first explored via numerical simulation before being realized in experiments. But many such computer models are themselves significantly expensive, making manual or brute force search intractable. In these situations, surrogate-based optimization techniques, such as Gaussian-Process-based Bayesian Optimization, are commonly used. However, these methods could have their limitations on emerging exascale compute platforms, which could generate sufficiently large enough quantities of high-dimensional design data as to make Gaussian Processes inefficient. As an alternative approach, we are developing novel algorithms and scalable uncertainty metrics that enable Bayesian Optimization with neural networks. These networks can efficiently model response surfaces in high dimensions and with sparse data, making them attractive for digital design at the exascale. In this work, we explain our new models and show how they outperform standard optimization techniques on both analytic and ICF design examples.



This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344 and was supported by the LLNL-LDRD Program under Project No. 21-ER-028. LLNL-ABS-836594.