Artificial neural networks for chemistry representation. Part 1: Generation of optimal ANNs using a pattern search algorithm

Surrogate-based derivative-free optimization is applied to design an artificial neural network (ANN). Optimization is performed using a mixed variable extension to the generalized pattern search method. This method offers the advantage that categorical variables, such as the type of the neuron transfer function or the network connectivity, can be used as parameters in optimization. When used together with a surrogate, the resulting algorithm is highly efficient for expensive objective functions. Results from a chemistry example demonstrate the effectiveness of this method in optimizing an ANN for the number of neurons, the type of transfer function and the connectivity between layers.