Exploiting Sparsity in Artificial Neural Networks for Spectroscopic Data

Models based on Artificial Neural Networks (ANNs) are widely used in spectroscopy for various tasks (classification, regression, dimension reduction), often with state-of-the-art performances. Commonly utilized ANNs have hundreds of thousands to millions of weights, resulting in a huge over-parametrization. While the over-parametrization seems to be beneficial for the performance (e.g., double descent behavior) and generalizability, it significantly worsens the interpretability of the model. Such a black-box behavior of the models limits the applicability in high-stake applications and slows scientific progress. In this work, we exploit lottery tickets (i.e., iteratively pruned, sparse networks with the same or slightly better performance than their dense counterparts) for the interpretability of ANNs that were trained for classification and regression on spectroscopic data. We show that lottery tickets in a contrastive regime (where we compare two sparse models) can detect task-important features in the data and allow for better model interpretability. The concept is demonstrated on Laser-Induced Breakdown Spectroscopy data but can be extended to other techniques, considering the availability of a sufficient amount of data and similar properties. The results are critically evaluated and compared to a baseline approach, the feature visualization by an input optimization technique.