Evaluating Generalization in Classical and Quantum Generative Machine Learning Models: Part I

In many applications, the real advantage behind machine learning is unlocking the model's ability to generalize. In supervised ML, this power is given by the model's capacity to correctly classify unseen data. In contrast, generalization within generative unsupervised ML determines the model's capability to generate valuable data beyond the training set. In the latter, this property remains highly unexplored, both in the classical and quantum ML communities, given the challenge to propose robust metrics for evaluating the quality of newly generated samples. In this work, we present a novel approach for comparing and contrasting the generalization capabilities of generative models.

In part I of this two-part presentation, we introduce new benchmarks and metrics that allow us to conduct a 3D evaluation of a model's ability to generalize efficiently. Additionally, by using data sets where we can measure the quality of the generated samples, we provide a metric for quantitatively assessing the practical value of generalization.

In part II, we use the problem agnostic metrics to study a specific data set from a financial application. Here, we evaluate and compare the generalization power of three types of generative models (classical, quantum-inspired, and hybrid quantum-classical).