Double descent, linear regression, and fundamental questions in materials model building

Though many data science concepts are just glosses on ideas that predate the data science revolution by years or even decades, some suggest altogether new approaches or raise fundamental questions. The phenomenon of double descent behavior in neural networks defies intuition and may seem to violate the "no free lunch" theorem. Is double descent behavior peculiar to neural networks? Or is it more general? We illustrate double descent in a simple linear regression model and then revisit basic questions in alloy model building, using the cluster expansion and machine learned interatomic potentials as illustrations. How is convergence impacted by the range of interaction? Or the order of an n-body interaction? How completely must we span configuration space with our expansions? We address these questions from the perspective of both mathematics and physics and discuss the implications for practical alloy models.