Benchmarking progress toward utility-scale quantum computing

The rapid pace of development in quantum computing technology has sparked a proliferation of benchmarks for assessing the performance of quantum computing hardware and software. Good benchmarks empower scientists, engineers, programmers, and users to understand a computing system's power, but bad benchmarks can misdirect research and inhibit progress. Following the Perspective laid out in arXiv:2407.08828, I survey the science of quantum computer benchmarking from the perspective of driving and measuring progress towards the long-term goal of useful quantum computations, i.e., "quantum utility". Different kinds of benchmark quantify the performance of different parts of a quantum computer, and existing quantum computer benchmarks can be organized using this principle. I propose a framework for measuring progress toward quantum utility, and highlight the most critical open research problems on the road to doing so.