Automated Quantum Benchmarking with metriq-gym

The metriq-gym project is an automated pipeline for running quantum computing benchmarks on the Metriq platform. Metriq enables users to track, share, and compare quantum technology performance, showcasing individual contributions and state-of-the-art results. Traditionally, users manually extract benchmark data from sources like papers and repositories. Metriq-gym automates this process by running benchmarks on simulators and hardware according to a set cadence, automatically uploading results to the platform. This provides users with real-time insights into the progress of quantum hardware performance on critical computational tasks, offering a scalable solution for benchmarking efforts. This work accelerates the dissemination of quantum computing progress and helps the community better understand hardware and algorithm performance improvements over time.