Transformer Quantum State: A Multi-Purpose Model for Quantum Many-Body Problems

Recent advancements in machine learning have led to the introduction of the transformer, a versatile, task-agnostic architecture with minimal requirements for hand-crafting features across different tasks. Here, we show that with appropriate modifications, such an architecture is well suited as a multi-purpose model for the solution of quantum many-body problems. We call the resulting model the transformer quantum state (TQS). In sharp contrast to previous Hamiltonian/task-specific models, TQS is capable of generating the entire phase diagram, predicting field strengths with as few as one experimental measurement, and transferring such knowledge to new systems it has never seen before, all within a single model. When focusing on a specific task, fine-tuning on a pre-trained TQS produces high-accuracy results with small computational cost. Versatile by design, the TQS architecture can be easily adapted to new tasks, thereby pointing towards a general-purpose model for various challenging quantum problems.