Using symmetries and transformers to build better latent spaces for di-jet representation learning

We investigate a method of model-agnostic anomaly detection at the Large Hadron Collider (LHC) through studying jets, collimated sprays of particles produced in high-energy collisions. We train a transformer neural network to encode simulated quark and gluon ("physical-space") di-jets into a high-dimensional ("latent-space") representation. We optimize the representations using the contrastive loss, which tells the transformer to preserve known physical symmetries of the di-jet when making its representations. We then run a signal (consisting of di-jets from simulated new particles) vs. background (di-jets from quarks and gluons) linear classifier test on the latent-space representations and compare the classifier performance with that of a dense binary classifier neural net trained on the physical-space di-jets. We finally explore the possibility of using the transformer net to encode LHC events at the event level -- rather than at the jet level -- into the latent space. Such a representation makes no assumptions on the relationships between particles in a given event while still enforcing known physical symmetries for particle collision events on the whole. This could provide a maximally agnostic event representation that could be used to search for new physics of any type.