Nematic Order on the Surface of a Three-dimensional Topological Insulator

We develop a field theoretic description of nematic order and investigate its consequences on the surface of a three-dimensional topological insulator with a single Dirac cone in both the nodal (the Fermi energy is at the Dirac point) and doped limit. In the nodal limit at zero temperature, we find a first order phase transition to a nematic phase at the mean-field level. Upon increasing the temperature, we find a tri-critical point and a continuous critical phase boundary. In the doped limit, we find a continuous phase transition to a nematic helical Fermi liquid. One of the unique features of this nematic helical Fermi liquid phase which, in principle, can be observed via spin-resolved angle-resolved photoemission spectroscopy, is the breakdown of spin-momentum locking, except for certain highly-symmetric momenta. Finally, we discuss the non-Fermi liquid behavior at and near the isotropic-to-nematic phase transition.