Chiral spin texture in the anomalous Hall antiferromagnet CoNb3S6

CoNb₃S₆ (CNS) is an intercalated transition metal dichalcogenide exhibiting a large anomalous Hall effect (AHE) that cannot be explained by the collinear magnetic order previously observed by neutron diffraction. Thus, complex chiral, non-collinear, or non-coplanar spin orders have all been proposed as explanations for the observed large AHE in CNS. Here we carry out resonant elastic x-ray scattering (REXS) at the Co L₃ edge to obtain a more detailed description of the magnetic ordering in CNS. We determine the average in-plane spin orientation of the previously known q = (0.5 0 0) collinear order using full linear polarization analysis (FLPA). By comparing the results at two symmetry-related wavevectors, we find that the data is consistent with single-q, multi-domain order, rather than multi-q order. We also find satellite peaks around the collinear wavevector, indicating a long-wavelength incommensurate uniaxial modulation of the collinear order. The modulation wavevector exhibits a sample dependence which is correlated with the magnetic transition temperature, indicating that the modulation depends on Co stoichiometry. The modulation in the under-intercalated samples is consistent with a spin density wave, while that in the nearly stoichiometric sample is consistent with a helical structure, as indicated by the circular dichroism of the satellite peaks. These results suggest that a long-range chiral spin texture provides the necessary symmetry-breaking mechanism for the large anomalous Hall effect in CNS.