Markerless tracking of an entire insect colony

From cells in tissue to human crowds, living systems display a stunning variety of group behaviors. Yet quantifying such phenomena first requires tracking a significant fraction of the group members in natural conditions, a substantial and ongoing challenge. We present a method for the markerless tracking of nearly all individuals in a colony of honey bees Apis mellifera. We leverage advances in machine vision to solve two interrelated problems; (1) detection of highly similar objects in dense configurations and (2) matching of these detections into trajectories based on visual features which are largely invisible to the human eye. We apply the detection method to demonstrate months-long monitoring of sociometric colony fluctuations. These fluctuations include ~24 h cycles in the counted detections, negative correlation between bee and brood, and nightly enhancement of cell-bees. Our tracking method recovers ~79% of bee trajectories from five observation hives over 5 min timespans. The trajectories reveal important individual behaviors, including waggle dances and crawling inside comb cells. Our results provide new opportunities for the quantitative study of collective bee behavior and for advancing tracking techniques of crowded systems.