Automated neuron tracking using deep learning and targeted augmentation allows fast collection of C. elegans whole brain calcium activity during behavior

With advances in optical imaging and fluorescent proteins, it is now possible to record calcium activities from the whole brain of the roundworm C. elegans during freely moving behavior. However, tracking the position and shape of each neuron is a major analysis bottleneck limiting the throughput. The data presents numerous challenges: the animal moves, rotates and deforms rapidly; the limited frame rate causes motion blur; manually annotating 3D images is very difficult. While convolutional neural networks(CNNs) are highly effective for image analysis, they generally require a large training set.

To tackle this problem, we present a method to harness the robustness, adaptability and accuracy of CNNs while requiring only 5~10 frames of annotations. We designed a new network, 3D Compact Network (3DCN) with atrous convolutions and resolution-aware pooling layers. Our network is faster, more memory efficient, more accurate, and generalizes better than a conventional U-Net. After an initial training phase with rigid augmentations, the neural network annotates frames close to the training examples in posture space. From this coarse prediction, it then deforms the closest grounth truth frame to imitate this posture. The CNN is then retrained using these targeted augmented frames.

Using this method, we have successfully tracked 80+ neurons in the C. elegans male tail with large deformations and segmented very dim neurites in the hermaphrodite head from a completely different imaging setup. We have also tracked 50+ neurons of interest in the hermaphrodite head with <13 training frames per dataset.

We briefly discuss preliminary findings drawn from neural activity and behavior under a thermal stimulus.