Identification, Tracking, and Reconstruction of 3D Trajectories of Flocking Birds via Multi-Camera Field Observations

The present work introduces a stereoscopic multi-camera, multi-object tracking framework that integrates neural network-based segmentation, modified DeepSORT tracking, and a Tracking Reconstruction algorithm to obtain three-dimensional trajectories of flocking birds in the field. Tracking individual birds in dense, fast-moving flocks is a major challenge due to occlusions, overlap, and rapid maneuvers; but overcoming these challenges is key to understanding natural flight control strategies in complex environments. For our framework, extrinsic calibration uses a drone carrying a wand of known length to sweep the stereo cameras' overlapping field of view, ensuring precise alignment and low reconstruction uncertainty. Birds are detected using classical background subtraction with a dynamic background mask to isolate moving objects. In dense roosting events, more than ten birds can merge into a single foreground detection, requiring segmentation to separate individuals. We evaluate three segmentation strategies: the STARFLAG and marker-controlled watershed methods, both widely used in prior studies, and a fine-tuned object detection neural network. We find that the neural network provides the most reliable identification of individuals in dense groups. For temporal association, we extend the DeepSORT framework by comparing motion models, including a constant velocity Kalman filter and a constant acceleration variant, while prioritizing motion-based gating over appearance cues and refining intersection over union (IoU) matching. The constant acceleration Kalman filter with tuned process noise weights for position, velocity, and acceleration states reduces identity switches, improving tracking robustness in 2D. A Tracking Reconstruction algorithm is being implemented to merge single-view tracks from multiple cameras into continuous trajectories while minimizing ghost tracks. These methods quantify bird kinematics, including velocities, accelerations, and responses to turbulence at navigation, gliding, and flapping scales. The resulting data will allow us to examine how birds maintain stability and potentially exploit turbulence, both individually and in flocks, with implications for improving unmanned aerial vehicle performance.