February 28, 2023
An interdisciplinary Mizzou team received a “Best Paper Award” after developing system that can automatically identify individual birds in images of waterfowl flocks.
Yang Zhang, a PhD student in computer science, presented the work at the Institute of Electrical and Electronics Engineers (IEEE) International Conference on Cognitive Machine Intelligence in December. Co-authors included Professor Yi Shang and collaborators from the School of Natural Resources and the Missouri Department of Conservation (MDC).
“I was shocked, but it was exciting,” Zhang said. “I’m grateful for Dr. Shang’s instruction and for the other authors. Teamwork was very important.”
Shang has led the project for the past four years in hopes of helping conservation officials more efficiently track waterfowl populations. Historically, the department has monitored waterfowl by manually counting them in the field. In 2019, MDC awarded Shang funding to develop techniques to autonomously identify and classify birds.
Shang and his students use images captured by drones to train machine learning models to identify waterfowl. While artificial intelligence can recognize a single bird in a photo with ease, it struggles when an aerial photo contains a flock of hundreds of birds along with tree branches, plants and other objects in the background. For this project, MDC also deployed decoys in some cases to evaluate the performance of the AI detectors.
“We use drones to collect images of waterfowl throughout different parts of the state, particularly in some of the parks and conservation areas,” Shang said. “Students have taken a lot of trips, going into the field and flying drones. We developed our own software to plan flight trajectories ahead of time so we can collect images at different altitudes and under different conditions. We’ve collected a pretty extensive data set.”
Students then labeled the images manually and fed the data into the machine learning pipeline to train detection and classification models. Now, the system can identify birds with about a 90% accuracy rate. The next step will be training the machine to identify different species of birds in images.
Shang’s team has also developed similar deep learning systems for other applications, including identifying invasive plants and wetland habitats. Last year, his students took first place in an Ameren-sponsored Plant Recognition University Challenge to identify plants and trees in rights of way in aerial images. Zhang presented findings from that competition at the IEEE conference in December, as well. In addition, his students competed in the iWildCam 2022 competition, which is part of the FGVC9 workshop at CVPR 2022 and is sponsored by Wildlife Insights. Again, Zhang’s team finished first.
“There is a nationwide community in the conservation space doing this type of work,” Shang said. “People from across the country are talking about how to use remote sensing and imagery from drones, satellites and planes to efficiently identify objects. We’re creating and building models that can be applied to a variety of real-world situations.”
Other authors who contributed to the paper were: Shiqi Wang, a PhD student in computer science; Zhenduo Zhai, a PhD student in electrical and computer engineering; Reid Viegut, a master’s student in natural resources; and Elisabeth Webb, associate cooperative professor in the School of Natural Resources. Andrew Raedeke and Joel Sartwell from the Missouri Department of Conservation were also recognized.
Interested in applying AI to real-world scenarios? Learn more about electrical engineering and computer science at Mizzou.