November 18, 2021

Images showing preliminary results of a Mizzou Engineering team’s methods to automatically identify trees. The green rectangles are predictions of trees generated by a deep neural network predictor, while the red dots are the ground truths of trees provided to the group by Ameren.

Mizzou Engineering students are developing innovative methods and software to automatically detect and identify tree species. It’s part of a Plant Recognition University Challenge sponsored by Ameren.

Ultimately, the utility company wants a system that would identify plant species and population densities in rights-of-way using images collected from unmanned aerial vehicles. That information would help inform decisions about environmental stewardship and conservation management on these right-of-way habitats.

Ameren is challenging university groups from across Missouri to help with that goal by developing new technologies to advance automated plant identification.

The competition was a good fit for Professor Yi Shang’s research group in electrical engineering and computer science.

“This is a real-world application to show students what people are looking for and what the challenges are in terms of defining the problem, applying advanced AI technologies, considering the tradeoffs, and working through a variety of challenges that come up,” said Shang, who is also director of graduate studies and Robert H. Buescher Faculty Fellow. “Working on a benchmark data set that is well formulated and cleaned up is easier than working with a real data set, which requires many decisions over the process to come up with effective solutions for the customer.”

For the Ameren challenge, teams were given a data set of images from the St. Louis Botanical Garden. The set includes traditional RGB images, as well as multispectral images and LiDAR data that provides distance measurements.

Teams will use this information to train deep learning models to identify plant locations and their species with the hopes that the system will be able to automatically detect them in the future.

Shang’s team may have a competitive edge in that they’ve developed a similar system to identify and count waterfowl.

However, trees are tougher, Shang said. Unlike birds, which are similar in shape and size, trees vary depending on species, maturity and time of year.

“Trees are not like other objects that can be clearly identified,” said Zhicheng Robert Tang, a PhD student in computer science. “They look different in different seasons. And from aerial images, sometimes different types of trees overlap, or large canopies may hide smaller trees.”

In addition to the data set provided, Shang’s team will use other publicly available data sets and their own images to help train the models. They also have information from a separate research project they’re conducting, which aims to identify invasive plant species.

And the engineering group is getting help from Reid Viegut, a master’s student in natural resources. Viegut is collecting images of trees and plants from across Missouri and labeling the species.

The group will have the rest of this semester and most of next semester to develop the system. They’ll then test its accuracy against other student-developed models at the Challenge Day on April 22. The winning school will receive a $70,000 prize.

“We feel pretty confident because we do have a lot of experience processing aerial imagery and developing deep learning models,” Tang said. “If we have a reliable system, it would be very beneficial not only to Ameren but also for other applications in conservation or natural resource management. What we want to do is automate the whole process and replace human efforts in these scenarios. The ultimate goal is to provide a product for a variety of uses in the future.”

Yang Zhang, a PhD student in computer science, and Zhenduo Zhai, a PhD student in electrical and computer engineering, are also on the team.