December 21, 2021

This figure shows how multiple drones would work together within a network to survey an area, track objects and transmit the best possible information back to a ground station. Graphic by Alicia Esquivel Morel.

Several years ago, a group of Mizzou Engineering graduate students had an idea. What if you could use machine learning to train drones to work together within a network to survey an area, track objects and transmit the best possible information back to a ground station?

After several iterations, the team’s latest publication on the topic has been accepted to INFOCOM 2022, a highly selective conference under the umbrella of the Institute of Electrical and Electronics Engineers. Fewer than 20% of submissions were accepted to the conference, which will be held in May.

The key was bringing in collaborators with the right expertise, said Rounak Singh, a master’s student in electrical engineering. Chengyi Qu, who’s completing a PhD in computer science, assisted with theoretical and mathematical logistics. PhD student Alicia Esquivel Morel provided expertise around drone operations and security and helped illustrate the concept graphically. Prasad Calyam, an associate professor of electrical engineering and computer science and Greg L. Gilliom Professor of Cyber Security, advised the group.

“The accomplishments from this work advance knowledge on how drones that are equipped with high-resolution sensors and cameras can benefit many applications such as e.g., disaster response management, smart farming and even parcel delivery” Calyam said.

Singh

The team’s success also shows the power of persistence and good teamwork, Singh said.

“I believed it would work, and even when we got peer-reviewer rejections for early ideas, I did not change my idea about it,” he said. “I tried to find the inconsistencies and fill in those gaps, but I was sure machine learning could be used for this particular application. I’m really happy for our team.”

First of its kind

Multiple drones are often deployed over an area to survey situations such as traffic crashes, wildfires or other disasters. Equipped with cameras, these unmanned drones transmit video footage back to a ground station in real time so that responders can make decisions based on what they see.

But there are challenges to this. Drones have limited battery life and sometimes veer off track. They also don’t always know what information is most important to those on the ground monitoring the footage.

Chingyi

With the proposed system, multiple drones would automatically know what to look for while surveying an area. And they would be able to avoid obstacles, such as birds, trees and each other.

The drones would also be able to make decisions such as when to use more power to zoom in and capture high-resolution images and when lower-resolution footage would be acceptable.

“We have to keep in mind all of these constraints and design an algorithm in such a way so the drone can make optimal decisions,” Singh said. “That’s one part. The other part is network protocols such as bandwidths and keeping the drone in a network with suitable signal strength characteristics to make sure it does not get disconnected.”

To train drones to make these real-time decisions, the team used deep reinforcement learning. Essentially, that means they developed algorithms to reward a drone when it makes correct decisions and penalize it for failing a task. That reward/penalty system assigns scores to each decision, helping a drone learn what it’s supposed to do when deployed, Qu said.

Esquivel Morel

They tested their algorithms using pre-existing datasets and generated new data sets to validate their results in a variety of scenarios.

That’s what makes the paper so novel, Qu said.

“We’re the first paper to use machine learning to guide multiple drones inside a network,” he said. “We compared numerous machine learning techniques to determine which would be best, and that’s the novelty of our paper. There is no previous work on comparing different learning results and how those methods could help for drone video analytics.”

Having the paper published at INFOCOM is just the beginning, Esquivel Morel said.

“It’s a good starting point for future publications,” she said. “For me, it is very exciting. We have a good foundation of a bold idea related to managing drones to fulfill application missions, and now we can continue to build on it.”