In the field: Cross disciplinary research uses robots for data collection
A computer scientist, a computational biologist and an electrical and computer engineer come into a room. The computer scientist and computational biologist share a research interest in maize phenotype mutants — visual characteristics that occur from a plant’s genetic interactions with the environment. The computational biologist spends countless hours, literally out in the field, collecting and recording a broad array of phenotypic data — crucial, exacting and exhausting work — followed by hours in the lab to mathematically model the collected photographic representations of lesions on maize.
The electrical and computer engineer said he had some ideas about automating data collection with a robot and asked if they’d consider sprouting a collaboration.
Not exactly a “three engineers enter a room” joke, the previous tongue-in-cheek scenario serves to illustrate that work of problem-solving engineers crosses disciplines in overlapping areas of expertise to address broader challenges.
The computational biologist — MU Engineering computer science Assistant Professor Toni Kazic — conducts multi-faceted maize research focused on both the phenotypic expression of maize lesions, or spots, and the biochemical networks behind them. As a computer scientist, she works to build algorithms that can infer the networks — form hypotheses about the network’s structure and function — based on collected phenotypic data.
In the past, Kazic joined forces with her colleague, Shumaker Endowed Professor Chi-Ren Shyu, on a research project to make databases of maize mutant images from a variety of sources semantically searchable. Shyu currently serves as director of the MU Informatics Institute, one of the several research umbrellas under which Kazic conducts her maize investigations, including MU’s Missouri Maize Center.
“I only learned of the phenotype measurements on plants a couple of years ago,” said Gui DeSouza, an associate professor of electrical and computer engineering. “I immediately thought, ‘We can replace the human measurement and create a robotic platform to do it.’”
Kazic’s experience shows that inferring networks from molecular data is error-prone, so she is instead focused on using phenotypic information as the crucial data. Each year, she and her research group hit the field to manually plant and pollinate maize with the phenotypic profiles she has selected to manipulate. They spend the summer recording information in the field and then go back to the lab to develop algorithms to process the images and extract the information. That information, together with methods they are developing, is key in their work to infer networks.
“We need to understand how the plant works so we can eventually manipulate it for better yield and disease resistance,” said Kazic of the importance of network inference. “A network of thousands of biochemical reactions working in concert are responsible for a plant’s response to the environment, and that network of reactions determines the plant’s characteristics and responses to stresses.”
Kazic was happy to provide DeSouza and his graduate students, Nick Smith and Siavash Farzan, with a sample patch of corn and soybeans, and Shyu provided seed money to help outfit and program the robot in a first attempt to streamline the complex data collection and processing aspects of the field research.
The three College of Engineering faculty members partnered with other researchers in MU’s Interdisciplinary Plant Group (IPG), including Associate Professor Felix Fritschi and his crop physiology lab, and Missouri Soybean Merchandising Council Endowed Professor Henry Nguyen, who directs the MU-based National Center for Soybean Biotechnology, to write a National Science Foundation Major Research Instrumentation (MRI) program grant aimed at developing a robotic platform to collect plant data in the field.
Nguyen is actively involved in the discovery of genetic diversity and gene functions in soybeans, with a developing focus on root phenotypes. Fritschi’s research interests are in the area of plant responses to abiotic stress — such as water deficit and high temperatures — on plant growth and productivity. Both see the potential of robotic data collection.
“The goal would ultimately be to have three robotic systems for data collection through and around the plots,” DeSouza said. “Aerial vehicles [quadcopters] would be used to identify areas where the robot should collect data.”
Coincidentally, the Leonard Wood Institute contacted DeSouza with the offer of a TALON robot that was used in a research project that had reached completion. Though it is a fairly large-sized unit, the robot was suited to use a prototype for the data collection work.
Kazic assigns labeled barcodes to each crop row and each plant to uniquely identify it with a hand-held BlueTooth scanner for data collection on an iPad. Every plant and every phenotype is different, but she and her team can’t scrutinize each of them every day, which can take up to a half-hour per row, but a robot potentially would have that ability.
“In the last 10 years, there’s been more interest in collecting data in greenhouses, but that’s just not good enough. Farmers work in fields. We’ve known from the beginning that we wanted to work in the field,” said Kazic.
“The robot has excited a lot of interest in the maize community,” she added.
DeSouza and his research assistants equipped the robot for its new plant science functions over the summer, putting it through its paces at the research plot. The project serves as proof of concept for the MRI proposal.
“An RFID [radio frequency identification device] reader with sensors records the specific plant, humidity, temperature and light intensity and a photo is taken,” said Farzan. “Two small computers trigger the sensors and everything is recorded on an Excel spreadsheet.”
The robot uses the RFID tags to identify the plant, but the human researchers use the paper barcodes, which also have human-readable identification data. Everything about the plant is keyed to its barcoded ID. Kazic hopes a way can be devised to incorporate the barcodes into the automated process, possibly by printing them on both sides of the tags.
In addition to recording humidity, light and temperature, the robot can photograph whole plants. Kazic hopes the robot will eventually be able to manipulate the plants to photograph individual leaves at high resolution as well as measuring plant height. Others in the IDP would like to include chlorophyll and gas measurements.
The entire plant group is very interested in a robot that can plant and weed a test plot.
DeSouza’s system was designed to be scalable. It can be expanded to accommodate additional computers as necessary and more cameras for photos from a variety of angles.
“The cool thing is, everything is synched,” said Smith. “The robot finds the RFID tag, stops, communicates to take the snapshot and readings and then makes two beeps [to indicate it has completed the sequence] in less than a second. And everything operates on one small battery.”
The robot currently is being operated by remote control, so the next big challenge will be to fully automate the process.
“It sounds easy, but it is computationally not trivial,” DeSouza said. “For now, we just want to show that we can collect data — prove the concept.”
The “footprint” of the prototype TALON is a little too large, especially with the camera arm, to fit down the normal width of a row in corn and soybean fields, so the team has primarily been testing it on outside rows of the field. But, said DeSouza, the system can be mounted to any robot.
DeSouza is looking into the future beyond phenotype profiling to additional uses for such a system.
“Even simple market gardening scale tasks could be a great market for this technology,” he said.