URBANA, Ill. – Agricultural producers and manufacturers often need information about crop attributes, from nutrient content to chemical composition, to make management decisions. In recent years, multispectral imaging has emerged as a useful tool for product analysis, but the required equipment is expensive. Standard RGB cameras are much more affordable, but their images show only visible attributes.
However, if RGB images can be "translated" to multispectral images, pictures taken with a smartphone or any regular camera can yield sophisticated information. This process requires complex computer modeling and machine learning, but once the techniques are developed, they can be applied to simple devices anyone can use.
In two new papers, researchers at the University of Illinois Urbana-Champaign explore the reconstruction of multispectral and hyperspectral images from RGB for chemical analysis of sweet potatoes and maize.
"An RGB camera captures only the visible range in three bands, red, green, and blue. The pictures cannot provide any chemical information, which you often need for crop analysis. We reconstructed images from these three bands to include information from the near-infrared range, which you can use to determine chemical composition," said Mohammed Kamruzzaman , assistant professor in the Department of Agricultural and Biological Engineering (ABE), part of the College of Agricultural, Consumer and Environmental Sciences and The Grainger College of Engineering at the U. of I. He is corresponding author on both studies.
"This work has many potential applications in the agricultural industry and can significantly lower costs. While a multispectral camera costs $10,000 or more, you can get an RGB camera for a few hundred dollars," he added.
Analysis of sweet potato attributes
In the first paper, the researchers provide a large dataset of reconstructed images for chemical analysis of sweet potatoes that anyone can access and use for their own modeling.
"Most existing image reconstruction models focus on non-biological objects like tables and chairs, which are very different from biological objects. Our goal was to create an RGB-to-hyperspectral image dataset for a biological sample and make it publicly available," said lead author Ocean Monjur , doctoral student in ABE.
Sweet potatoes are a popular food source, and they are also used for a wide range of industrial purposes including textiles, biodegradable polymers, and biofuels. Assessing quality attributes such as brix (sugar content), moisture, and dry matter is important for determining the usage and value of potatoes. Chemical laboratory analysis is time-consuming and destroys the samples. Hyperspectral imaging (HSI) is fast, accurate, and non-destructive, but it is expensive and complicated.
That's why the researchers created Agro-HSR, a large database of reconstructed RGB to HSI images for the agricultural industry. The dataset includes 1322 image pairs from 790 sweet potato samples, collected from one or both sides of each potato. For 141 potato samples, they measured brix, firmness, and moisture content to evaluate the accuracy of the reconstructed images, finding them to be highly correlated with the actual measurements.
They tested their dataset on five popular hyperspectral imaging reconstruction models to determine which performed best, finding that two models (Restormer and MST++) consistently outperformed the others on all metrics.
"To our knowledge, this is the largest dataset for hyperspectral image reconstruction, not just for agriculture but overall. We are providing this database so anyone can use it to train or develop their own models, including models for other agricultural products," Kamruzzaman said.
Evaluating chlorophyll content for maize growth
In the second paper, the researchers describe a novel method for multispectral image reconstruction to analyze chlorophyll content in maize. They also introduce a simple device that people can use to take pictures in the field and get immediate results.
"Our target measure is chlorophyll content, which is an indicator of plant growth. With this device you can take a picture, get the chlorophyll content, and determine the crop's growth status," Kamruzzaman said.
To develop their model, the researchers collected images from three different locations: a research field in Hengshui, China; the U. of I. Plant Biology Greenhouse; and the U. of I. Vegetable Crops Research Farm.
At each location, they divided the area into varying levels of soil fertility, and at the Illinois research farm, they subjected the maize to three levels of stress by flooding throughout the growth period.
In all of these settings, they tested several modeling approaches to reconstructing multispectral images from RGB. Based on their findings, they created a novel model called Window-Adaptive Spatial-Spectral Attention Transformer (WASSAT), which more accurately aligned with the actual data.
"We combined spectral and spatial attention modes to establish an adaptive window that can discern crops from soil and other elements, capturing the complexity of a field environment. Then we reconstructed 10-band images to predict chlorophyll content, and we found our results performed better than other models," said lead author Di Song , doctoral student in ABE.
