Hyperspectral imaging (HSI), or imaging spectroscopy, captures detailed information across the electromagnetic spectrum by acquiring a spectrum for each pixel in an image. This enables precise identification of materials through their spectral signatures. HSI supports Earth remote sensing applications such as automated classification, abundance mapping, and estimation of physical and biological properties like soil moisture, sediment density, water quality, biomass, leaf area, and pigment content.
Although HSI offers detailed insight into a remote sensing scene, HSI data may not be readily available for an intended application. Recent studies have attempted to combine HSI with traditional red-green-blue (RGB) video acquisition to lower costs and improve performance. However, this fusion technology still faces technical challenges.
In a recent study published in the Journal of Applied Remote Sensing , researchers from the Chester F. Carlson Center for Imaging Science at the Rochester Institute of Technology developed a bimodal video platform that combines a 371-band hyperspectral imaging system, operable in a low-rate video mode, with a standard RGB video camera. Led by Chris H. Lee, the team designed this system to bridge the gap between high-cost hyperspectral imaging and widely available RGB video technology.
The team demonstrated their proof-of-concept by capturing video data of the Lake Ontario shoreline at Hamlin Beach State Park in Rochester, New York. "We developed a workflow that links reflectance data from a line-scanning hyperspectral imaging spectrometer with RGB video frames to predict hyperspectral imagery," Lee says. "We established a correlation between the two data streams during a specific time segment, then used it to predict hyperspectral frames both before and after that segment using only RGB video."
They captured visible to near-infrared hyperspectral video using a Headwall Hyperspec micro-High Efficiency imaging spectrometer, operating in its low-rate video mode. RGB data came from a widely available, low-cost GoPro Hero 8 Black. Lee's group pushed the systems to their operational limits, acquiring video data at rates on the order of milliseconds and correlating the RGB and HSI data in both time and space.
To assess the accuracy of their workflow, the researchers compared the predicted reflectance with measured reflectance, after correcting for sensor and atmospheric effects. The results varied by wavelength range. In the visible spectrum, the platform predicted 95% of the water scene within 2% absolute reflectance, or about 30% of the water signal level. In contrast, the near-infrared range showed larger errors: for 95% of the scene, the normalized residual error reached up to 90%. The team attributed this increase to the limited spectral data in RGB video in the shallow water scene.
"Our platform shows that we can predict hyperspectral frames from RGB video with reasonable accuracy in the visible range," Lee notes. "The drop in performance at longer wavelengths highlights the need for broader spectral coverage of fewer-band data for the prediction algorithm."
Looking ahead, Lee sees opportunities to enhance the system, "Future improvements will focus on aligning and calibrating the spectrometer and camera fields of view more precisely, and on developing more advanced prediction models."
By combining affordable RGB cameras with hyperspectral technology, this new platform opens the door to more accessible environmental video monitoring. With further refinement, it could support a broad range of applications, from water quality assessment to vegetation analysis and beyond.
Read the Gold Open Access paper by Chris H. Lee et al., " Coordinating high-resolution hyperspectral and RGB video acquisition of dynamic natural water scenes ," J. App. Rem. Sens. 19(2) 024507 (2025) doi 10.1117/1.JRS.19.024507 .
