By pairing a retina-like sensor with a spiking neural network, a new technique with potential applications for self-driving cars, search-and-rescue drones or medical imaging can reveal objects invisible to standard cameras.
PROVIDENCE, R.I. [Brown University] - Researchers from the Brown University School of Engineering have developed a new imaging system that can see and track moving objects in partially opaque environments like dense fog or muddy water.
The technique, described in a study published in Advanced Science, combines a dynamic vision sensor - a camera with pixels that report only changes in brightness, rather than capturing full frames - with a spiking neural network computer model that reconstructs the moving objects spied by the sensor. In laboratory tests, the system's reconstructed images matched the hidden targets with structural similarity scores of up to 96% in turbid water and drifting fog that would hide the objects from a traditional camera, while they simultaneously tracked the targets' positions as they moved.
The brain-inspired system is the first end-to-end technique that can both track and image randomly moving objects hidden by such dense scattering media, the researchers say.
"The reason it's hard to see objects in fog is that the light bouncing off of objects gets scattered by the fog before it reaches our eyes or the camera lens," said Ning Zhang, a postdoctoral researcher in engineering at Brown. "This research project is about using novel imaging cameras and a brain-inspired model that can filter out that scattering process and reconstruct what's behind the fog."
