A team at Sandia National Laboratories is creating heavy rain and thick fog in an indoor test facility to study how commercial security cameras perform during bad weather. (GIF by Joel Ortiz)Many buildings and sites tied to national security rely on cameras and sensor networks to detect people and other objects nearby, but weather conditions can quickly obscure their view. To understand how these systems perform, Sandia National Laboratories researchers are deliberately recreating thick fog and heavy rain in a controlled chamber.
Using commercially available cameras and sensors, the team is measuring how reliably the systems detect people as visibility declines and whether machine learning can improve recognition of people and other objects.
The work takes place in Sandia National Laboratories' fog chamber, where researchers can make and measure thick fog and, with a recently added capability, heavy rain. By reproducing known conditions, researchers can compare sensors under the same conditions rather than waiting for the right weather outdoors.
"Generating these environments in a controlled chamber allows us to plan scientific studies that require reliable, repeatable conditions instead of waiting for exactly the right weather to strike," said optical engineer and lead facility researcher Christian Pattyn.
The new rain system expands the chamber's testing capabilities beyond fog and gives researchers a broader range of degraded visual environments in which to evaluate cameras and other sensors.
Testing in rain and fog
Pattyn and the fog chamber team run the facility and help internal and external customers test sensors for a variety of applications.
For this project, computer engineer Maria Primavera and her team brought six off-the-shelf cameras, including a thermal camera, into the chamber. The goal is to collect benchmark data on how commercial cameras perform in low visibility.
The researchers mounted the cameras in two rows inside a clear protective case and ran them at the same time. Primavera said commercial systems can come with different claims about how they perform in challenging environments, but running the cameras at the same time makes the comparison fair. If the cameras were tested one at a time, she said, the fog and the target both might be slightly different.
"Different vendors have different claims about how their systems perform. In degraded visual environments, we want to see if those claims are true, what the bounds of performance are, and where performance breaks down," Primavera said.
From a control room outside the chamber, researchers connected the cameras to computers that displayed live views of what each camera was seeing. Before each test, they checked the feeds and adjusted focus as needed.
The team also synchronized the cameras and used a data-logging tool to timestamp each recording so they could compare frames captured at the same instant across all cameras.
The results can help teams understand what current camera systems can and cannot do in low-visibility conditions and provide data for evaluating machine-learning approaches.
Making bad weather repeatable
Pattyn and the facility team create fog by preparing a mixture of water, ice and salt and piping it into the chamber, where it sprays from nozzles near the ceiling. Salt helps the particles grow by absorbing water, while ice helps lower the temperature.
During a test, some members of the research team walk through the chamber along planned paths while others monitor the live video feeds.
For rain tests, the facility can produce a brief but heavy downpour inside the chamber, using about 25 to 30 gallons of water to create 30 to 60 seconds of rain. Water sprays from ceiling nozzles onto layers of netting, where it collects and forms larger droplets before raining down into the chamber.
The chamber uses 16 rain nozzles and 64 fog nozzles. LED and incandescent lighting can be turned on or off to create different light profiles.
"We tested the cameras in a full range of environments, including conditions with no precipitation, up to the most severe fog and rain conditions the chamber can produce," Primavera said.
