The wealth of information provided by our senses that allows our brain to navigate the world around us is remarkable. Touch, smell, hearing, and a strong sense of balance are crucial to making it through what to us seem like easy environments such as a relaxing hike on a weekend morning.
An innate understanding of the canopy overhead helps us figure out where the path leads. The sharp snap of branches or the soft cushion of moss informs us about the stability of our footing. The thunder of a tree falling or branches dancing in strong winds lets us know of potential dangers nearby.
Robots, in contrast, have long relied solely on visual information such as cameras or lidar to move through the world. Outside of Hollywood, multisensory navigation has long remained challenging for machines. The forest, with its beautiful chaos of dense undergrowth, fallen logs and ever-changing terrain, is a maze of uncertainty for traditional robots.
Now, researchers from Duke University have developed a novel framework named WildFusion that fuses vision, vibration and touch to enable robots to "sense" complex outdoor environments much like humans do. The work was recently accepted to the IEEE International Conference on Robotics and Automation (ICRA 2025), which will be held May 19-23, 2025, in Atlanta, Georgia.
"WildFusion opens a new chapter in robotic navigation and 3D mapping," said Boyuan Chen, the Dickinson Family Assistant Professor of Mechanical Engineering and Materials Science, Electrical and Computer Engineering, and Computer Science at Duke University. "It helps robots to operate more confidently in unstructured, unpredictable environments like forests, disaster zones and off-road terrain."
"Typical robots rely heavily on vision or LiDAR alone, which often falter without clear paths or predictable landmarks," added Yanbaihui Liu, the lead student author and a second-year Ph.D. student in Chen's lab. "Even advanced 3D mapping methods struggle to reconstruct a continuous map when sensor data is sparse, noisy or incomplete, which is a frequent problem in unstructured outdoor environments. That's exactly the challenge WildFusion was designed to solve."
