Our eyes do not always tell us exactly where things are – and that may be a feature of how biological vision works, rather than simply a flaw. A new study by York University researchers uses a common illusion to ask if artificial intelligence is meant to see more like us, should it make some of the same systematic perceptual "mistakes"?
For example, after staring at something moving steadily in one direction, a stationary object viewed immediately afterward can appear slightly displaced in the opposite direction. This well-known visual illusion, called a motion aftereffect, gives scientists an unusual window into the computations underlying perception: the image itself has not moved, but our experience of where it is has changed.
"Today's AI vision systems are impressive, but they still do not always see the world the way we do. This study captures the promise of NeuroAI and what it can do when neuroscience and artificial intelligence are brought together. By using smart experiments to reveal the computations biological vision uses and AI still lacks, we can use those insights to build better, more brain-like artificial systems," says senior author York Assistant Professor Kohitij Kar, the Canada Research Chair in Visual Neuroscience and a member of York's Centre for Vision Research and Centre for Integrative and Applied Neuroscience .
Current AI vision systems can often determine where an object is accurately, but they generally do not reproduce the way recent visual experience can reshape that answer. In humans, staring at motion can make a subsequently viewed stationary object appear displaced even though its pixels have not moved. The researchers found a corresponding change in the primate visual cortex – but not in the AI models they tested.
To gain better insight into the issue, the researchers examined whether artificial neural network models capture the same history-dependent changes in spatial representations seen in biological vision, or whether their position representations primarily reflect the physical properties of the image. The researchers, including the paper's first author and York graduate student Elizaveta Yakubovskaya, used precise measurements to find out where AI differs from biological vision and how that gap could be bridged.
"Combining recordings from primate visual cortex with human perception experiments, we used motion adaptation to induce a visual illusion and make a stationary object appear slightly shifted in position, then asked whether the brain and AI showed the same effect. Human observers reported the illusion, and neural representations of position in the primate inferior temporal (IT) cortex shifted in the same direction, even though the image itself had not changed," says Yakubovskaya.
The researchers leveraged motion adaptation to show where perceived and pixel-based positions diverge, allowing them to test the behavioral relevance of IT codes. The findings not only further our understanding of IT's role in spatial information encoding but provide a new benchmark to evaluate dynamic vision models.
"There is a growing question in AI about whether increasingly capable systems will become more like us or increasingly different from us," says Kar of the Faculty of Science and a member of the York-led Connected Minds . "If we want AI that works with humans and understands the world in more human-compatible ways, we cannot focus only on whether it gets the right answer. We also need to understand the computations that produce human perception and behavior. Neuroscience gives us a way to discover those computations and, potentially, build them into AI," adds Kar.
The study titled, The macaque IT cortex but not current artificial vision networks encode object position in perceptually aligned coordinates , is published today in Current Biology.
