Researchers at the University of Toronto Scarborough have harnessed artificial intelligence (AI) and brain activity to shed new light on the other-race-effect (ORE) - a phenomenon in which people struggle to accurately recognize faces of individuals from different races.
Combining AI and electroencephalography (EEG) brain monitoring, the researchers unearthed new insights into how people perceive other-race faces, including visual distortions more deeply ingrained in our brain than previously thought.
"What we found was striking - people are so much better at seeing the facial details of people from their own race," says Adrian Nestor, associate professor in U of T Scarborough's department of psychology and co-author of the studies.
"This is important because we should want to know why we have trouble recognizing faces from other races, and what influence that might have on behaviour."
In one study, published earlier this year in the journal Behavior Research Methods , the researchers used generative AI to look at individual responses to seeing images of faces.
Two groups of participants - one East Asian, one white - were shown a series of faces on a computer screen and asked to rate them based on similarity.
The researchers were able to generate visual representations of faces using a generative adversarial network (GAN), a type of AI that can be trained to create life-like images. Using the GAN's image generating ability, the researchers were able to see the mental images the study participants had of faces.
They discovered that faces from the same race were reconstructed more accurately than those from different races, and that people tend to see faces of other races as more average looking.
A second study, recently published in the journal Frontiers , looked more closely at brain activity that might be involved to explain ORE. Brain activity, which occurs in the first 600 milliseconds of seeing the images, was used to digitally reconstruct how the participants visually process faces in their mind.
Using EEG data, researchers found that the brain processes faces from the same race and faces from different races in distinct ways. Neural recordings associated with visual perception showed less differentiation for other-race faces (Nestor's lab first showed the potential of harnessing EEG for visual perception back in 2018. Since then the algorithms they used have improved significantly).
"When it comes to other-race faces, the brain responses were less distinct, indicating that these faces are processed more generally and with less detail," says Moaz Shoura, a PhD student in Nestor's lab and co-author of the studies. "This suggests that our brains tend to group other-race faces together, leading to less accurate recognition and reinforcing ORE."
One of the most intriguing findings from this study was that other-race faces appeared not just more average-looking, but also younger and more expressive in the minds of the participants, even when they weren't.
"This could explain why people often have difficulty recognizing faces from other races. The brain isn't processing facial appearance as distinctly and accurately," says Nestor.
Nestor says the findings improve our understanding of how bias forms in the brain, and have a variety of implications. They could be used to improve facial recognition software, gather more accurate eyewitness testimony and assist in diagnosis of mental health disorders such as schizophrenia or borderline personality disorder.
"It's important to know exactly how people experience distortions in their emotional perception," says Nestor.
Shoura adds that by further exploring the effect of perceptual bias, it might help in a range of social situations, from job interviews to combating racial bias.
"If we can better understand how the brain processes faces, we can develop strategies to reduce the impact bias can have when we first meet face-to-face with someone from another race."
The research was supported by funding from a Natural Sciences and Engineering Research Council of Canada (NSERC) grant.
