Brain Researchers Afflict LLMs With injuries Similar To Those Of Stroke Victims To Unpack AI Language Mechanisms

Cell Press

For more than 150 years, doctors and scientists have mapped the functions of various brain regions by studying people with brain injuries and determining how those injuries contributed to specific behavioral deficits. Now, a team of researchers that specialize in post-stroke language and speech difficulties report that the same framework can be applied to the study of large language models (LLMs). The research, publishing in the Cell Press journal Patterns on November 19, shows that inducing "injury" in these models and then administering tests given to patients with brain injury provides insight into how these models use language and could help patients with communication challenges in the future.

"LLMs are being deployed in situations where failure has consequences, but we have very little ability to say why a given system fails the way it does," says corresponding author Julius Fridriksson of the University of South Carolina. "It turns out that methods used in clinical neurology can be applied to understanding artificial intelligence. This is the first paper to use these methods in this way."

Fridriksson's lab focuses on the study of aphasia, or difficulties with language and speech. His research employs a number of brain imaging techniques to study patients who are experiencing aphasia due to strokes. The efforts of the lab are primarily focused on helping patients recover, but at the same time, the team has revealed many new details about the human brain and how different networks within the brain control language.

"Our motivation for doing this research was simple. Language models are trained on human language, and aphasia is an impairment of human language," Fridriksson says. "So, the question was whether we could perturb layers in the language model, give it the same clinical tests we give our patients, and see whether the output looked like aphasia."

To do this, they established an approach they call lesion-based interpretability, in which they intentionally introduced errors, or lesions, into the LLMs. They found that these "injuries" led to errors in language processing that were similar to their aphasia patients.

"LLMs make errors in language and cognition and comprehension that are very similar to those made by humans," says study co-author Roger Newman-Norlund, also of the University of South Carolina. "The approach of lesioning these models in the same way that human brains have lesions and seeing how the symptoms and errors overlap was possible because of the large amount of data our lab has collected."

"Until now, comparisons between human and artificial intelligence language have been almost entirely correlational," Fridriksson adds. "This approach for the first time establishes cause and effect." But he notes that it's important to recognize that what the paper is suggesting is correspondence between patterns of failure, not correspondence between actual structures.

The researchers say these models can help to interpret what's happening inside the "black box" of LLMs to study the efficiency of LLMs by identifying parts of the models that are redundant and unnecessary. This could lead to future LLMs that are smaller and more energy efficient. The models could also be used to create "digital twins" of patients with aphasia, to simulate clinical trials and speed up the translation of new discoveries.

Fridriksson says he believes this framework has applications for studying neurological conditions beyond aphasia. "I hope this approach will be embraced to make scientific discoveries that help patients, because that's really where my passion is," he says.

/Public Release. This material from the originating organization/author(s) might be of the point-in-time nature, and edited for clarity, style and length. Mirage.News does not take institutional positions or sides, and all views, positions, and conclusions expressed herein are solely those of the author(s).View in full here.