If you ask a child to draw an animal that doesn't exist, they'll often cobble together components from real ones-say, the body of a seal with an elephant's trunk, four octopus arms, and one lizard eye.
This imaginative ability is theorized to stem from our larger capacity to learn symbolic units-an arm or a leg in the aforementioned example, or perhaps a word-and then envision how those symbols could be reused in a new context. Neuroscientists call this facility for recombining familiar elements into fresh ideas compositional generalization, and it is hypothesized to be key to problem solving, making sense of new situations, and creative thinking.
In new research published in Nature, Rockefeller University's Laboratory of Neural Systems has found the first evidence of the neural substrates that underlie this process. The team located it in the ventral premotor cortex, a section of the frontal lobe. The region appears to act as a sort of mediator between the prefrontal cortex, where higher-level thinking such as planning occurs, and the motor cortex, which enables movement.
In their findings, the researchers not only illuminate fundamental properties of neural function but also see implications for improving computer-brain interfaces (BCIs) and studying brain disorders.
"The discovery solves a long-standing problem in cognitive neuroscience: Where do symbols-the basic units of thought-come from?" says Winrich Freiwald, head of the lab. "It also points to a future-a near future-in which we can understand thinking mechanistically."
Action symbols
Compositional generalization is an influential hypothesis in neuroscience for explaining the wide variety of human abilities that use abstract thought to generate new ideas, including math, written and spoken language, drawing, dancing, handwriting, and musicianship. It may also characterize cognitive abilities we share with other animals, such as reasoning, object manipulation, and tool use.
However, there hasn't been definitive neuroscientific evidence of symbols. "The idea behind our research was, if these reusable components exist, what would their neural activity look like?" says first author Lucas Tian, a postdoctoral fellow in the lab. "If there are units that are being reused in different situations, then you should be able to see that in the neural data."
Designing an experiment to locate such neural mechanisms, however, was no mean feat. Only humans do math, use language, or draw, and the methods currently used for measuring brain activity in humans do not have the necessary resolution to monitor the activity of nerve cells in the brain.
To bypass that technical limitation, Tian worked with macaque monkeys. "We wanted to develop an animal model in which we can actually observe compositionality in action in the animals' behavior while simultaneously doing neural recordings to understand how the brain might be doing this," Tian describes.
But he still had to confront the problem of finding a behavioral paradigm for the animals that could uncover their compositional abilities. Tian's idea was to teach them to trace simple geometric figures on touchscreens-lines, squares, arcs, circles, triangles-and then task them with re-creating new shapes, all while observing their brain activity through sensors. Each simple shape was considered its own discrete knowledge unit, or action symbol-action because they had to physically execute the drawing of each one.
Then he built novelty into the experiment by testing how the monkeys drew new, more complex shapes. "I gave them a lot of symbol variation rather than having them repeat one simple task over and over. They had to learn how to grapple with new and changing factors, which is the sort of environment you'd find compositional generalization useful for," Tian describes.
He found that even though they could have drawn these new images by using a simple tracing strategy-moving their fingers along the edges of the shapes-they instead chose to recombine the symbols they had learned to create new complex combinations. This revealed that they had understood these actions as symbols-building blocks for creating novel drawings.
Surprising activity
Tian used an array of electrodes to observe hundreds of neurons across eight brain regions simultaneously throughout these activities.
"It was important for us to cast a wide net," he notes, "because no one knew whether-or where -compositional generalization might be occurring in the brain."
The study found that one particular region activated as the monkeys drew: the ventral premotor cortex, an area of the frontal lobe traditionally associated with the planning and execution of movement-especially hand movements. Tian and his colleagues found that the activity was not simply involved in motor execution but represented a high-level cognitive representation of the action itself.
"What Lucas found forces us to re-think the role of this part of the brain," Freiwald says. "It is not simply a part of the motor system one step removed from the control of the finger, but an area that generates a sort of mental typewriter. It specifies in an abstract format the 'key' to press when you want to express yourself in writing, and then instructs another area to turn that key into a stroke."
Insights into disorders of the human brain
The researchers believe their novel approach could develop into a foundational experimental paradigm that could be used in humans as well. Drawing is a widely used tool for diagnosing cognitive disorders; specific disorders result in specific drawing impairments. "One possibility is that the things we learn could lead to new insights into psychiatric disorders such as schizophrenia or action-planning disorders like constructional apraxia, where people have trouble creating complex action sequences even though they understand the task at hand and retain basic motor abilities," says Tian.
To that end, they plan to collaborate with neurosurgeons and their patients to gather brain activity data from people who have had a procedure involving brain implants, such as for epilepsy.
They also see possibilities for the improvement of BCIs. "Knowing how thinking works mechanistically will improve our ability to read the activity of the human brain and express it into speech or action through brain-machine interfaces, where such expression is not otherwise possible," Freiwald says.
Moreover, there are essential questions about cognition at play, he adds. "This is basic research on a fundamental quality of human nature-thinking, which is altered in many psychiatric disorders. We conduct this work with the goal of improving the human condition."
