AMHERST, Mass. — Researchers in the Riccio College of Engineering at the University of Massachusetts Amherst have demonstrated that redesigning both the hardware and the algorithm can make AI applications on edge devices more efficient. As proof of concept, their system achieved 95.24% accuracy in language identification while reducing computing resources by 90%—the highest reported accuracy from a system of its kind.
AI is everywhere these days. It's embedded into edge devices—so called because they operate at the edge of a network, processing data locally instead of relying on remote data centers—such as smartphones and cameras, navigation and traffic control tools, and entertainment and shopping platforms.
But the expansion of AI into our everyday lives brings high requirements for energy and hardware. It's also tough for edge devices, including home automation hubs and onboard car interfaces, because they have limited computational capacity. It would be convenient if every device contained the processing capability of a data center, but that isn't realistic.
Some of the computational, energy and hardware requirements revolve around processing, which can be slow, cause problems with heat and run down the battery on edge devices. Take language, for example: We humans interact with AI directly and indirectly to perform ever-increasing tasks, which means many of these applications need to process human language, whether by recognizing, translating, or understanding speech or text. Even determining which language is being used—English, French, Arabic, etc.—places demands on processing that can tax small, battery-powered devices.
To facilitate processing on edge devices, Qiangfei Xia, the Dev and Linda Gupta professor of electrical and computer engineering and head of the Nanodevices and Integrated Systems Lab in the UMass Riccio College of Engineering, collaborated with colleagues across academia and industry for a redesign of not only the algorithm but also the hardware.
Conventional approaches typically implement AI algorithms on existing hardware; by contrast, Xia's team designed the algorithm alongside the hardware, allowing each to play to the other's strengths.
The team's solution harnesses the power of hyperdimensional computing (HDC) algorithms and analog in-memory computing (IMC) hardware to enable edge AI applications, including language identification.
HDC is a brain-inspired form of AI that represents information using large mathematical patterns instead of working with precise numbers. These patterns enable certain AI tasks to be performed with simpler, more efficient computations.
IMC hardware—specifically, an array of memristors , which are electrical components that both store and process data in a single physical location, reducing the need to move data between separate memory and processing units—allows for HDC-based algorithms to be deployed on edge devices.
The platform, Xia said, can both encode language features and process language identification. "The encoding part leverages the intrinsic randomness of memristive devices, considered by many to be a drawback for this device technology," he said. Essentially, the researchers have turned a flaw into a feature: The natural variability of the memristive chip offers useful randomness for efficient data encoding.
The system demonstrated 95.24% accuracy in language identification with a 90% reduction in computing resources—the highest reported accuracy from an HDC implementation on an emerging hardware platform.
It's a natural evolution of previous work done in Xia's lab. "Scientific research follows an ascending spiral," he said. "Each iteration builds on prior ideas, leading to deeper understanding and greater scientific advances."
"This work represents another milestone in our memristor research," Xia added. "The system-on-chip builds on our previous research, spanning analog memristor devices , memristor-CMOS circuit integration, memristive crossbar arrays and machine intelligence applications."
The idea first arose in 2019, Xia said, when Daniel Belkin, then an undergraduate research fellow at UMass, started studying a small memristive crossbar array. But developing the approach took time and experimentation. "Only as the technology advanced to the system-on-chip level did practical language-processing demonstrations become feasible," Xia said.
This project is only one in a series of applying memristive chips to AI applications, he noted. "The same chip was used for RF signal processing and smart sensing ." Another article is coming up on applications of memristive chips for wireless receivers, he said.
Future applications of the current research could include processing spoken languages as well. "We demonstrated written language processing in this work; we believe it will also be capable of spoken language processing, leading to energy-efficient natural language processing on edge devices such as phones, speakers, cars, robots, etc.," Xia said.
Xia's collaborators for this project include former UMass postdoctoral researcher Yi Huang, who is now an assistant professor at the University of Tennessee Knoxville; UMass Ph.D. student Alireza Jaberi Rad; former student Belkin; Ning Ge of TetraMem Inc.; J. Joshua Yang of TetraMem and the University of Southern California in Los Angeles; and Miao Hu of TetraMem.
"The interdisciplinary nature of this work suggests that developing a new AI hardware requires collaborative efforts from academia and industry, and complementary expertise from across the stack, such as device, circuits, algorithm, architecture and system," Xia said.
The work is published in Nature Communications .
About the University of Massachusetts Amherst
The flagship of the commonwealth, the University of Massachusetts Amherst is a nationally ranked public land-grant research university that seeks to expand educational access, fuel innovation and creativity and share and use its knowledge for the common good. Founded in 1863, UMass Amherst sits on nearly 1,450-acres in scenic Western Massachusetts and boasts state-of-the-art facilities for teaching, research, scholarship and creative activity. The institution advances a diverse, equitable and inclusive community where everyone feels connected and valued—and thrives, and offers a full range of undergraduate, graduate and professional degrees across 10 schools and colleges and 100 undergraduate majors.
