Texas Engineers teamed up with the world's largest semiconductor foundry to fabricate and test an emerging memory technology that could help deal with the increasing energy demand of artificial intelligence.
Together with Taiwan Semiconductor Manufacturing Company (TSMC), researchers tested SOT-MRAM, a type of memory that can retain information even when power is off. It uses magnetic properties, making it faster while also consuming less energy than other memory technologies.
"The unique combination of speed, energy efficiency and endurance makes SOT-MRAM perfectly suited for AI applications, especially in devices where resources like power and memory are limited," said Sam Liu, the first author of the new paper published in Science Advances and recent UT Austin Ph.D. graduate. "SOT-MRAM hasn't been considered for AI hardware since it can only hold two states, but we designed it so we can take advantage of the binary state while still being accurate."
The research team tested the chips on various AI tasks, including neural network inference, binary neural network training, and probabilistic graph modeling.
They took just 2 nanoseconds per write operation—a standard switching of the stored data between 0 and 1—while consuming only 2 picojoules of energy per write. Other memory technologies can take between 5x to hundreds of times longer—several milliseconds—to perform these tasks and consume hundreds of picojoules or more.
AI and the data centers needed to power it have led to rapid increases in demand in Texas and worldwide. Texas is expected to become the U.S. capital of data centers in the next few years, and that could lead to a 5x increase in statewide energy usage .
There are two ways to reduce AI's energy footprint: make the technology more efficient and reduce reliance on data centers. Improved memory capabilities, like SOT-MRAM, could do both.
"We show that SOT-MRAM AI accelerators can provide the energy efficiency, with enough accuracy, to eventually replace CPU-based AI accelerators in edge devices such as sensors," says Jean Anne Incorvia, associate professor in the Cockrell School of Engineering's Chandra Family Department of Electrical and Computer Engineering and the faculty leader on the project. "For example, take a robotic hand that senses heat. Just like a human, the local AI in the hand can make a quick decision with enough accuracy to move the hand, without even transmitting the neural signal to the brain. When very high accuracy is needed, then the robot can connect to GPU-based data centers in the cloud."
The researchers will continue their work on this technology by refining key characteristics that enable the chips' speed and efficiency and by reducing variation between devices, which can reduce the neural network accuracy.
