HOBOKEN, NJ., May 4, 2026 — In a novel attempt to improve how large language models learn and make them more capable and energy-efficient, Stevens researchers devise a groundbreaking algorithm that improves AI data sharing, boosts performance and reduces power consumption.
Large language models like ChatGPT are huge. Letting many people train them together without sharing users' private data—an approach called federated learning—is slow and inefficient. To collaborate, the models must share their updated versions of the entire data all the time—and that's a huge amount of information to exchange. This approach uses a lot of network bandwidth memory and is energy intensive. As a result, models can't be synchronized as often as necessary, resulting in outdated versions.
"It's too much data to share," says Stevens PhD candidate Yide Ran, who was the driving force behind the effort to improve the process. "It's like sending in an entire encyclopedia when you only need to change a few entries. But you really don't need to do that."
Working together with his advisors Zhaozhuo Xu, Assistant Professor of Computer Science at the School of Engineering who studies machine learning at Stevens Department of Computer Science and Denghui Zhang , Assistant Professor in Information Systems and Analytics at the School of Business, Ran sought to improve how language models share their data.
The team built upon the previously known concept that effective learning in large language models is often driven by a surprisingly small but well-chosen subset of parameters. The result is a more agile, faster-working model that also uses less energy. The researchers named the model MEERKAT after the animal, known for its dexterity and speed.
The team outlined their findings in a paper titled Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity , which was presented at the International Conference on Learning Representations and published in OpenReview.net.
Instead of sharing the entire giant AI model, MEERKAT shares updates to only 0.1 percent of the model, which includes the most important parameters. "So you are no longer sending the entire encyclopedia when only a few key definitions have changed," explains Zhang. That shrinks communications by over 1000 times. "Updates that used to be gigabytes are now just a few megabytes," Zhang says.
MEERKAT's other efficiency secret is using a different error-checking approach. Standard AI training requires an intense mathematical process called backpropagation, which stands for backward propagation of errors, in which AI performs self-checks to avoid mistakes. Although it's a core algorithm used to train neural networks by minimizing the difference between predicted and actual outputs, backpropagation consumes huge amounts of memory and energy. MEERKAT simply tweaks the model slightly and checks the results, completely bypassing backpropagation.
Finally, small updates allow for more frequent synchronization of data, which is another breakthrough, as it keeps models up to date. "Because updates are so tiny, data can be now sent back and forth more often," says Xu. "The result is a much better shared model."
This new approach substantially reduces computational and communication costs, helping make advanced AI adaptation more feasible for resource-constrained institutions, researchers say. Their work will also support more equitable deployment of AI in domains such as healthcare, education and cross-institutional collaboration, where centralized data collection is often difficult to achieve due to privacy and other issues.
About Stevens Institute of Technology
Stevens is a premier, private research university situated in Hoboken, New Jersey. Since our founding in 1870, technological innovation has been the hallmark of Stevens' education and research. Within the university's three schools and one college, more than 8,000 undergraduate and graduate students collaborate closely with faculty in an interdisciplinary, student-centric, entrepreneurial environment. Academic and research programs spanning business, computing, engineering, the arts and other disciplines actively advance the frontiers of science and leverage technology to confront our most pressing global challenges. The university continues to be consistently ranked among the nation's leaders in career services, post-graduation salaries of alumni and return on tuition investment.
