Software can help developers assess AI model energy use, which is necessary to lower costs and strain on the grid
Studies: The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization (NeurIPS)
Where Do the Joules Go? Diagnosing Inference Energy Consumption (arXiv)
AI users and developers can now measure the amount of electricity various AI models consume to complete tasks with open-source software and an online leaderboard developed at the University of Michigan.
Companies can download the software to evaluate private models run on private hardware. And while the software can't evaluate the energy costs of queries run on proprietary AI models at private data centers, it has allowed U-M engineers to measure the power used by open-weight AI models in which the parameters under the hood are publicly available. The power requirements can be viewed on an online leaderboard, which was updated this month. Their results have revealed trends on how AI energy use varies with model design and implementation.
"If you want to optimize energy efficiency and minimize environmental impact, knowing the energy requirements of the models is critical, but popular benchmarks for assessing AI ignore this aspect of performance," said Mosharaf Chowdhury, associate professor of computer science and engineering and the corresponding author of a study describing the software.
Tools for informed decision-making
The researchers measured energy use across several different tasks, including chatting, video and image generation, problem solving and coding. For some tasks, the energy requirements of open-weight models can vary by a factor of 300. With the results, Chowdhury's team has developed tutorials for developers to learn how to measure and lower the energy costs of their models. They gave their latest tutorial at the Neural Information Processing Systems (NeurIPS) Conference in December.
The researchers designed their software with partial funding from the National Science Foundation to help solve AI's growing energy demands. Between 80% and 90% of the sector's energy is consumed when a trained model processes a request at remote data centers-what the industry calls inference.
As AI models grow in size and are used more often, they need more power. Data centers in the United States consumed about 4% of the country's total power in 2024-or about as much as Pakistan uses in a year. Data centers are projected to use twice as much power by 2030, according to a study by the Pew Research Center. But many estimates on AI growth rely on 'envelope' calculations, which are made by multiplying the maximum power draw per GPU by the number of GPUs. It's only an estimate of the highest possible energy cost.
"A lot of people are concerned about AI's growing energy use, which is fair," Chowdhury said. "However, many who worry can be overly pessimistic, and those who want more data centers are often overly optimistic. The reality is not black and white, and there's a lot we don't know because nobody is making direct measurements of AI power use available. Our tool can provide more accurate data for better decision-making."
Why do some AI models use more power?
The team's assessments of open-weight models revealed larger trends in how an AI's design could affect its energy requirements. A key factor was the number of generated tokens-the basic units of data processed by AI. In LLMs, tokens are pieces of words, so wordier models tend to use more energy than concise models. Problem-solving or reasoning models also use more energy because they generate "chains of thought" that contain 10 to 100 more tokens per request.
But the energy requirement of even a single model can change, depending on how it's run at the data center. Processing queries in batches, for example, will result in less energy use at the data center overall, although larger batches take longer to run. The choice of software for allocating computer memory to queries can also impact AI's energy requirements.
"There are many ways to deploy AI and translate what the model wants to do into computations on the hardware," said Jae-Won Chung, U-M doctoral student in computer science and engineering and the study's first author. "Our tool can automate the search through that parameter space and find the most efficient set of parameters based on the user's needs."
The research was also supported by grants and gifts from VMware, the Mozilla Foundation, Cisco, Ford, GitHub, Salesforce, Google and the Kwanjeong Educational Foundation.
