Inside some fusion energy systems, particles hotter than the core of the sun can become unruly in a few thousandths of a second, far faster than any human operator can react. A new framework for software developed by researchers at the U.S. Department of Energy's (DOE) Princeton Plasma Physics Laboratory (PPPL) and Princeton University hands those split-second decisions to artificial intelligence (AI), while keeping the machine safe and humans firmly in charge of the goals.
Known as PACMAN (a novel abbreviation for Prediction And Control using MAchiNe learning), the AI framework was successfully tested on a real fusion system in five different experiments. The framework's design and first results are detailed in a new paper in the journal Nuclear Fusion.
Fusion could one day serve as a virtually unlimited source of electricity. Scientists are working on several ways to perfect the process here on Earth, including devices called tokamaks, which use powerful magnetic fields to hold a plasma: an electrically charged gas often called the fourth state of matter. Keeping the plasma hot, dense and stable requires constant adjustments to the tokamak, including its heating systems, magnets and gas injectors. The fusion reaction can be thwarted by small disturbances in the plasma, known as instabilities, that grow in milliseconds.
Combining AI and plasma expertise to speed up simulations
Plasma is also hard to predict. The sophisticated computer programs used to simulate the behavior of a plasma can take days or months to run. That approach is far too slow to work in real time during an experiment that might only last minutes.
"That's great for preparing for the next experiment in a year, but for control we need models that make a decision in the moment," said co-lead author Hiro Farre Kaga, a graduate student in the Princeton Program in Plasma Physics, which is a joint program of Princeton University and PPPL. "Machine learning models can describe the plasma behavior very well, and importantly, they are the only way we have to model the plasma in millisecond times. The speed of these models is what's key for control."
One AI framework that works with many models to push fusion energy research forward
Machine learning has already shown great promise for taming fusion plasmas. But most attempts to use it to control a fusion plasma were built from scratch, without a set of overarching design principles to ensure the models could be easily combined, and multiple models are needed to monitor and control different aspects of the fusion system.
"We developed this framework so that models could communicate, outputs from those models could be shared and we could do exciting physics in one integrated system," said Andy Rothstein, a graduate student at Princeton University's Department of Mechanical and Aerospace Engineering and co-lead author of the paper.
The result is a control loop involving multiple machine learning models that operates at a pace no human could match. "A really focused human operator can respond on the order of seconds," Rothstein said. "The whole PACMAN framework typically runs in about 20 milliseconds, and it's not running once. It's running again and again and again. It can see small things happening in the plasma and adjust in a way that a human would never be able to do."
PACMAN works like an assembly line with four stations. First, it gathers all of the tokamak's real-time measurements, including temperatures, densities and magnetic signals. Then it checks those values for errors and puts them in a single package. Next, AI models read whichever measurements they need and predict what the plasma is doing or about to do. Then, controllers use those predictions to calculate commands, such as turning up a heating beam. Finally, an output stage resolves conflicts between controllers, enforces strict hardware safety limits and sends the commands to the tokamak. Because each model and controller works independently, researchers can add new ones without disturbing the rest.
Proving it in a real fusion energy experiment
The team demonstrated that flexibility in five experiments on the DOE's DIII-D National Fusion Facility tokamak in San Diego. During the experiments, PACMAN:
- Allowed an AI model trained through a trial and error approach known as reinforcement learning to take complete control of the heating systems.
- Predicted sudden bursts of energy from the plasma's edge.
- Detected and controlled waves in the plasma driven by fast particles.
- Adjusted the plasma's density and rotation to targets set by the researchers.
- Predicted an instability called a tearing mode and stopped it before it happened.
Conventional controllers can only detect a tearing mode after it has started. "Then they try to suppress it, and that can come with a lot of performance degradation," Farre Kaga said. "In one of the experiments we present, a machine learning model predicts the tearing mode about 200 milliseconds in advance, so the plasma can be changed to avoid it in the first place."
The framework also simultaneously steered all six of DIII-D's gyrotrons (systems that heat the plasma with powerful microwave beams) to achieve complex goals set out by the researchers in advance by retargeting the gyrotron mirrors and adjusting their power in real time. "There was no algorithm to find that optimal solution before," Farre Kaga said. "When the shot ended and we looked at the data, it was doing exactly what we hoped, simultaneously moving all six in an optimal way to reach the goal."
Faster iteration, with humans in charge
For Rothstein, the biggest surprise was how quickly the framework showed its value. Building PACMAN and installing the first model took months. "Then we went to put in the second model, and it took a couple of days. The testing was easier, and there were far fewer bugs," he said. "DIII-D is first and foremost a research machine, and sometimes things don't work out the way you expected. If you can put a model on in a week, you can retrain it and put a new one on the week after. It allows for iteration that wasn't possible previously."
The researchers stress that PACMAN doesn't remove people from the process. The framework enforces safety limits despite what an AI model suggests, and physicists review each experiment to tune the controllers for the next one. "No matter how sophisticated your controllers, in the end it's a human operator that sets the parameters for that control," Farre Kaga said.
Because the framework is modular, its creators believe the approach can travel well beyond DIII-D to tokamaks with different sizes, shapes and instruments, including machines that haven't been designed yet.
"PACMAN uses a flexible setup where building block AI algorithms can be put together. You can add a new one, swap one out or run several at once without touching the rest of the system," said Egemen Kolemen, associate professor of mechanical and aerospace engineering at Princeton University, jointly appointed with the Andlinger Center for Energy and the Environment and PPPL. "That modularity is what turns AI plasma control from a series of one-off demonstrations into infrastructure the whole fusion community can build on."
Other authors on the paper include Ricardo Shousha, Keith Erickson and SangKyeun Kim from PPPL, Jalal-ud-din Butt, Peter Steiner and Azarakhsh Jalalvand from Princeton University and Takuma Wakatsuki from Japan's National Institutes for Quantum Science and Technology. The research was supported by the DOE Office of Science using the DIII-D National Fusion Facility under awards DE-FC02-04ER54698, DE-SC0015480 and DE-AC02-09CH11466 and by the National Science Foundation Graduate Research Fellowship under grant DGE-2039656.
