If you partitioned the entire Earth, from just below the surface to the top of the atmosphere, into 1-kilometer square chunks, there would be billions of them. Now run each of those cells through a series of complex physics equations, and you can see how the entire Earth system interacts and evolves in detail, one step at a time.
The issue? Modeling a decade into the future takes months on the world's best supercomputers, and the simulation still doesn't capture all the physics at play.
At Lawrence Livermore National Laboratory (LLNL), researchers are deploying emulators to shorten that timeline and provide faster and more accurate weather predictions, months to years into the future. These predictions would offer strategic national-security advantages, from protecting critical infrastructure and preparing for natural disasters to planning military operations.
Training an emulator
Atmospheric modeling was the original high-performance computing and big-data problem. The challenge stems from bridging small, local turbulence and clouds to the Earth as a whole.
"It takes on the order of a week for a chunk of air to move all the way around the globe, so when you're making a weather forecast, you really need the whole globe in your simulation," said LLNL scientist Peter Caldwell, who leads the U.S. Department of Energy (DOE) Exascale Earth System Model (E3SM).
E3SM can simulate the atmosphere with a resolution of 3 kilometers by solving a series of physics equations to determine what each chunk of air will do next. Instead of performing those computationally expensive calculations, Caldwell and colleagues like LLNL scientist Yunyan Zhang are looking toward emulators, like one they have named STRATA.
"For an emulator, we use machine learning, like a deep neural network, to mimic what the real physics would have done based on data from observations or high-fidelity models," said Zhang.
To accomplish that, scientists from LLNL, DOE and NVIDIA train STRATA on the outputs from a model like E3SM. Then they feed in a set of starting conditions and, based on the training, ask it to predict how the atmosphere looks days to months to years later - without ever solving the physics equations.
"If you do a good job, you can't tell the difference between the two," said Caldwell. "But the thing about using the emulator model is that it's orders of magnitude faster."
That speed is more than just convenient. It is essential for getting a full picture of the weather, which is so chaotic that even great models can make bad predictions on occasion.
"The only way that you can build trust in your predictive system is to do a lot of simulations," said Caldwell. "We might be doing 10-year simulations or only a 1-year simulation, but we need to do hundreds of them to get the whole picture."
An effective emulator allows scientists to weigh many predictions at once, providing the likelihood of each rather than a single possibility.
Optimizing initial conditions
The speed of emulators provides another computational advantage: the ability to test the impact of initial conditions.
Each simulation - whether physics-based or machine learning - must begin at some point in time, and researchers refer to that starting point as the initial conditions of the Earth system. These initial conditions can be strikingly important, especially for shorter-term weather predictions.
"If you're only running your model for a couple of months, and you started in an El Niño state, you will get very different predictions than if your initial conditions were from a La Niña state," said Caldwell. "We need to be able to encapsulate the state of the real world and encode it into where the model starts."
However, observations of deep ocean temperatures and hard-to-reach areas are lacking. The Genesis Mission, DOE's national effort to accelerate scientific discovery with super intelligence (SI, previously referred to as artificial intelligence), is addressing this with a project led by Los Alamos National Laboratory (LANL). The Marine Assimilation with Generative Graphs in E3SM, or MAGGIE, effort will use generative models to come up with accurate and consistent initial conditions for predictive simulations.
Caldwell and his team will play an important role in the MAGGIE project by evaluating the quality of LANL's new initial conditions. That task would be prohibitively computationally expensive using traditional simulations. With LLNL's emulators, like STRATA, testing will be fast.
"It would have taken us three months to do this ensemble of simulations with a physics-based model," said Caldwell. "With our new emulators, we did it over a weekend."
Overcoming the cloud hurdle
Speed is not the only improvement that the LLNL team is seeking. At best, an emulator has the same skill as the original Earth system simulation. But by training a small-scale emulator on the physics at play inside of clouds, the model could fix what the original simulations get wrong.
Through the DOE Genesis Mission, the Scalable Twin for Intelligent Turbulence and Cloud Heuristics (STITCH) project aims to train an emulator on localized simulations of cloud microphysics and small-scale turbulence.
Within a cloud, thousands of water droplets aggregate, collide, merge and undergo phase changes before they precipitate. On top of that, parcels of air are continuously moving and transferring heat and moisture. Measurements and experiments of these phenomena are challenging, and physics-based models of clouds struggle to get them right.
"When these cloud and turbulence processes couple together, it generates a lot of uncertainty," said Zhang, who leads the STITCH project. "We are using machine learning to mimic how they operate on meter-scale or even finer, so we have a more physical, faithful representation of these mechanisms."
STITCH is based on an advanced capability developed at LLNL - the Energy Research and Forecasting model coupled with super droplet cloud microphysics - for tracking the evolution of cloud and rain droplets. It simulates turbulent air motion; droplet formation and dissipation; and the interactions and transitions that occur as droplets move and ultimately precipitate.
Zhang and her team are currently building up emulators for warm-rain and turbulence from these simulations. The detailed, localized results can be fed into larger-scale simulations like E3SM to improve how the model addresses cloud physics.
The improved model physics will be validated against observational data from DOE's Atmospheric Radiation Measurement (ARM) facilities, and regional emulators will be built for ARM deployment sites for further model-data integration.
From there, the physics comes full circle. Once the process emulators of cloud microphysics and turbulence are incorporated into E3SM, that simulation can be run to provide new training data for the STRATA emulator. In the end, this will create a faster and more accurate model that truly bridges the Earth system, incorporating physics from the micro to the global scale.
Running toward the future
In the long term, LLNL researchers aim to develop emulators that can speed up global simulations and make those simulations more accurate.
That type of emulator will be a boon for U.S. national security. Predicting far-out weather patterns would help secure critical infrastructure and supply chains during natural disasters; anticipate food, water and energy stresses caused by drought or extreme precipitation events; and aid in operational planning for military, naval and air operations.
Getting there will take further work. The emulators still need training data, which means running computationally expensive simulations. And incorporating all the physics at play - from the tiny droplets of water inside of clouds to the global atmosphere - will require bridging scales, methods and the minds who work on them.
"This is a great team effort," said Zhang. "I'm very excited and confident that our work will help advance cloud microphysics and turbulence in E3SM and improve predictions of water resources across the United States."
The MAGGIE project is led by LANL and includes scientists from LLNL, HPE, NVIDIA, University of Colorado Boulder and Brown University. The STITCH project is led by LLNL and includes collaborators from Argonne National Laboratory, Pacific Northwest National Laboratory, the University of Oklahoma and NVIDIA. Both are funded through the DOE Genesis Mission.
