The story so far: On December 4, Google DeepMind unveiled GenCast, an artificial intelligence (AI) model the company said could forecast the weather better than most existing tools as well as more days in advance. Details of the model were published in a peer-reviewed paper in the journal Nature.
How do we forecast weather?
“Weather predictions … are produced by running multiple numerical simulations of the atmosphere,” Vassili Kitsios, senior research scientist at the Commonwealth Scientific and Industrial Research Organisation of Australia, wrote earlier this month. “Each simulation starts from a slightly different estimate of the current weather. This is because we don’t know exactly what the weather is at this instant everywhere in the world. … By solving equations describing the fundamental physical laws of nature, the simulations predict what will happen in the atmosphere.”
This process is called numerical weather prediction (NWP). The best NWP forecasts require the use of powerful supercomputers as well as high-quality data about the weather at a particular location. Even then NWPs can predict the weather only a week or so in advance.
Ensemble forecasts entered the picture in the 1990s. Here, scientists use an NWP model to produce multiple forecasts at a certain location in time, with different starting conditions. This collection of forecasts is called an ensemble and indicates the range of meteorological possibilities.
How does GenCast perform?
Google’s GenCast uses ensemble forecasting too but the options in the ensemble come from an AI model rather than an NWP. Engineers at Google trained this AI model on 40 years of reanalysis data, from 1979 to 2019. According to the European Centre for Medium-Range Weather Forecasts (ECMWF), “Reanalysis data provide the most complete picture currently possible of past weather and climate. They are a blend of observations with past short-range weather forecasts rerun with modern weather forecasting models.”
GenCast was trained in two steps: step I in 3.5 days and step II in 1.5 days, both with 32 TPU v5 instances. ‘TPU’ is short for ‘tensor processing unit’, an integrated circuit Google developed to run machine-learning models, sold via Google Cloud. In December 2023, Google Cloud launched a TPU called v5p: it contains 8,960 chips interconnected with a bandwidth of 4,800 Gbps/chip, and costs $4.2 per chip-hour on demand.
Just like ChatGPT is good at identifying what the next word in an unfinished sentence could be, GenCast is good at guessing what the weather will be in the next moment given the weather until some point. According to the Nature paper, GenCast had “greater skill than ENS on 97.2% of 1,320 targets we evaluated and better predicts extreme weather, tropical cyclone tracks and wind power production.” ENS refers to the ensemble forecasts generated by ECMWF, considered one of the best in NWP.
Google also said GenCast was more accurate than ENS on 99.8% of the 1,320 targets when asked to predict the weather more than 36 hours in advance.
How does GenCast work?
The AI model described in the paper had a neural network with 41,162 nodes and 2.4 lakh edges. Each node is a point in the network where some input data is accepted, manipulated, and an output is generated as the input for another node. An edge is a connection between nodes.
For how this setup processes data, see the diagram below. Caption: The globes at the bottom show a weather prediction at four points of time, one after the other. Each prediction is generated by combining existing weather data with a noisy input. GenCast’s challenge is to extract from the noisy input – the globes on the top – a weather prediction for the next moment in time. To do this, the model runs the combination through a refinement (green box), produces a less noisy prediction, then combines this again with the input data, runs a second refinement, then combines the new output with the input data, runs a third refinement, and so on until it finishes 30 refinements. The final de-noised output, called X1, is the final weather prediction for the next moment in time. To predict the weather for the moment after, the model begins by accepting X1 as the input and starts afresh with a noisy input. The green boxes have the neural networks.
The ability to de-noise a noisy input is a common feature of a diffusion-type AI model, which GenCast is. Other famous apps that use diffusion models include OpenAI’s text-to-video model Sora and Stability AI’s text-to-image model Stable Diffusion, both of which are also examples of generative AI.
GenCast produces at least 50 forecasts at a time for the ensemble, and Google has said each forecast can be produced in parallel. In all, the ensemble contains forecasts for 15 days at a time, with a spatial resolution of 0.25° x 0.25° (latitude-longitude) and temporal resolution of 12 hours. The researchers found this entire process took GenCast running on one TPU v5 unit eight minutes, much shorter than the several hours required by supercomputers for NWP.
Will GenCast replace NWP?
GenCast’s forecasts are probabilistic rather than deterministic, i.e. “there will be 25% chance of rain in Chennai on December 25” rather than “there will be 5 mm of rain in Chennai on December 25”. Current NWP models and their ensembles are deterministic. Experts have said probabilistic weather forecasts are better at revealing the possibility of extreme weather events.
“We should make more use of these probabilistic forecasts for extreme events instead of relying on quantitative predictions. Probabilistic forecasts provide more lead time, which can be used for better preparation,” former secretary to the Indian government Madhavan Rajeevan wrote in The Hindu in December 2023.
This said, while GenCast’s performance suggests AI weather models will soon surpass the abilities of NWP models, both NWP and GenCast are founded on more fundamental weather data still acquired using the laws of physics. Experts have said understanding the weather using these laws remains important because the weather is changing rapidly in many parts of the world, in ways in which historical weather conditions can’t prepare us for.
GenCast itself requires more reanalysis data to train itself. As Google said in a public statement: “We deeply value our partnerships with weather agencies, and will continue working with them to develop AI-based methods that enhance their forecasting. Meanwhile, traditional models remain essential for this work. For one thing, they supply the training data and initial weather conditions required by models such as GenCast.” The code to run GenCast is available on GitHub.
DeepMind has also been working on a model called GraphCast to develop “deterministic medium-range forecasts”. Google Research has been developing a model called NeuralGCM that combines AI and NWP models to generate deterministic forecasts, and at least two other models to predict extreme floods and to quantify forecasting uncertainties. Elsewhere, Huawei’s Pangu-Weather model can predict the weather one week at a time with accuracy comparable to NWP but much faster. Nvidia’s FourCastNet model can already outperform a state-of-the-art NWP facility at ECMWF at predicting extreme rainfall, in less than two seconds.
Published – December 21, 2024 08:04 am IST
