Turn Your Latest Observations Into Timely Weather Decisions With NVIDIA Earth-2

來源原標題

Turn Your Latest Observations Into Timely Weather Decisions With NVIDIA Earth-2

作者

Elizabeth Goodman

原始標籤

Data Science · Developer Tools & Techniques · Simulation / Modeling / Design · Climate / Weather / Ocean Modeling · Earth-2 · Energy · PhysicsNeMo

原始連結

https://developer.nvidia.com/blog/turn-your-latest-observations-into-timely-weather-decisions-with-nvidia-earth-2/

摘要原文

Weather-sensitive industries increasingly have access to observations that offer an earlier, more local view of changing conditions. Energy companies collect measurements across wind and solar assets, emergency management teams rely on radar and local sensors, and satellite providers continuously observe the Earth. This data helps organizations understand and manage physical risk across sectors such as capital markets, insurance, agriculture, and logistics. With the AI data assimilation tools in NVIDIA Earth-2, you can process these observations more efficiently. By incorporating proprietary or third-party data, you can use these tools to issue forecasts more frequently, keep estimates aligned with real-time conditions, and tailor your forecasting pipeline to specific regions and applications. This tutorial covers two techniques: Constraining diffusion models with point observations, typically used for regional models. Assimilating disparate datasets into a consistent state, typically used for global models. Prerequisites For this tutorial, you will need: A development environment with Earth2Studio installed An NVIDIA RTX PRO or data center GPU A basic knowledge of Python Approximately 30 minutes Improve regional forecasts with observations You might run a regional weather forecasting pipeline for managing energy production and demand, drawing observations from wind and solar parks, transmission corridors, or densely populated areas. With AI data assimilation, you can use these observations to constrain your forecast where local accuracy matters most, helping you improve operational decisions. The same techniques can support other sectors using observations from production sites, event venues, logistics networks, or other assets where local conditions directly drive decisions. You can use Score-Based Data Assimilation (SDA) to incorporate observations into diffusion-based AI downscaling and forecasting models such as CorrDiff and StormCast. SDA guides the model toward predictions that are consistent with your observations without requiring to retrain the model. Figure 1, below, shows how the process works under the hood. Diffusion models generate high-resolution predictions through a sequence of denoising steps. At each step, SDA compares the intermediate prediction with your observations and nudges the model in the right direction. The output of SDA is probabilistic, with less uncertainty near observation locations and a wider spread further away, where predictions are increasingly governed by the other model inputs and the underlying AI simulations. Figure 1. SDA leverages the multi-step denoising process of diffusion models. At each step, the intermediate result is compared with observations to guide the next denoising step. The final output is an observation-informed, high-resolution weather prediction To nudge the model, you define an observation operator, which maps the model output to the quantity you would expect to observe at each measurement location. This is particularly straightforward for in situ measurements of physical quantities such as temperature or wind speed. In this case, the simplest form of an operator interpolates nearby grid values to each observation location. It is also possible to create operators for proxy measurements or observed impacts. For example, the power output of a wind turbine can act as a proxy measurement of wind speed. SDA unlocks two major capabilities: Update forecasts more rapidly. Numerical analyses require substantial processing time and are released on fixed schedules. SDA enables you to incorporate observations continuously. Figure 2, below, shows a concrete example of a pipeline forecasting at one-hour intervals and depending on a global analysis with a six-hour dissemination schedule. Incorporate proprietary, regional or domain-specific observations. Numerical analyses draw on a broad range of observations. SDA lets you incorporate data from your own sources to focus your forecast on specific asset locations or downstream applications. The effectiveness of SDA depends on several factors: the number, spatial distribution, and accuracy of your observations; the characteristic length scales of the field you are predicting; and the quality and well-posedness of the observation operator. Figure 2. Example pipeline using SDA for downscaling (CorrDiff + SDA) and high-resolution forecasting (StormCast + SDA). CorrDiff creates the high-resolution initial conditions for StormCast. SDA bridges the gap between the analysis reference time and current time by incorporating observations into downscaling and forecast steps that lie in the past How to run CorrDiff-SDA in Earth2Studio CorrDiff is a technique for AI-based downscaling. Earth2Studio provides a CorrDiff model pretrained over Europe that turns 0.25° weather fields into 2.2-km predictions. Using AI data assimilation, you can improve these predictions with observations where local accuracy matters. With the refined outputs, you can then initialize a regional forecast or create a reanalysis dataset for calibrating downstream models. Start by loading the pretrained model. We limit the domain to a part of the Netherlands and northwestern Germany and choose to assimilate 10-meter wind speeds. from datetime import datetime from earth2studio.data import GHCNHourly from earth2studio.models.da import CorrDiffCosmoEra5SDA domain = dict(lat_min=50.2, lat_max=53.8, lon_min=4.6, lon_max=10.4) sda = CorrDiffCosmoEra5SDA.load_model( CorrDiffCosmoEra5SDA.load_default_package(), assimilate_variables=("u10m", "v10m"), resolution="rea2", domain=domain, number_of_samples=1, sampler_steps=12, amp=True, ).to("cuda") Fetch the ERA5 data for low-resolution conditioning and GHCN wind observations over the domain. # Fetch and regrid ERA5 inputs onto the high resolution regional grid # Follow the link to the example below for the full implementation init_time = datetime(2024, 1, 26) x = fetch_and_regrid_era5(init_time, domain) # Fetch GHCN hourly 10-m wind observations over the model domain lat, lon = sda.model.lat_output_numpy, sda.model.lon_output_numpy bbox = lat.min(), lon.min(), lat.max(), lon.max() ghcn = GHCNHourly(stations=GHCNHourly.get_stations_bbox(bbox)) obs = ghcn(init_time, ["u10m", "v10m"]).dropna(subset=["observation"]) Lastly, run the model with the input data. We perform two runs to measure how the additional observations affect the results. prior = sda(x) # free downscaling, no observations analysis = sda(x, obs) # guide the diffusion toward the observations For a complete implementation, see the example in Earth2Studio , from which the code above was adapted. Figure 3. CorrDiff-COSMO downscaling with and without SDA. In this example, SDA reduces the wind-speed RMSE at held-out stations by 54%. Left: wind speed downscaled without SDA. Middle: wind speed downscaled with SDA using GHCN-Hourly station observations. Right: difference in wind speed between the SDA and no-SDA experiments How to run StormCast-SDA in Earth2Studio StormCast is a technique similar to CorrDiff but designed for high-resolution, regional forecasting. Earth2Studio includes a StormCast model pretrained over the contiguous U.S. (CONUS) that is initialized with HRRR and makes predictions at a 3-km resolution. The computation and dissemination of a new HRRR analysis takes some time, but you can use SDA to combine the currently available analysis with the latest observations to update your forecast. Start by loading the pretrained model. We limit the domain to the central U.S. import numpy as np from earth2studio.data import GHCNHourly from earth2studio.models.px import StormCastCONUS # Limit the domain to the central U.S. hrrr_lat_lim, hrrr_lon_lim = (305, 785), (595, 1203) model = StormCastCONUS.load_model( StormCastCONUS.