Weather: [More data + more computers = better forecasts]

Been beefing about weather forecasts? Did the “experts” miss a thunderstorm, botch the rainfall prediction, mistake cloudy for sunny or windy for calm?

Photo: U.S. Army Corps of Engineers

The April 27, 2011 tornado left many homes in shambles and killed 64 in Tuscaloosa, Ala. The twister — one of 358 tornadoes in the largest outbreak of tornadoes in United States history — reached a width of 1.5 miles. Much of the effort in weather forecasting concerns getting faster, more accurate predictions of the giant convective cells that drive severe thunderstorms and tornadoes.

ENLARGE

You’re not alone. Forecasts of weather are already way better than forecasts of, say, unemployment or grain harvests, but that doesn’t lead us to predict that the caterwauling over weather forecasts will dampen.

See the Classroom Activity Page for this story.

Yet the irony is that forecasts continue to improve. Although summer is a forecaster’s tar pit, “For 75 percent of the year, on Thursday morning we can say what it will be like next Tuesday: will it be like today, or warmer or cooler,” says Jonathan Martin, a professor of atmosphere and oceanic science at the University of Wisconsin-Madison (UW-Madison). “What will be the wind direction, cloud cover, precipitation? Twenty years ago, a five-day forecast was a complete pipe dream.”

In Madison, Wis., in the past two years, Martin says, “schools were cancelled, and no one regretted it, based entirely on an 18- to 24-hour forecast for snow. With two parents working, the question of what to do with the kids makes this a more momentous decision, and it was made before the snow started to fly.”

Starting roughly 20 years ago, forecasters began to forecast with numerical prediction, a computer-based technique that swallows data from multiple sources, and then spits out data. “You can make a one- or two-day forecast at almost any time of the year except summer, and it’s just about guaranteed to be an accurate depiction of what the day will be like,” Martin says. “If you don’t bet on it being right, you are a fool, and that was not true 20 years ago.”

Photo: Emily Mills

Even before a blizzard struck the Madison, Wis., area on Dec. 20, 2012, every school district in the county cancelled classes. A decade ago, forecasts were not accurate enough to make those decisions so far in advance.

ENLARGE

We’ll explain the summer-weather snafu later, but we’ve certainly come a long way, baby, since the first efforts to know if you would need an umbrella or a snow shovel in a couple of days. The ancients, going back before Aristotle, tried to predict the weather with various hare-brained schemes. Some of the earliest scientific efforts, in the 1800s, rested on the recognition that an invisible factor — air pressure — was implicated in tomorrow’s weather.

Got it on my screen

Numerical weather prediction began to stir in the 1950s, at the dawn of the computer age, and computers — inhaling an endless storm of data from balloons, ground stations, ships, airplanes and satellites — are now the engines of weather prediction.

Forecasting and issuing severe-weather warnings has three fundamental components, said National Weather Service Director Louis Uccellini during an August visit to UW-Madison, where he began his education in meteorology: making observations, squeezing out their information, and issuing forecasts or warnings. “The processing must be done in real time, so we need the computing capacity and the dissemination capacity,” he said.

As the public and businesses demand ever-greater accuracy, Uccellini said, “We are seeing prospects for major change, if not a revolution, in these areas.”

Radar, a critical part of short-term warnings, particularly for thunderstorms and tornadoes, continues to improve. Uccellini said dual-pole radar, a new upgrade to the nationwide Nexrad radar array, “adds a vertical component, so we get more fidelity in the return signal — is it rain, sleet or snow?” The result is a better picture of convection — vertical movement powered by differences in density — and severe weather.

National Weather Service, NOAA

Think weather forecasters don’t care? Here’s a heart-to-heart message from a forecaster just before Hurricane Sandy flooded New Jersey and New York in 2012.

ENLARGE

Another innovation, the phased array radar developed for anti-missile systems, can speed the detection of storms likely to spawn tornadoes, Uccellini says. “With current radar, to scan the entire volume of the atmosphere and get a signal back for processing takes five minutes. With phased array, we get it back immediately, so you could theoretically gain five minutes, which is a lot of time in these situations.”

Phased array radars, Uccellini adds, have fewer moving parts, so they are easier to maintain, and they can do double duty by replacing air-traffic control radars.

“Squeezing something from nothing.” Honest?

The starting point in weather forecasting is the current status of the atmosphere — and the land and ocean below. We were surprised to learn how little is known about the air above us.

In winter, frontal systems tend to be thousands of kilometers long, and last from three to seven days, Martin says; both factors ease the winter-forecasting problem.

The forecast picture is much darker for warm-weather convective systems, which are energized by differences in temperature and moisture, and cause dangerous thunderstorms and tornadoes.

Satellites show the big picture, but are expensive, and can have trouble seeing how conditions change with altitude. Weather balloons provide the best reading of pressure, temperature and humidity at exact altitudes, but are only released twice a day — at only 60 locations in the United States. Because convective systems “can grow and die in just two hours,” says Martin, you are lucky to see them with a balloon.

Furthermore, because forecasting models chunk the atmosphere into rectangles (typically 12-by-20 kilometers), convective systems can disappear inside the forecasting cells.

Because convective storms fall through the cracks in today’s forecasting system, it is now “impossible” to forecast them accurately 48 or 96 hours ahead, Martin says.

One way to wring new data from existing systems starts with Doppler radar, which shows the location and movement of thunderstorms, but nothing about temperature or humidity. A new technique examines the radar image in the context of observations about surrounding conditions, Martin says. “In isolation, you could associate wind speed and direction from the radar with any temperature or structure, but in the context of known temperatures and structure, you have much less wiggle room,” he says.

To use an analogy, if you hear a huge ruckus in a closet, is it a rodeo? Not likely — galloping horses don’t fit the facts you can observe from outside the closet. Peering deeper into the Doppler data is a good example of “squeezing something from nothing,” Martin says.

Necessity: mother of invention?

Another example of repurposing technology exploits the radio signal from a GPS navigation satellite. Because the atmosphere bends and delays the signal, precise measurements reveal conditions along the path to a second satellite.

Because this “radio occultation” technique looks almost horizontally, it produces a pile of data on the layered structure of the atmosphere, says Richard Anthes, director emeritus of the University Corporation for Atmospheric Research (UCAR) in Boulder, Colo. “Every time the satellite rises or sets, we get a scan of Earth’s atmosphere … that is better than any remote-sensing system, and close to weather balloons,” he says.

For a reasonable cost, the COSMIC (Constellation Observing System for Meteorology, Ionosphere and Climate) satellites can extract information on water vapor, temperature and pressure. “The system looks at the entire Earth, more or less evenly spaced, all the time, day and night, in all weather,” Anthes says.

Since they were first launched in 2006, Anthes says, the six COSMIC satellites and two European counterparts have had an impact on weather forecasts “that is typically ranked in the top five of all observing systems, in terms of accuracy. That includes 25 to 30 systems, including balloons, all surface, ship data, satellite, aircraft data.”

The COSMIC satellites are wearing out, and a dozen replacements are under development. The first six of those, with significant support from Taiwan and the U.S. government, are due for launch in 2016.

NASA used occultation in the late 1960s to measure the density of atmospheres on Mars, Jupiter and Venus, Anthes says. “It’s called occultation because the signal is occulted by the Earth. It’s not some mysterious, magical thing,” he says.

Who GOES there?

While COSMIC looks sideways, geostationary satellites stare straight down, at the same part of the globe, month after month. The GOES (Geostationary Operational Environmental Satellites) satellites carry instruments providing the basic data for many aspects of environmental monitoring and weather forecasting.

A new GOES satellite, scheduled for launch in 2015, will carry an instrument reading 16 channels of the electromagnetic spectrum, versus five in the existing GOES satellites.

Previously, meteorologists had to focus either on the big picture or on small details, says Timothy Schmit, a research meteorologist with the National Oceanic and Atmospheric Administration at UW-Madison. The new Advanced Baseline Imager will “see” both at once, producing more detail without sacrificing the big picture.

The instrument is also quite speedy. In round numbers, existing GOES need 5 minutes to make an image that the new ABI can do in one minute. Even if the same features appear in both images, “by definition, you have to wait longer for the 15-minute images,” says Schmit. “And then, sometimes you wonder, did that feature just develop, or did it move into that position? If you are looking at the one-minute image, you can see it evolving.”

Although geostationary satellites snap amazing photos (as they prove every year during hurricane season), they have trouble measuring temperature at specific altitudes. That matters, since the energy in convective storms such as thunderstorms and hurricanes occurs when a warm, moist updraft “punches through” into the stratosphere.

Unless you can see the different levels, you will miss the first stages of a thunderstorm, explains Henry Revercomb, director of the Space Science and Engineering Center at UW-Madison. “It’s the vertical circulation that determines whether that weather is going to become severe. If there is a cap on top, it won’t break through, so bad things will not happen,” he says.

Richard Hurd

Form follows function on the roof of the UW-Madison Atmospheric, Oceanic and Space Sciences building. Underneath, massive amounts of satellite data are digested and researched.

Revercomb has been working for years on an improved spectrometer — an instrument that calculates temperature based on measurements of radiation from gas in the atmosphere. Newer, high-resolution spectrometers can view hundreds or thousands of narrow bands of radiation and temperature to the correct altitude.

But that spectrometer has yet to be launched. Now, researchers at Wisconsin, NASA and Utah State University are negotiating with a company that wants to build and launch the instrument, and sell data to those who need better warnings of severe weather.

Courtesy of Laura D’Alessandro, Cumulonimbus Software

A user views a graph and map of recent, local measurements reported by other users via a phone app pressureNET.

ENLARGE

“The first six hours is difficult for a numerical forecast to deal with,” Revercomb stresses. “This would provide data before the radar can see anything, when you have clear skies but the atmosphere is unstable, so you could put out a warning.”

Meteorology: the crowd-sourcing solution

Air pressure is a critical part of understanding the atmosphere, and forecasters today get readings from roughly 20,000 locations around the United States. That is not really enough, says Clifford Mass, a professor of meteorology at the University of Washington, who is starting to use smartphones equipped with a pressure sensor intended to detect altitude.

So far, the apps are delivering 10,000 pressure readings an hour, which Mass calls “a drop in the bucket.” If the app catches on, Mass says, the 20,000 sources “will fade into meaninglessness, compared to [readings from] tens of millions of smartphones.”

Photo: Bill Bellon, Space Science and Engineering Center, UW-Madison

Louis Uccellini, head of the National Weather Service, discussed “the Valley of Death” and a “Weather-Ready Nation” on April 22, 2013.

Mass says pressure data could improve local forecasts concerning, say, the arrival of new weather systems, which can change wind-turbine output. Owners “need to have a very good one-to-two hour forecast so they can tell how much power they are going to generate,” he says. In the face of a rapid decline in output, alternate sources of electricity must be readied.

But could it confusing to feed a billion data points into forecast models? “More is always a good thing” in the weather business, Mass says. “We can handle it, we already have a huge amount of data coming in, just the satellite data alone is an extraordinary volume. We knew about big data before they figured out the term.”

Testing, testing

Cynics may think weather forecasters get a free ride — they get paid no matter how balky the forecast. But meteorologists do check their work, says Glen Romine, a research meteorologist at UCAR, although the testing standards vary according to the goal. “If you are the FAA [Federal Aviation Administration], you concerned with aircraft icing or wind shifts at terminals. If you are involved with storm prediction, it’s when and where thunderstorms are going to occur,” he says.

“Reforecasting” — running different systems side by side — is a standard way to test forecast models and the data they start from, Romine says. “You make one change, and if over time the forecast does better, you would put it up for consideration” for addition to the overall Weather Service forecast model, he says.

NASA/NOAA

Data from the Cross-Track Infrared Sounder (CrIS) instrument aboard Suomi National Polar-Orbiting Partnership (NPP) shows composite data from January 21, 23 and 25, 2012. Orange shows very warm sea surfaces; magenta shows cold temperatures and tops of high-altitude clouds. CrIS tracks 1,305 channels of infrared radiation, and can tell temperature and moisture content at different altitudes.

Brainstorming about storms: Crossing the “Valley of Death”

When Uccellini visited Madison, we asked him about the move from the lab to the forecasting computers, and he told us the shift “from research to operations has been driving my career since I was a student at Madison.”

Part of the problem is bureaucratic, Uccellini said: “I have been told by research managers that they will stop work if something starts getting too operational,” because an operational agency (such as the Weather Service) should be funding it.

The gap between research and operations has been called “the Valley of Death,” Uccellini said. “Even today, you find examples of good ideas in weather technology, observation, science that could take years or a decade to put into practice.”

Uccellini says the valley can be crossed in less time. “If you are coding your research and your algorithm on our model, using what we employ operationally in real time, and you discover something, I can cut that time to a year or less, not seven years,” he said.

Uccellini said one sign of acceleration is embodied in a satellite named for Verner Suomi, the UW-Madison engineer who invented the weather satellite. “Suomi NPP went up in the fall, and we were using the microwave data within six months of launch,” he said. “This was the first time we began using data from a research satellite so quickly.”