Weather: [More data + more computers = better forecasts]

Print Friendly
Weather: [More data + more computers = better forecasts]

Been beefing about weather forecasts? Did the “experts” miss a thunderstorm, botch the rainfall prediction, mistake cloudy for sunny or windy for calm?

A tree was uproyouoted and fell upon devastated houses

The April 27, 2011 tornado left many homes in shambles and killed 64 in Tuscaloosa, Ala. The twister — one of 358 tornadoes in the largest outbreak of tornadoes in United States history — reached a width of 1.5 miles. Much of the effort in weather forecasting concerns getting faster, more accurate predictions of the giant convective cells that drive severe thunderstorms and tornadoes.

You’re not alone. Forecasts of weather are already way better than forecasts of, say, unemployment or grain harvests, but that doesn’t lead us to predict that the caterwauling over weather forecasts will dampen.

See the Classroom Activity Page for this story.

Yet the irony is that forecasts continue to improve. Although summer is a forecaster’s tar pit, “For 75 percent of the year, on Thursday morning we can say what it will be like next Tuesday: will it be like today, or warmer or cooler,” says Jonathan Martin, a professor of atmosphere and oceanic science at the University of Wisconsin-Madison (UW-Madison). “What will be the wind direction, cloud cover, precipitation? Twenty years ago, a five-day forecast was a complete pipe dream.”

In Madison, Wis., in the past two years, Martin says, “schools were cancelled, and no one regretted it, based entirely on an 18- to 24-hour forecast for snow. With two parents working, the question of what to do with the kids makes this a more momentous decision, and it was made before the snow started to fly.”

Starting roughly 20 years ago, forecasters began to forecast with numerical prediction, a computer-based technique that swallows data from multiple sources, and then spits out data. “You can make a one- or two-day forecast at almost any time of the year except summer, and it’s just about guaranteed to be an accurate depiction of what the day will be like,” Martin says. “If you don’t bet on it being right, you are a fool, and that was not true 20 years ago.”

Two men walking in a blizzard with two dogs

Even before a blizzard struck the Madison, Wis., area on Dec. 20, 2012, every school district in the county cancelled classes. A decade ago, forecasts were not accurate enough to make those decisions so far in advance.

We’ll explain the summer-weather snafu later, but we’ve certainly come a long way, baby, since the first efforts to know if you would need an umbrella or a snow shovel in a couple of days. The ancients, going back before Aristotle, tried to predict the weather with various hare-brained schemes. Some of the earliest scientific efforts, in the 1800s, rested on the recognition that an invisible factor — air pressure — was implicated in tomorrow’s weather.

Air-pressure map of Europe, December 10, 1887

a yellowed weather map of Europe with black isobars

During the late 1800s, air pressure emerged as one of the first foundations for scientific forecasting. Lines show pressure levels in millimeters of mercury, the fluid used in early barometers.

Got it on my screen

Numerical weather prediction began to stir in the 1950s, at the dawn of the computer age, and computers — inhaling an endless storm of data from balloons, ground stations, ships, airplanes and satellites — are now the engines of weather prediction.

Forecasting and issuing severe-weather warnings has three fundamental components, said National Weather Service Director Louis Uccellini during an August visit to UW-Madison, where he began his education in meteorology: making observations, squeezing out their information, and issuing forecasts or warnings. “The processing must be done in real time, so we need the computing capacity and the dissemination capacity,” he said.

As the public and businesses demand ever-greater accuracy, Uccellini said, “We are seeing prospects for major change, if not a revolution, in these areas.”

Radar, a critical part of short-term warnings, particularly for thunderstorms and tornadoes, continues to improve. Uccellini said dual-pole radar, a new upgrade to the nationwide Nexrad radar array, “adds a vertical component, so we get more fidelity in the return signal — is it rain, sleet or snow?” The result is a better picture of convection — vertical movement powered by differences in density — and severe weather.

A slide titled ‘personal plea’ has the following text: If you are being asked to evacuate a coastal location by state and local officials, please do so. If you are reluctant to evacuate, and you know someone who rode out the ’62 storm on the barrier islands, ask them if they would do it again. If you are still reluctant, think about your loved ones, think about the emergency responders who will be unable to reach you when you make the panicked phone call to be rescued, think about the rescue/recovery teams who will rescue you if you are injured or recover your remains if you do not survive. Sandy is an extremely dangerous storm. There will be major property damage, injuries are probably unavoidable, but the goal is zero fatalities. If you think the storm is over-hyped and exaggerated, please err on the side of caution. You can call me up on Friday (contact information is at the end of this briefing) and yell at me all you want. I will listen to your concerns and comments, but I will tell you in advance, I will be very happy that you are alive & well, no matter how much you yell at me. Thanks for listening. Gary Szatkowski – National Weather Service Mount Holly

National Weather Service, NOAA
Think weather forecasters don’t care? Here’s a heart-to-heart message from a forecaster just before Hurricane Sandy flooded New Jersey and New York in 2012.

Another innovation, the phased array radar developed for anti-missile systems, can speed the detection of storms likely to spawn tornadoes, Uccellini says. “With current radar, to scan the entire volume of the atmosphere and get a signal back for processing takes five minutes. With phased array, we get it back immediately, so you could theoretically gain five minutes, which is a lot of time in these situations.”

Phased array radars, Uccellini adds, have fewer moving parts, so they are easier to maintain, and they can do double duty by replacing air-traffic control radars.

“Squeezing something from nothing.” Honest?

The starting point in weather forecasting is the current status of the atmosphere — and the land and ocean below. We were surprised to learn how little is known about the air above us.

In winter, frontal systems tend to be thousands of kilometers long, and last from three to seven days, Martin says; both factors ease the winter-forecasting problem.

The forecast picture is much darker for warm-weather convective systems, which are energized by differences in temperature and moisture, and cause dangerous thunderstorms and tornadoes.

Phased array radar: Fast and furious

The Multi-Function Phased Array Radar project was designed to simultaneously track aircraft, wind and weather with a single radar. Rollover to see scans of the different levels of a tornado-producing storm.

Satellites show the big picture, but are expensive, and can have trouble seeing how conditions change with altitude. Weather balloons provide the best reading of pressure, temperature and humidity at exact altitudes, but are only released twice a day — at only 60 locations in the United States. Because convective systems “can grow and die in just two hours,” says Martin, you are lucky to see them with a balloon.

Furthermore, because forecasting models chunk the atmosphere into rectangles (typically 12-by-20 kilometers), convective systems can disappear inside the forecasting cells.

Because convective storms fall through the cracks in today’s forecasting system, it is now “impossible” to forecast them accurately 48 or 96 hours ahead, Martin says.

One way to wring new data from existing systems starts with Doppler radar, which shows the location and movement of thunderstorms, but nothing about temperature or humidity. A new technique examines the radar image in the context of observations about surrounding conditions, Martin says. “In isolation, you could associate wind speed and direction from the radar with any temperature or structure, but in the context of known temperatures and structure, you have much less wiggle room,” he says.

Radar shows a violent convective “supercell”

A radar image shows a color shape like a hook

The “hook echo” in radar reflections has been associated with tornado-producing storms since the 1950s, as shown on this Doppler radar image of an F5 tornado in Oklahoma. The bright red at the tornado location is neither rain nor hail, but pieces of cars, houses and trees hoisted thousands of feet into the sky!

To use an analogy, if you hear a huge ruckus in a closet, is it a rodeo? Not likely — galloping horses don’t fit the facts you can observe from outside the closet. Peering deeper into the Doppler data is a good example of “squeezing something from nothing,” Martin says.

Necessity: mother of invention?

Another example of repurposing technology exploits the radio signal from a GPS navigation satellite. Because the atmosphere bends and delays the signal, precise measurements reveal conditions along the path to a second satellite.

GPS gives a jump-start to weather forecasts

Signals emitted by a GPS satellite, shown in magenta lines, are bent by the Earth’s atmosphere and eventually received by a COSMIC satellite

Looking sideways at the radio signals from a global positioning satellite gives a fine-grained view of conditions over land and ocean.

Because this “radio occultation” technique looks almost horizontally, it produces a pile of data on the layered structure of the atmosphere, says Richard Anthes, director emeritus of the University Corporation for Atmospheric Research (UCAR) in Boulder, Colo. “Every time the satellite rises or sets, we get a scan of Earth’s atmosphere … that is better than any remote-sensing system, and close to weather balloons,” he says.

For a reasonable cost, the COSMIC (Constellation Observing System for Meteorology, Ionosphere and Climate) satellites can extract information on water vapor, temperature and pressure. “The system looks at the entire Earth, more or less evenly spaced, all the time, day and night, in all weather,” Anthes says.

The benefits of being COSMIC

Hurricane Ernesto clearly appears in a 66-hour forecast of cloud-water content in the eastern Caribbean in August 2006, based on standard weather data and 15 COSMIC profiles of conditions in the region. Rollover image: Remove the COSMIC data, run the model, and presto! No Ernesto!

Since they were first launched in 2006, Anthes says, the six COSMIC satellites and two European counterparts have had an impact on weather forecasts “that is typically ranked in the top five of all observing systems, in terms of accuracy. That includes 25 to 30 systems, including balloons, all surface, ship data, satellite, aircraft data.”

The COSMIC satellites are wearing out, and a dozen replacements are under development. The first six of those, with significant support from Taiwan and the U.S. government, are due for launch in 2016.

NASA used occultation in the late 1960s to measure the density of atmospheres on Mars, Jupiter and Venus, Anthes says. “It’s called occultation because the signal is occulted by the Earth. It’s not some mysterious, magical thing,” he says.

Who GOES there?

While COSMIC looks sideways, geostationary satellites stare straight down, at the same part of the globe, month after month. The GOES (Geostationary Operational Environmental Satellites) satellites carry instruments providing the basic data for many aspects of environmental monitoring and weather forecasting.

A new GOES satellite, scheduled for launch in 2015, will carry an instrument reading 16 channels of the electromagnetic spectrum, versus five in the existing GOES satellites.

Six-day video taken during Hurricane Sandy (2012) mocks up some of the detail that will be available from the new GOES-R instrument, which takes a photo every minute — but at higher resolution than is shown here. Low-level clouds feeding the storm rotate counterclockwise; higher clouds spat out by the storm rotate clockwise.

Previously, meteorologists had to focus either on the big picture or on small details, says Timothy Schmit, a research meteorologist with the National Oceanic and Atmospheric Administration at UW-Madison. The new Advanced Baseline Imager will “see” both at once, producing more detail without sacrificing the big picture.

The instrument is also quite speedy. In round numbers, existing GOES need 5 minutes to make an image that the new ABI can do in one minute. Even if the same features appear in both images, “by definition, you have to wait longer for the 15-minute images,” says Schmit. “And then, sometimes you wonder, did that feature just develop, or did it move into that position? If you are looking at the one-minute image, you can see it evolving.”

Although geostationary satellites snap amazing photos (as they prove every year during hurricane season), they have trouble measuring temperature at specific altitudes. That matters, since the energy in convective storms such as thunderstorms and hurricanes occurs when a warm, moist updraft “punches through” into the stratosphere.

Unless you can see the different levels, you will miss the first stages of a thunderstorm, explains Henry Revercomb, director of the Space Science and Engineering Center at UW-Madison. “It’s the vertical circulation that determines whether that weather is going to become severe. If there is a cap on top, it won’t break through, so bad things will not happen,” he says.

Five satellite dishes and a dome placed on the roof of a building

Form follows function on the roof of the UW-Madison Atmospheric, Oceanic and Space Sciences building. Underneath, massive amounts of satellite data are digested and researched.

Revercomb has been working for years on an improved spectrometer — an instrument that calculates temperature based on measurements of radiation from gas in the atmosphere. Newer, high-resolution spectrometers can view hundreds or thousands of narrow bands of radiation and temperature to the correct altitude.

But that spectrometer has yet to be launched. Now, researchers at Wisconsin, NASA and Utah State University are negotiating with a company that wants to build and launch the instrument, and sell data to those who need better warnings of severe weather.

A hand holds a cellphone that shows a map with markers and numbers

A user views a graph and map of recent, local measurements reported by other users via a phone app pressureNET.

“The first six hours is difficult for a numerical forecast to deal with,” Revercomb stresses. “This would provide data before the radar can see anything, when you have clear skies but the atmosphere is unstable, so you could put out a warning.”

Meteorology: the crowd-sourcing solution

Air pressure is a critical part of understanding the atmosphere, and forecasters today get readings from roughly 20,000 locations around the United States. That is not really enough, says Clifford Mass, a professor of meteorology at the University of Washington, who is starting to use smartphones equipped with a pressure sensor intended to detect altitude.

So far, the apps are delivering 10,000 pressure readings an hour, which Mass calls “a drop in the bucket.” If the app catches on, Mass says, the 20,000 sources “will fade into meaninglessness, compared to [readings from] tens of millions of smartphones.”

A suited man is giving a lecture in front of the audience.

Photo: Bill Bellon, Space Science and Engineering Center, UW-Madison
Louis Uccellini, head of the National Weather Service, discussed “the Valley of Death” and a “Weather-Ready Nation” on April 22, 2013.

Mass says pressure data could improve local forecasts concerning, say, the arrival of new weather systems, which can change wind-turbine output. Owners “need to have a very good one-to-two hour forecast so they can tell how much power they are going to generate,” he says. In the face of a rapid decline in output, alternate sources of electricity must be readied.

But could it confusing to feed a billion data points into forecast models? “More is always a good thing” in the weather business, Mass says. “We can handle it, we already have a huge amount of data coming in, just the satellite data alone is an extraordinary volume. We knew about big data before they figured out the term.”

Testing, testing

Cynics may think weather forecasters get a free ride — they get paid no matter how balky the forecast. But meteorologists do check their work, says Glen Romine, a research meteorologist at UCAR, although the testing standards vary according to the goal. “If you are the FAA [Federal Aviation Administration], you concerned with aircraft icing or wind shifts at terminals. If you are involved with storm prediction, it’s when and where thunderstorms are going to occur,” he says.

“Reforecasting” — running different systems side by side — is a standard way to test forecast models and the data they start from, Romine says. “You make one change, and if over time the forecast does better, you would put it up for consideration” for addition to the overall Weather Service forecast model, he says.

Global map show that continents and oceans are marked in a range of color from magenta, blue, green, yellow to orange.

Data from the Cross-Track Infrared Sounder (CrIS) instrument aboard Suomi National Polar-Orbiting Partnership (NPP) shows composite data from January 21, 23 and 25, 2012. Orange shows very warm sea surfaces; magenta shows cold temperatures and tops of high-altitude clouds. CrIS tracks 1,305 channels of infrared radiation, and can tell temperature and moisture content at different altitudes.

Brainstorming about storms: Crossing the “Valley of Death”

When Uccellini visited Madison, we asked him about the move from the lab to the forecasting computers, and he told us the shift “from research to operations has been driving my career since I was a student at Madison.”

Part of the problem is bureaucratic, Uccellini said: “I have been told by research managers that they will stop work if something starts getting too operational,” because an operational agency (such as the Weather Service) should be funding it.

The gap between research and operations has been called “the Valley of Death,” Uccellini said. “Even today, you find examples of good ideas in weather technology, observation, science that could take years or a decade to put into practice.”

Uccellini says the valley can be crossed in less time. “If you are coding your research and your algorithm on our model, using what we employ operationally in real time, and you discover something, I can cut that time to a year or less, not seven years,” he said.

Uccellini said one sign of acceleration is embodied in a satellite named for Verner Suomi, the UW-Madison engineer who invented the weather satellite. “Suomi NPP went up in the fall, and we were using the microwave data within six months of launch,” he said. “This was the first time we began using data from a research satellite so quickly.”

– David J. Tenenbaum


Terry Devitt, editor; S.V. Medaris, designer/illustrator; Yilang Peng, project assistant; David J. Tenenbaum, feature writer; Amy Toburen, content development executive