2005 Computerworld Honors Program

	Severe Storm Forecasting The Center for Analysis and Prediction of Storms & Pittsburgh Supercomputing Center Norman, OK USA Year: 1997 Status: Award Recipient Category: Science Nominating Company: Silicon Graphics/Cray Research
	Accurate four to six hour numerical forecasts of individual spring and winter storms help mitigate loss of property and lives, and reduce economic loss in agriculture, commercial transportation, and related areas.
	I.Introduction April 29, 1995. Around 10 p.m. a vicious thunderstorm rolls into Dallas-Ft. Worth airport. Marble to softball-sized hail pelts planes on the ground. More than 60 commercial jets must be removed from service for repair. The direct insured loss exceeds $20 million, and further loss from canceled flights over the next few weeks runs to $300 million. What if there had been four hours warning? Incoming flights could have been diverted, and with normal flight turnover, about two hours, virtually all the planes would have been removed from harm's way, cutting the airlines' loss and avoiding much of the subsequent inconvenience to travelers. With countless other severe storms as well, including the fierce tornadoes that rampage the Great Plains each spring, better forecasts could greatly reduce property damage and save lives. Since 1986, severe weather has hit the insurance industry with unprecedented high wind-related losses, and industry studies indicate that more than $14 billion a year of this loss could be avoided with better forecasts. Current forecasting gives about 30 minutes warning for storms, with fairly imprecise information about extent and severity. The Center for Analysis and Prediction of Storms (CAPS) at the University of Oklahoma has developed a new storm-prediction capability that is setting milestones in the field. In 1995, using the CRAY T3D at Pittsburgh Supercomputing Center (PSC), their state-of-the-art computer model, the Advanced Regional Prediction System (ARPS), successfully predicted the location and meteorological structure of individual storms six hours in advance, the first time anywhere this has been accomplished. In 1996, ARPS did better yet, successfully forecasting the position and timing of storms seven hours in advance, even though the storms hadn't yet begun to form when the model was run. II.Storm-Scale Weather Forecasting The weather reports we watch on TV derive from computer models at the National Centers for Environmental Prediction that predict atmospheric structure over the continental United States as often as every three hours. The local forecaster extracts from these models to show a regional map, covering perhaps several states, with predictions that weather the next day will be rainy, cloudy, sunny, etc. In contrast, Kelvin Droegemeier and his colleagues at CAPS work on storm-scale forecasting, a much tighter focus -- a few miles square in space and about 15 minutes in time -- that corresponds to the scale on which individual storms evolve. "What we're getting down to," says Droegemeier, "is to say that over Pittsburgh this afternoon from 3:30 to 3:50 there will be a thunderstorm with winds of 30 miles per hour, golfball-sized hail, two-and-a-half inches of rain, and by 3:50 it will be gone. And to give you that forecast six hours in advance." III.Progress CAPS' goal is to develop storm-scale forecasting to the point where it can be turned over to the National Weather Service early in the next century, and their progress to date proves the feasibility of their approach. Notable among the challenges they overcame since starting in 1988 is gathering the input data necessary to run a storm-scale model. CAPS innovations now make it possible to gather all the data needed -- pressure, temperature, wind speed and more -- from Doppler radar. Since 1993, CAPS has carried out experiments during Oklahoma's spring storm season to see how ARPS works in an operational setting. Initializing data each morning feeds the computer model running in Pittsburgh. The output in turn feeds back to forecasters in Norman, Okla. Early results were encouraging. In 1993, running on the CRAY C90 with limited data and a limited version of the model, forecasters used ARPS information in an official National Weather Service forecast. In spring 1995, CAPS used more ARPS capability, and for the first time exploited parallelism on Pittsburgh Supercomputing Center's CRAY T3D. This "highly parallel" computing system divides the computing among hundreds of individual processors (up to 512) so that each processor can work simultaneously on part of the job. Availability of this very advanced computing system has been crucial to ARPS success. "In meteorology," says Droegemeier, "getting results quickly is essential. If you can't predict the weather significantly faster than it evolves, the prediction has no value. If you're going to create a four-to-six hour forecast, you better do it in half an hour. Parallel computing and high-performance networking are crucial." With the CRAY T3D harnessed to ARPS, the 1995 spring forecasting experiment broke new ground in meteorology, successfully forecasting the timing, location and general structure of a June 8 storm line that extended from the northeast Texas panhandle across northwestern Oklahoma up into Kansas. "The T3D's distributed, globally addressable memory," says Droegemeier, "gave us the ability to run at storm scale over a big area at high resolution." The model did remarkably well, especially considering that initializing data for these runs lacked the resolution radar can provide. Nevertheless, the June 8 forecast showed accuracy exceeding any prior storm modeling efforts. As a result of this success, American Airlines contracted with CAPS to develop this embryonic technology as a severe storm "smoke alarm" for airports. In 1996, the spring operational test incorporated several model enhancements, including more detail at ground level, that resulted in better temperature forecasts and more precision in predicting the time when storms would develop. With these improvements, ARPS successfully forecast storms on eight of the 10 days when they occurred, an unprecedented success rate. On several of these stormy days, data that fed the model showed no convection, the earmark of storm conditions, and the model still correctly forecast that storms would develop seven hours later. Just as importantly, the model also correctly predicted no storm on a day when conditions indicated a strong likelihood. With this growing success record, CAPS researchers see their goals as within reach, although challenges remain, perhaps none bigger than the extreme computational challenge built into their work. In 1996, it took about 100 minutes to run a seven-hour forecast using 256 processors of the CRAY T3D. Fine for this stage of development, says Droegemeier, but too long for real-world forecasting. The significantly improved performance of newer computing technologies, such as the CRAY T3E installed at PSC during 1996, will further boost ARPS ability to forecast storm-scale phenomena.
	Since 1986, insurance companies have paid unprecedented, and unanticipated $60 billion in catastrophe losses. Nearly 90 percent of these losses result from wind-related peril. The insurance industry estimates that preventable loss from storms across many sectors of U.S. industry -- agriculture, aviation, construction, communications, power, manufacturing and transportation -- exceeds $14 billion a year. Current forecasting gives about 30 minutes warning of impending tornadoes or severe thunderstorms. With improved data and forecasting technology, it will be possible to provide up to six hours warning, with detailed information about the degree of severity and locations where storms will strike. This will greatly reduce financial loss, and it will save innumerable lives.
	Every facet of CAPS is dependent on high-performance computing and communications. In other areas of science and engineering, improved performance allows one to view results more quickly, or perhaps in a higher-quality form. In operational weather prediction, speed is the bottom line: If the numerical model does not run significantly faster than the weather evolves, producing a six-hour forecast in 15 minutes for example, the model output has no practical use. Because of this, ARPS requires the most advanced computing systems available. As recognition of the computer-programming efficiency and design excellence incorporated into ARPS, in 1993 it was a finalist for the prestigious Gordon Bell Prize in high-performance computing.
	The ARPS system employs highly innovative techniques for initializing the computer model with Doppler radar data. A fundamental problem that CAPS overcame since it started in 1988 was how to gather the initializing data for the model. The singular difference in their approach from other storm-modeling efforts has been to rely on Doppler radar, which gives only the wind component parallel to the radar beam and windfall intensity. From this information, CAPS has developed new techniques -- based in control theory and mathematical minimization -- to infer the other necessary data: the full three-dimensional wind field, pressure, temperature and water distribution. This is a major technical accomplishment. ARPS also makes use of a sophisticated nested-grid system for producing high resolution forecast data for localized storm-scale weather. During 1996 operational tests, for instance, two six-hour forecasts were made each day. The first used a 15 kilometer resolution horizontal grid for a 1,200 by 1,200 kilometer area centered over western Oklahoma. The second forecast used three kilometer grid spacing for a 336 by 336 kilometer area. This second fine-grid forecast area was determined from the daily severe weather target area and nested within the large-grid area. This method provides detailed forecast accuracy while efficiently exploiting the available computing resources. CAPS developed ARPS as a versatile program that can be run on all classes of computers. ARPS users, which include several foreign countries, have found that the user's guide itself represents an innovation in software documentation. At the Pittsburgh Supercomputing Center, ARPS demonstrated its versatility by first running on the CRAY C90, a traditional parallel, vector supercomputing system. When the scalable, parallel CRAY T3D system became available, CAPS adapted ARPS to exploit the capability of this system to produce even faster results.
	During the 1995 spring storm season in Oklahoma, with PSC's CRAY T3D harnessed to ARPS, CAPS broke new ground in meteorology, successfully forecasting the timing, location and general structure of a June 8 storm line that extended from the northeast Texas panhandle across northwestern Oklahoma up into Kansas. As a result of this success, American Airlines negotiated a three-year contract with CAPS, investing $1 million to test the new prediction technology as a "smoke alarm" for airports. In 1996, the spring operational test incorporated several model enhancements, including more detail at ground level, that resulted in better temperature forecasts and more precision in predicting the time when storms would develop. With these improvements, ARPS successfully forecast storms on eight of the 10 days when they occurred, an unprecedented success rate. On several of these stormy days, data that fed the model showed no convection, the earmark of storm conditions, and the model still correctly forecast that storms would develop seven hours later. Just as importantly, the model also correctly predicted no storm on a day when conditions indicated a strong likelihood. Although further improvements in computing will be necessary to realize the full potential of ARPS, these results demonstrate that it can successfully meet the goals of reliable storm forecasts four to six hours in advance.
	a.The first and most critical obstacle to overcome was, as mentioned above, the retrieval of unobserved quantities from single-Doppler radar data. Doppler radar measures only the radial (along-beam) component of the wind field in the region where precipitation occurs. A numerical prediction, however, needs the full three-dimensional wind field, as well as other variables (e.g., pressure, temperature, humidity, turbulence). Because the fine-scale observations needed for storm prediction are available only from Doppler radars (the new national NEXRAD network), methods had to be developed to retrieve, from a time series of observations from a single Doppler radar, the cross-beam and polar (vertical) wind components and other variables. This posed an enormous challenge. CAPS has developed numerous techniques to overcome this difficulty, ranging from simple methods to very complex and theoretically complete approaches based on control theory. While theoreticians were concerned about the uniquesness of the retrieved solution, experiments showed that this was unlikely to be a problem. Rather, computational efficiency is the key issue, as described above. b.The obstacle looming most prominently in storm-scale prediction is the extent to which events in the small-scale atmosphere can be predicted. It is now well-known that the evolution of nonlinear dynamical systems can, in many cases, be highly sensitive to the initial conditions (the Lorenz predictablity and chaos problems). Given that thunderstorms and other storm-scale phenomena are highly three-dimensional and last for tens of minutes or an hour, it was clear that classical predictability theory (which estimates the time needed for errors or uncertaintites in initial and boundary conditions to grow to the extent that they render a forecast indistinguishable from one chosen at random) did not apply. So what on the small-scale is predictable, and for how long? Most meteorologists felt that the small-scale structures present in events like thunderstorms would preclude any element of practical predictability. Through a series of yearly realtime operational tests of its forecast system, however, CAPS has demonstrated that storm-resolving models do indeed have "skill." Their forecasts can often pinpoint storm location to within one or two counties some six to nine hours in advance. Based on this success, as noted above (long summary), American Airlines has invested a significant amount of money to transfer this technology to is operations system in Fort Worth, Texas. All in all, CAPS success represents an important breakthrough with incalculable societal benefits.