|About | Affiliations | Contact | FAQ | Science, Service, Society | Search | Volunteer|
|Education||Career Center||Exhibits||Meetings||Member Services||News||Policy
Enhancing Weather Information with Probability Forecasts
An Information Statement of the American Meteorological Society
Producing weather forecasts in probabilistic form for many weather parameters will require improvements in, or the implementation of, techniques for quantifying uncertainty, such as ensemble forecasting. Forecasters will need to be trained not only on how to use probabilistic information in their final forecasts, but also in the diverse requirements of those who use probability forecasts. In addition, users will require information on how to interpret and use probabilistic forecast information, needs that must be met if the communication of uncertainty in weather forecasts is to be effective.
While much progress has been made in developing methods to create probabilistic forecasts, currently only a small fraction of the elements of weather, hydrologic, and climate forecasts are expressed probabilistically. Forecasts of the probability of precipitation occurrence have been made for several decades and are well accepted, even if not always properly interpreted. More recently, in the United States the NWS2 has issued probability forecasts for a variety of weather phenomena, ranging from daily outlooks of tornado hazard and wind-speed fields in tropical storms to weekly and seasonal outlooks for temperature and precipitation.
Explanation of Probabilities
Uncertainty can be expressed in ways other than probabilistic terms, such as odds or frequencies.4 But studies by social scientists have indicated repeatedly that expressing uncertainty in qualitative terms, such as “likely,” creates unnecessary ambiguity, with one user interpreting the same term as reflecting a higher probability than would another user.
Brief explanations of key concepts related to the interpretation of probability forecasts are provided in the table below.5
Users may be familiar with the climatological probability of a weather event, for example, the probability of precipitation as estimated by the historical relative frequency that precipitation has occurred in the past at a given location for a particular time. A “conditional” probability is a modification of this climatological probability, formulated by focusing on the relative frequency with which a particular weather event is associated with a given set of conditions (e.g., an El Niño year).
One highly desirable property of any probability forecast is that it be “reliable” (or “well-calibrated”). For instance, over the long term, precipitation should occur on approximately 20% of the occasions for which the forecast probability is 20%. Unreliable probability forecasts indicate that the uncertainty has not been properly estimated, so that a user will not necessarily be able to select the most appropriate action. Given reliable probability forecasts, the potential benefits of a forecast system are directly related to the extent to which an individual forecast differs from the climatological probability (a verification measure called “sharpness”).6
The definition of the event being forecast must be clearly understood in order for probability forecasts to be communicated effectively and acted upon appropriately. For example, for a forecast of 30% probability of precipitation for Boston tomorrow, a person may be unsure as to whether that means: (a) it will rain over 30% of the Boston area tomorrow; (b) it will rain for 30% of the time tomorrow somewhere in Boston; (c) there is a 30% probability it will rain somewhere in Boston tomorrow; or (d) at any given location in the Boston area, there is a 30% probability that it will rain tomorrow. The definition of a precipitation event used by the NWS is measurable precipitation within the stated time period at any point in the area for which the forecast is valid (i.e., (d) is the correct answer).
As more weather forecasts begin to include probabilistic information, significant efforts will be required not only to effectively communicate this new information, but also to ensure that users understand the definition of the event being forecast.7 Examples of such efforts are the ongoing discourse between the NWS, emergency managers, and the media regarding new forecast products that provide tropical cyclone wind-speed probabilities and the “cone of uncertainty” for tropical cyclone tracks issued by the National Hurricane Center. The fact that users often misunderstand the cone of uncertainty illustrates that additional work at communication of the concept to the public is still needed.
Production of Probability Forecasts
Each of these approaches has certain advantages and disadvantages, with the most appropriate one depending on the particular situation. In particular, the first and second approaches are often used in combination because of the current limitations of ensemble predictions.
Ensemble forecasting methods. Ensemble forecasting methods involve evaluating a set of runs from an NWP model, or different NWP models, from the same initial time. Each of the model runs either begins from subtly different initial conditions (reflecting incompleteness and uncertainty of the present weather observations) and/or uses different model assumptions and parameters (reflecting imperfect knowledge of atmospheric processes). Each of the model runs produces a different forecast. The result is a collection (or “ensemble”) of forecasts. The differences among these forecasts reflects the uncertainty in the initial conditions and/or in model physics. Ensemble methods are now starting to be applied to a variety of different kinds of prediction problems, from short-range forecasts of thunderstorms and hurricane motion to seasonal predictions of temperature and precipitation. Because of certain deficiencies, probability forecasts based on ensembles are still not yet widely disseminated, especially without post-processing (see below).
Statistical post-processing or calibration methods. These methods are typically applied to improve forecasts, such as ensemble predictions which are presently not capable of accounting adequately for all sources of uncertainty, especially imperfections in NWP models. This process commonly makes use of information from prior forecasts and observations to produce probability forecasts or to improve their reliability. A variety of statistical and numerical methods are used to calibrate and improve the predictions, with the best approach depending on the weather element being forecast as well as a number of other operationally oriented factors. A well known technique, Model Output Statistics (MOS), produces well calibrated probability forecasts by relating past observations to NWP output.8
Observationally based statistical forecasting methods. These methods are based on relationships between current observations (predictors) and unknown future observations (predictands). They are useful at a variety of time scales including very short lead times, for which forecasts from NWP models are unavailable, and lead times of approximately two weeks and longer, for which forecasts from NWP models degrade very substantially due to their inherent inaccuracies and sensitivity to the initial conditions.
Subjective forecasting methods. These methods are based on the experience, knowledge, and judgment of human forecasters. The forecasters interpret information from current observations, NWP models, and other sources to decide on the content of the official forecast. With adequate training and feedback, human forecasters in operational settings consistently demonstrate the ability to produce skillful and reliable probability forecasts.
Probability forecasts are produced by a number of national weather services around the globe as well as the private sector. Examples of some of the types of probability forecasts that are currently produced by the NWS, and the methods used to generate them, are shown in the table below.
Benefits of Probability Forecasts
Three examples of the economics of weather-based decision making appear in the table below. The columns represent the decision maker, the weather event having an impact, the costs and benefits to compare, and the probabilistic information needed as compared to the non-probabilistic information typically provided. These examples should be viewed as prototypical, neglecting certain details that would be important in practice.
In all three of the above examples, non-probabilistic information in the form of the most likely event can be quite different from the probabilistic information required by the decision maker to adopt the best strategy. Unfortunately, studies that document the use and economic value of weather forecasts, in general, and probability forecasts, in particular, remain rather limited, with quantitative estimates of economic value only rarely being obtained.9
Challenges and Opportunities
Nevertheless, the opportunities are great. By providing users with uncertainty information communicated in an effective manner, forecasters can allow users to make better decisions resulting in greater economic and social benefits.
[This statement is considered in force until May 2013 unless superseded by a new statement issued by the AMS Council before this date]
© American Meteorological Society, 45 Beacon Street, Boston, MA 02108-3693
1Related information can be found in the AMS Information Statement on Weather Analysis and Forecasting (Bull. Amer. Met. Soc., 88, 2007); the National Academy of Sciences (NAS) Report “Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts” (NAS, 2006); and in the Glossary of Meteorology (AMS, 2000).
9 For a summary of recent case studies of the economic value of weather and climate forecasts, see http://www.isse.ucar.edu/staff/katz/esig.html.