III Issues in the Design of Early Warning Systems
- Catherine Pattillo, Andrew Berg, Gian Milesi-Ferretti, and Eduardo Borensztein
- Published Date:
- January 2000
Much of the empirical work on currency crises has been aimed at characterizing the stylized facts in periods leading up to crises (event studies) or testing particular models of crises. Event studies, which assess whether the behavior of particular variables is discernibly different in the months before a crisis from average behavior during tranquil periods,13 are systematic methods for the important first stages of “looking at the data.” A prominent example is International Monetary Fund (1998), which studies currency crises in 50 advanced and emerging market countries. Many other studies test different theoretical models of currency crisis, with either multiple- or single-country data. Early warning systems, in contrast, are not concerned with explanations of specific crises or tests of particular theories; instead, they focus on finding the best methods that can be used to forecast crisis probabilities. This study concentrates exclusively on cross-country analyses with focus on developing countries.
An early warning system consists of a precise definition of crisis and a mechanism for generating predictions, including a set of variables that may help predict crises and a systematic method to obtain a prediction from those variables. Different models have followed different approaches to address a number of conceptual and practical issues that arise concerning both the definition of crisis and the design of the method to predict a crisis. The most important issues concern the definition of crisis, the methodology to apply, and the choice of variables to serve as predictors.
What Are We Trying to Predict?
In the theoretical work discussed in Section II, currency crises are typically characterized by sudden attacks on a pegged exchange rate regime. A “failed” attack may result in reserve losses or higher interest rates but no devaluation, while a successful attack results in a step devaluation or a flotation and depreciation, perhaps after reserve losses and/or interest rate increases. In practice, exchange rate systems subject to speculative attack include not only pegs but also more-or-less managed floating exchange rates before the crisis. For example, a steep depreciation of a floating currency may also constitute a crisis. The need to systematically distinguish crises from other movements in exchange rates and reserves implies that translating the concept of speculative attacks into an empirical definition of crises is not straightforward.
Models that attempt to predict only successful attacks define a currency crisis as a sufficiently large change in the nominal or real exchange rate over a short period of time. For example, in their 1996 study, Frankel and Rose defined a currency crisis as a change in the nominal exchange rate of over 25 percent in one year, using annual average data. Some approaches attempt to predict speculative attacks, rather than only currency crises. That is, they target failed as well as successful attacks. For example, Kaminsky, Lizondo, and Reinhart (1998) combine information on reserve changes and exchange rate changes into a crisis index.14 Models that also predict failed attacks are more useful to policymakers since they are interested in anticipating any speculative attacks, which, of course, ex ante cannot be known as failures or successes. Whether or not a devaluation occurs depends on the resolve of the authorities and the measures they take.
A difficulty with most empirical definitions of crisis arises from the need to distinguish currency crises from nominal devaluations associated with high inflation. Frankel and Rose (1996) restricted the definition of crisis to include only cases when the nominal devaluation was at least 10 percent higher than the previous year. This definition does not work very well in cases of high inflation; for example, it would consider a country to have a crisis if it had a constant annual inflation rate of 80 percent and a nominal depreciation of 74 percent one year and 85 percent the next. Kaminsky, Lizondo, and Reinhart (1998) follow an alternative approach that basically takes different benchmarks to define a crisis in periods of very high inflation (observations in which the country had a six-month inflation rate above 150 percent).
The studies this paper analyzes combine different elements: a selection of countries, a methodology for the estimation, and a definition of crisis. We try to make the selection of countries homogeneous across studies, but we maintain the “original” definition of crisis (25 percent depreciation for Frankel-Rose, and so on) so as to follow the original study as closely as possible. As mentioned above with reference to the Frankel-Rose study, focusing on a smaller set of more homogeneous developing countries does affect the results.15 In addition, it should be highlighted that the definition of crisis is not uncontroversial: as already mentioned earlier, the Frankel-Rose definition considers high inflation/depreciation episodes as currency crises and the Kaminsky-Reinhart method defines low inflation episodes as those with six-month inflation below 150 percent, implying that only hyperinflationary episodes get classified as high inflation ones. Of course, other approaches that follow the same methodology but change the definition of crisis are also possible. For example, MilesiFerretti and Razin (1998) consider alternative definitions of crises within the Frankel-Rose framework, focusing in particular on depreciation episodes that follow periods of relative exchange rate stability. While the overlap between the different definitions of crises is still fairly substantial, some of the results actually change.16 Esquivel and Larrain (1998) also use panel techniques with annual data to explore the determinants of currency crises, but they define a crisis based on higher-frequency (that is, monthly) changes in the real exchange rate (so as not to give undue weight to high-inflation episodes).17
How to Generate Predictions
Once a set of crises has been identified, the question arises what methodology to use to predict the crises. One can divide the possible frameworks into three groups. The first approach is to analyze a particular crisis episode or set of crises that occur together in time. Sachs, Tornell, and Velasco (1999a), for example, analyze the incidence of currency crisis across a group of countries after the Mexican crisis as a function of a variety of precrisis factors. This approach cannot hope to shed light on the timing of crises. Rather, it may answer the question of which countries are most likely to suffer serious attacks in the event of an unfavorable change in the global environment. The justification for this approach is twofold. First, the timing of a crisis may be much harder to predict than its incidence across a group of countries. Knowing which country is most vulnerable in the event of a worldwide shock could still be useful information. Second, by focusing on a set of crises occurring at one particular time, the model avoids the problem posed by the possible changes in the determinants of crisis episodes over time. However, although this type of model may help to identify more vulnerable countries, its specification limits its usefulness for predicting future crises.18
The remaining two approaches examine data on a sample of countries through time (that is, a “panel” of data). The “indicators” approach of Kaminsky, Lizondo, and Reinhart (1998) considers a number of indicator variables (such as the degree of real exchange rate overvaluation) and calculates threshold values such that the indicator issues a “signal” of forthcoming crisis when its value is above this threshold. The third method, exemplified by Frankel and Rose (1996), uses a regression in which the dependent variable takes a value of unity when there is a crisis and zero otherwise (called a probit regression). The independent variables are the various potential predictors suggested by economic theory.19
There are important additional design issues for methods such as the indicators and probit regression approaches that attempt to predict both the timing and the cross-country incidence of crises. First, a choice must be made regarding how far in advance the prediction is to be made. Kaminsky, Lizondo, and Reinhart (1998), for example, attempt not to predict the exact timing of the crisis but rather the likelihood that a crisis will occur some time in the next 24 months. Although this relatively wide time window could be considered a drawback of the method, it may be a realistic approach, particularly in light of the discussion in Section II that suggested that the timing of crises may not be predictable at all in conditions of multiple equilibria and self-fulfilling attacks.
A further question involves what set of historical crises to use in calibrating the model. The inclusion of more years and more countries would in principle allow more precise estimation of the relationship between the various predictive variables and subsequent crises. However, the crises used to estimate the model must be reasonably similar to the crises the model is trying to predict. Different approaches can be distinguished along this dimension. The Sachs, Tornell, and Velasco (1996a) cross-section approach represents one extreme, in which no historical information is used, with the model being fit to only one set of crises occurring in a fairly small sample of similar countries at one point in time. At the other extreme are the approaches of Kaminsky, Lizondo, and Reinhart (1998) and Frankel and Rose (1996). Both use data from as far back as 1970. Frankel and Rose, in addition, use data from as many countries as possible, over 100. Evidence suggests that this sample is too large and diverse, in that the impact of various determinants on the probability of crises in a smaller set of more homogeneous developing countries can be more reliably estimated.20
What Variables to Include and How to Measure Them?
The discussion of the nature of currency crises in Section II suggests a large number of variables that might help predict currency crises. Indeed, a variety of predictive variables have been considered for inclusion in early warning systems. These variables can be classified into several groups. First, measures of the exchange rate itself, typically assessing the over-valuation of the real exchange rate compared with a trend or its long-term average. Second, various measures of macroeconomic imbalances, such as fiscal deficits and output growth, in the spirit of the “first-generation” crisis models discussed above. Third, variables designed to capture unsustainable external positions, such as reserve adequacy measures, external debt, and the size of the current account deficit. Fourth, problems in the domestic financial sectors; given the difficulties in obtaining measures in a consistent way across countries and time, rather crude measures have been applied such as growth rates and levels of domestic credit as indicators of overleverage. Fifth, indicators that reflect market expectations, such as interest rate differentials or the forward exchange rate.21 Sixth, financial market contagion variables, such as the number of crises in recent months in other countries.22
For the purpose of estimation of an early warning system, a given variable must be reasonably comparable across time and countries. Many factors that experience suggests might help predict crises are not easily measured and do not meet that standard. Perhaps the most glaring examples involve data regarding the health of financial systems, such as rates of nonperforming loans and capital adequacy. Similarly, variables may be badly mismeasured for various reasons. Accurate and comprehensive information on short-term external debt of the private sector, for example, is not available for most countries. These measurement or availability problems imply that it is difficult to incorporate this information into early warning systems that are calibrated using historical episodes.