Comments: Evaluating IMF Programs
- Alexander Swoboda, and Peter Kenen
- Published Date:
- December 2000
This is a very interesting and somewhat provocative paper. As with all of Takatoshi Ito’s papers, this one has a message; in this case, the message is twofold. First, that IMF advice to countries engulfed in the Asian crisis was flawed in some fundamental sense. And second, that a regional monetary fund for Asia, possibly modeled along the lines of the IMF, would have helped in containing the crises, presumably by offering more financing or different policy advice, or both. I will deal only with the former in these comments as the author does not really develop the Asian Monetary Fund idea in this paper.
The arguments that have been made for and against IMF policy advice are well covered, in the context of IMF-supported programs in Indonesia, Korea, and Thailand from the latter part of 1997. In the point-counterpoint approach taken in the paper, it is sometimes not very clear where the author comes out on the issues. But on balance he sides with the critics in making a case that IMF policy advice should have been different. One good indicator of the author’s position in the debate is the amount of space devoted in the paper to criticism of the IMF, as opposed to its defense. It should come as no surprise that I feel a strong case can be made for the IMF’s position and that the jury is still out on this matter.
Clearly the Asian crisis was a big international economic event, arguably the biggest since the 1930s, although the debt crisis of the 1980s would give it a run for its money. Naturally, it has received considerable attention but it should be stressed that the role of IMF advice—the title of Ito’s paper—obviously goes beyond the Asian crisis. After all, the IMF has supported hundreds of programs over the years. To judge IMF policy advice, one should try and see if this advice has been generally good, bad, or indifferent in the context of the much larger sample of programs. More evidence can be brought to bear on this particular question.
In a recent paper,1 Nadeem Haque and I surveyed a number of cross-country empirical studies that try to assess whether IMF-supported programs led to better economic performance, notably an improved balance of payments and current account balance, lower inflation, and higher growth. Of the 16 published studies included in our survey, about half were undertaken by IMF staff and the remainder by outside academics, which obviously reduces the possibility of an institutional bias in the results. Before discussing the results yielded by these various cross-country empirical studies, it is worthwhile to point out some of the methodological issues raised in the Haque-Khan survey because they apply importantly to the “event” analysis that Ito undertakes in judging the effectiveness of IMF policy advice to Asian countries.
An IMF-supported program seeks to achieve an adjustment in policies, and hence in macroeconomic outcomes. Therefore, the proper measure of the effectiveness of a program has to isolate the impact of the program on the outcomes and compare that to the alternative of what would have happened in the absence of the program. In other words, a comparison has to be made between the actual outcome due to the program with the counterfactual—that is, the macroeconomic outcome that would have resulted had there been no program. The counterfactual is the right yardstick by which to assess program performance and the standard most widely used in economics to measure the impact of government policy interventions.
But of course the counterfactual is not observed and, therefore, must be estimated. Several approaches have been suggested as to how to do this. One is the popular “before-after” approach, which compares macroeconomic performance under a program and performance prior to the program. While it is easy to calculate, the before-after method suffers the problem—and it is an overriding one—that it assumes that all other things remain equal. In reality, however, nonprogram factors do change over time and that means that the before-after estimation of program effects will typically be both biased and unsystematic over time.
To combat the problems of the before-after approach, alternative methods have been proposed in the literature. One is the “with-without” approach, which compares performance of the program country with a control group of nonprogram countries. But the with-without approach has problems as well, stemming mainly from the fact that program countries are not randomly selected. Instead, they are adversely selected because of their relatively poor performance prior to the program period. This means the results of the with-without approach will be biased. For example, if past economic difficulties signal fewer current difficulties—even in the absence of a program—then the with-without approach will overstate the positive effects of an IMF-supported program. This is a sort of “reversion-to-the mean” problem but one can correct for initial conditions and reduce the problems of nonrandom selection of countries. More recent studies that gauge the effects of programs have made good use of available statistical techniques to handle the problems associated with both the before-after and with-without approaches, and thus are able to get a better handle on the true counterfactual comparison. Their results, therefore, are more robust.
What are these results? Basically, they show that IMF-supported programs do improve the current account balance and the overall balance of payments. The results for inflation are less clear, with most studies finding that the rate of inflation falls, but the change is generally not significant. In the case of growth, the consensus seems to be that output will be depressed in the short run as the demand-reducing elements of the policy package dominate. Over time, growth does revive. These newer empirical results indicate that, on average, Fund-supported adjustment programs have been pretty effective in achieving their objectives. Admittedly, these programs were not dealing with systemic crises. But for many of the countries in the samples, the balance of payments crisis was as severe in magnitude for them as it has been for East Asian countries. Overall, it is clear from the empirical studies to date that IMF policy advice, in the context of programs, has been quite good.
In his paper, Ito focuses specifically on the elements of IMF policy advice in the context of the recent programs for Indonesia, Korea, and Thailand—the crisis cases. He performs an “event” study and concludes that the market did not “buy” these programs, as evidenced by the fact that the national currency in each case kept depreciating even after announcement of the program—the “event.” But the event analysis that the author uses is simply an application of the before-after method, and, as outlined above, suffers from serious problems. Recall that the biggest problem in the event study is that it assumes everything else is held constant. Can anyone really make this assumption, given the economic and political turmoil during the period immediately following the program announcements in these three countries? I doubt it.
When it comes to specifics, the author basically restates the now standard trilogy of criticisms of IMF policy advice in the Asian cases—fiscal policy was too tight, monetary policy was too tight, and too much emphasis was placed on structural reforms. So much has been said and written about these three questions that I can be very brief in dealing with them.
First, was fiscal policy too tight? It is now well accepted, even within the IMF, that fiscal policy was too tight at the beginning of the programs, but not because the IMF subscribes to the “Herbert Hoover” approach to fiscal policy—that is, tighten fiscal policy in a recession! It was simply the case that the IMF, like most other observers, did not accurately predict the depth of the recession. When the IMF realized what was happening to output and employment, it did ease the fiscal stance significantly.
Second, was monetary policy too tight? The answer to this is no, although the debate continues. The strategy pursued by the Asian countries was to raise short-term interest rates to achieve two related objectives: stabilize the exchange rate and prevent an inflationary cycle from setting in. Even if one takes Ito’s results at face value and accepts that overshooting of the exchange rate was not prevented by tighter monetary policy, the policy was still remarkably successful in keeping inflation in check. For example, in Korea, inflation rose from 4.4 percent in 1997 to only 7.5 percent in 1998. In 1999, the price level has been virtually flat. This outcome was repeated in Thailand as well. Inflation in 1998 was 8 percent, compared to 5.6 percent in 1997. Also, the average monthly change in prices in 1999 has been negative. Only in Indonesia did inflation jump from 6.4 percent in 1997 to 60 percent in 1998. But again, 1999 saw a significant decline with monthly inflation averaging only 0.6 percent. So monetary policy, as advised by the IMF, did do the trick on the inflation front.
Third, was there too much emphasis on structural reforms? Here I will deviate somewhat from the IMP line and say that perhaps the list of structural actions got too long and there was not enough differentiation between critical actions and measures that “it would be nice to have.” As the crisis unfolded, the markets began to look to see if anything was being done about the widely identified problems of the weak financial sectors, corporate governance, and lack of transparency. It would have been very difficult to reestablish investor confidence without doing something about these structural problems. Perhaps the IMF could have been more selective in the structural reforms agenda.
What is really missing from the paper is a description of the alternative policy package favored by the author, and the counterfactual outcome if it had been applied. We can only guess at what the outcome would have been, and it could have been worse. Take the specific example of exchange rates. Under IMF advice, exchange rates continued to fall, but would they have fallen more or less under his advice? We just don’t know. One thing that is pretty certain is that lowering interest rates in the middle of an exchange rate crisis would be bizarre. The paper really needs to spell out an alternative package that can be compared with the packages that were put in place in the three crisis countries.
There is no question that crisis prevention, rather than crisis management, is critical. The author makes a number of useful suggestions in this regard and we know them all: sound macroeconomic management, an “appropriate” exchange rate regime, an orderly sequencing of capital account liberalization, and fixing structural problems in a noncrisis environment. I completely agree with him here.
Crisis prediction, however, is another matter. Ito says it would be useful to have early warning systems to predict crises. Such models are currently in their infancy and cannot really be relied upon in making difficult judgments. The IMF in its surveillance activities has always examined the overall macroeconomic situation and is now, increasingly, also paying attention to the strength of financial institutions. While early warning models may aid in monitoring an economy and occasionally providing warning signals of emerging problems, the goal of crisis prediction may be intrinsically unattainable. On the one hand, the very success of early warning indicator models would eliminate the phenomenon they were trying to predict if policymakers promptly took appropriate corrective action. On the other hand, if policymakers did not react to such models, but markets took such models seriously, early warning indicators of crises would essentially result in self-fulfilling prophecies.
In conclusion, I think there is still much to be gained from studying the origins of and policy reactions to past crises, and Ito has certainly provided us more food for thought in this regard. But for me, what emerges clearly from the Asian experience is that, for crisis prevention, more effective surveillance over economies and the international financial system is critical. We need surveillance that gives us a better grasp of how increasingly integrated markets, flexible exchange rates, technological and communications advances, and financial innovation are changing the international and domestic policy environment. On this I think the IMF and its critics could quite easily agree.
The views expressed here are the sole responsibility of the author, and do not necessarily reflect the opinions of the International Monetary Fund.
Nadeem Ul Haque, and Mohsin S. Khan, “Do IMF-Supported Programs Work? A Survey of the Cross-Country Empirical Evidence,” IMF Working Paper 98/169, December 1998.