- International Monetary Fund
- Published Date:
- December 1999
1. This section deals with the quality, relevance, depth, and scope of Fund advice. The questions posed by the Board that are most pertinent here are the following.37
- (i) How did the Fund’s advice correspond to the short-term objectives and medium-term strategies of existing policies?
- (ii) How did this advice correspond to the analysis and advice of other domestic and international institutions? Did the Fund’s advice add value?
- (iii) Has Fund surveillance paid sufficient attention to regional surveillance, to interaction among countries, and to the external effects of policies in major countries?
- (v) Have the frequency and general focus of the Fund’s surveillance been appropriate, with hindsight? Has advice been consistent? Has advice helped foster noninflationary economic growth?
- (vii) Did the advice take into appropriate account the institutional, political, and social framework? Did it pay adequate attention to the uncertainties and political constraints that lead to “small” deviations from first-best policies?
2. In this light, we first discuss bilateral surveillance, beginning with what we were told about its general quality and consistency and following with a more detailed review of individual policy areas. Then we look at the scope and coverage of Fund surveillance. This is followed by a brief review of special considerations that might relate to surveillance of small states and a discussion of multilateral surveillance. We conclude with a brief comparative survey of other institutions’ surveillance-type activities.
3. Fund surveillance was generally regarded as being of high quality and as a process of exchange of views that many officials found stimulating. To cite, just as one example, the views of a couple of experienced European central bank officials, “the Fund consultation discussions are stimulating and always to the issues,” and “the staff talk to many people and finish up with a view that is comprehensive and independent.” However, the high praise tended to be limited to the Fund’s work in the macroeconomic area. In more microeconomic, less demand-oriented, areas such as financial sector surveillance and structural questions, views were more critical.
4. Perhaps not surprisingly, therefore, the most favorable appraisals came from those whose lines of work bore close similarities to the Fund’s—central banks and, to a lesser extent, finance ministries. To the extent that one can generalize about less positive reviews, it is fair to say that these tended to come from policymakers in areas other than demand management, and particularly from those who, for example, were not particularly taken with the prescription of cautious fiscal policies and monetary policies aiming at low inflation, combined with liberalization of product and factor markets, including removal of all capital controls, sometimes labeled the “Washington consensus.”
5. Many favorable comments stressed that Fund staff were high quality, hard working, and well prepared. In general, staff visiting a country were seen to have a clear and reasonably comprehensive understanding of the principal economic problems. However, two criticisms recurred in a large number of our interviews, especially in nonindustrial countries:
- lack of flexibility in the Fund’s analysis; and
- failure to appreciate adequately the political environment in which decisions are taken, and/or to allow for it in policy advice.
6. On the first point, while most of these critics agreed that it was useful—and intellectually stimulating—to exchange views with the Fund, they were left with the general impression that missions came into countries with a preconceived template of ideas, based on a theoretical or textbook model, housed in Washington, into which they fed country information. This was perhaps particularly true in areas outside the Fund’s traditional expertise, such as labor market or tariff policy.38 In their opinion, the resulting policy recommendations did not adequately allow for, or perhaps were not permitted to allow adequately for, differences among countries. This tended to lead to a “one-size-fits-all” approach that they thought weakened the effectiveness of the exercise. As one Latin American official put it, “they come with a preconceived image of what the issues are, and this makes it difficult to look behind the surface to the real problems.”
7. It should also be noted, however, that this apparent rigidity may still have a positive impact, especially in the core macroeconomic areas, as the following revealing quotation from an Asian official shows: “Strangely, it is worthwhile to go through the Article IV consultations that dispense this very standard, undifferentiated advice. It is like a confessional—the government sits on one side of the booth and the IMF is on the other side. Each knows what the other is going to say, but by repeating the lessons of an introductory course in macroeconomics to the sinning government, the IMF priest does help limit temptations to deviate too far from the orthodoxy.”39 This view of the Fund as a useful reality check was not uncommon.
8. The second criticism just noted related to the pressures applied by the Fund to achieve speedy implementation of “first-best” recommendations, without giving sufficient weight to political or institutional constraints. The authorities in one country were particularly upset when the mission’s concluding statement suggested that legislation, recently passed with considerable difficulty and after much debate, “should be repealed immediately.” Although use of such patronizing language seems to have been exceptional, there was, nonetheless, a perception—although not a universal one we should point out—that Fund staff did not sufficiently see it as their function to come up with policies that, while less than first-best, moved the country in the right direction and were politically and institutionally feasible.40 At the same time, it was also pointed out to us that this argument should not be used to justify insufficient movement in policy to ward off a looming economic or financial problem.
9. On forecasting, the Fund’s work was generally considered to be reasonably accurate—not exceptionally good. If there was any bias, our interlocutors thought it tended toward optimism. Private sector observers in particular thought that this reflected the fact that the Fund was not good at looking for, or incorporating, information and data from sources other than the authorities. At the same time, it should be noted that in one of our sample countries the staff resisted considerable pressure to produce a more favorable forecast, one that would be more in line with the domestic authorities’ public target.
10. Another concern that should be ventilated here is one that featured strongly in the Fund’s internal review of surveillance following the Mexican crisis—namely, the tendency of the Fund, particularly staff in area departments, to be insufficiently frank and direct in its assessment of a country’s policies or economic situation (a culture of “clientism”). Not surprisingly, opinion among our interviewees was sharply divided on this topic. Only a few government officials whom we quizzed directly on this thought that the Fund was insufficiently frank in their own country’s case (although some officials in Thailand conceded that in retrospect the Fund should, if anything, have been stronger in its language earlier). However, most if not all staff and former staff—including those in area departments—agreed that on the evidence this remained a serious problem for effective surveillance, as did a number of outsiders. In summary, it was far from clear that matters had improved much since the Mexican crisis.
11. Some blamed the incentive structure for this problem. For example, a former Fund official said that staff got ahead not by challenging or criticizing country policies, or by asking hard questions, but rather by “not rocking the boat.” A senior staff member said that staff were still far too reluctant to risk forecasting a crisis that did not in fact happen. Many staff, especially in area departments, while recognizing the criticism, put the blame on officials in member countries. They said that a mission leader who was regarded as highly critical of government policy and insufficiently diplomatic in presenting such views could expect that next time he or she would not be granted meetings with more senior officials. This would prevent them from doing their job properly. In more extreme cases, it had been known for government officials to complain to the mission leader’s superiors. Either way, this was bad for mission leaders’ careers.
Changes Over Time
12. Since opinions on the quality of Fund advice over time differed significantly, it is difficult to generalize. However, there appeared to be a fairly broad sense among interviewees that surveillance had become more difficult to do well, largely because of the capital account and financial sector issues discussed below.
13. Some current staff members and some government officials believed that the quality of surveillance had improved significantly in the last few years. They judged that it had managed to adapt its focus to ensure that it kept up on the right issues, and that the main emphasis of discussions had shifted to include more discussion of those issues. However, other officials and some former staff members we spoke to thought that the arguments of the Fund had become more routine, relying on textbook-like positions in the face of situations that were not covered in the textbooks. A view we heard was that discussions of possible different techniques in coping with policy challenges tended to end too early, with missions being less open to new approaches to dealing with problems. One interpretation was that a lowering of quality in advice had coincided with the rapid increase in Fund membership since the early 1990s. The resulting staff expansion meant that latterly staff were on average less experienced.
14. If anything, the concern in countries we visited was that there was too much consistency in Fund advice rather than too little. As mentioned, a frequent criticism related to lack of recognition of country-specific characteristics. A central banker from a nonindustrial country considered that “it would be better if the Fund took more account of the differences among regions and economies. Sometimes the policy advice is weakened because what it provides is merely a copy of advice elsewhere.” Some officials complained that the Fund was tougher on small and medium-sized countries than on, say, the United States or Germany. What we can add here is that while it may be debatable whether what the Fund says is tougher, what is less open to debate is that admonitions from the Fund probably matter more in smaller countries.
15. There was also a general feeling (particularly among staff members, but also among officials and other outsiders familiar with the institution) that the Fund devoted a disproportionate share of surveillance resources to precisely those countries that needed them least and where the impact was least, that is, the G-7 and medium-sized industrialized countries, especially in Europe. This applied to both the quantity and quality of resources—a number of staff members observed that the most prestigious area department, and in their view the best staffed at middle and junior levels, was European I (dealing primarily with Western Europe).
Specific Policy Areas
16. On monetary policy we can be very brief. While there were occasional criticisms (from within the Fund as well as outside) of Fund techniques of analysis (such as an excessive reliance on “net domestic assets” as a tool of analysis; inadequate recognition as to how the demand for money can shift as a result of financial policy reform; and an insufficiently broad appreciation of what might constitute a domestic “nominal anchor”), and while of course there was not always full agreement with the particular advice that was given, most felt that the advice and the technical assistance given on monetary issues were useful and of high quality.
17. Fiscal policy is front and center in most Fund documents, and there was general agreement that this was appropriate. The advice and, we should emphasize, the technical assistance provided are generally highly valued and highly rated. Furthermore, the Fund is seen as more sophisticated—and diplomatic—in dealing with fiscal issues now than in past decades.
Particular favorable mention was made of efforts directed at:
- making public finances more transparent;
- getting more complete public sector accounts;
- developing and clarifying the concept of a “quasi-fiscal deficit” stemming from central bank operating losses; and
- sorting out the analysis of structural and cyclical factors in government finance.
Some less positive comments were made, to the effect that
- the staff did not have a good understanding of the redistributive effects of some of the fiscal policy measures it proposed—or of the need to take such effects into account, and that
- missions tended to recommend fiscal tightening almost as a matter of principle, not distinguishing sufficiently between situations where tightening was urgent and essential, and others where it would merely be helpful.This made national authorities discount advice in the former situations.
18. Finally, mention should be made of the sharp internal Fund debate in recent years over its fiscal advice to Japan. Without venturing into the details of this debate, what it makes clear is that there is still ample room for disagreement over the potency of fiscal policy and the desirable mix of policies, even among economists who share the same broad approaches and institutional objectives. Furthermore, the Fund’s “line” on Japanese fiscal policy in recent years has shown a tendency to fluctuate. For a long period up to 1997, the main recommendation was that the Japanese authorities should not delay budgetary consolidation—though the line wavered as the Japanese economy ceased to grow for several years. When growth appeared to pick up in 1996 and early 1997, the Fund strongly supported raising the VAT rate from 3 to 5 percent. After the economy turned down later in the year, the Fund recognized this step as a mistake and subsequently advocated—in increasingly strong language—a budgetary stimulus despite the by then very large deficit.
Exchange Rate Policy
19. The analysis of exchange rate policies and appropriate exchange rate levels is bound to occupy a central role in surveillance. At the same time, what constitutes good, or even appropriate, exchange rate policy has been a question dividing the academic community and a prominent question in debates within the Fund. The institution’s present position is, by all reports, quite pragmatic. Indeed, from different sources we heard criticism that the Fund was excessively keen on exchange rate flexibility; that it was too much in favor of fixed rate systems; and that it had no clear line on the topic.
20. Most observers were happy with the pragmatism shown, since it gave countries an opportunity to select a system that they thought was attuned to their history and characteristics. Accordingly, the Fund has tended, at least in nonprogram countries, to go along with national preferences in terms of the exchange rate system adopted.41 However, it is also worth noting that it has seemed to emphasize pretty consistently, and appropriately, the need for other policies—basically fiscal, monetary, and wage policies—to be consistent with the exchange rate system chosen.
21. That being said, some of those interviewed were critical of what they saw as a lack of a consistent approach to exchange rate policy. The contrast between the high degree of flexibility recommended in the case of some Asian countries, while Russia was being encouraged to defend its nominal exchange rate peg came in for much comment. An academic observer noted that the Fund had bitterly opposed a currency board in Indonesia while effectively imposing one in Bulgaria, but in his view had not explained adequately the reasons for these different approaches.42
22. Many of the countries selected as part of our “representative sample” (but not selected for this reason) had traumatically abandoned their pegged exchange rate systems after prolonged experiences with them (Brazil, Chile, the Czech Republic, Korea, South Africa, Sweden, and Thailand). Many of those interviewed in these countries—even those who had earlier supported the pegged rate, and still thought that pegging was constructive in the right circumstances—criticized the Fund for not pressing them enough to abandon the peg at an earlier stage. The need to discuss exit strategies and alternative regimes was frequently mentioned. Several referred to a recent Fund Occasional Paper on this topic43 that was felt to be a very useful reference point.
23. Some, both in governments and academia, noted that the current conventional wisdom on exchange rate systems seemed to be shifting toward the view that floating or firmly fixed exchange rates were distinctly preferable to anything in between. This would appear to be the emerging view in the Fund as well.44
Capital Mobility and Capital Account Convertibility
24. The issue of capital mobility and capital account convertibility remains a lively topic within and outside the Fund. It has persistently divided both the academic community and government officials, becoming even more contentious since the Asian crisis.
25. Many of those we spoke to were critical of the Fund’s advice in this area. There were two main criticisms. The first was that the Fund, being more wedded in general to analysis of flows rather than stocks, simply did not take capital account issues seriously enough, and as a consequence its surveillance in this regard had been too sanguine. For example, one particularly outspoken individual, in the financial sector in Europe, thought that the principal weakness of Article IV consultations was “the lack of depth and rigor in the treatment of capital account issues.” Another comment, from a senior Asian finance official, was that “the Fund has not yet adjusted to an era when the capital account has become prominent in the diagnosis of, and remedies to, foreign exchange crises.” Some observers thought that this lack of attention to capital account developments helped to explain the Fund’s failure to foresee the Asian crisis. (We should in fairness note here that no institution, public or private, can be reckoned to have genuinely “foreseen” the Asian crisis.)
26. A second criticism was that while the Fund had been very opposed to the use of controls to limit short-term inflows, its opposition was based more on ideology than on a careful consideration of the evidence and the policy alternatives. While the Fund’s line on this has changed recently, the general perception is that the Fund lagged rather than led the general consensus on this topic.
27. In this context, the case of Chile is particularly interesting. The Chilean officials we spoke to were familiar with the general arguments in favor of freedom of capital movements, and therefore believed they understood why the Fund favored the removal of controls. However, they were frustrated by what they saw as a failure of Fund staff to appreciate why their controls were a reasonable second-best response to the problems they faced—pointing out that the Fund did not produce any empirical evidence about the costs and benefits of Chile’s control regime. Given this, they thought that the advice on the topic was neither convincing nor useful. More recently the Fund’s interpretation of the Chilean experience has become distinctly more positive; the Fund now assesses the main impact to have been a modification in the maturity of foreign debt rather than in its total volume, resulting in a helpful reduction in the vulnerabilities in the Chilean financial sector.45
28. Everyone who spoke to this topic agreed that having the assessment of financial systems and their vulnerabilities become an integral part of Article IV consultations was highly appropriate.
29. In our interviews, the Fund was often criticized for not having alerted countries to financial weaknesses and the imminence of a financial crisis. For example, it was marked down for not having warned Sweden about the weakness of its financial system toward the end of the 1980s and the possibility of a financial crisis, notwithstanding the fact that similar crises had already occurred at the time in Argentina, Chile, Finland, and Norway. It was also taxed with having missed the same kind of problems in Korea and Thailand, notwithstanding the experience of Mexico. As regards Korea, while the Capital Markets group had identified the weakness of the domestic corporate sector, and the potential implications for the banking system, the Asia and Pacific Department—and consequently its Article IV staff reports—had not focused on this area.
30. Clearly, the Fund has not performed well in spotting mounting weaknesses in financial systems before they trigger crises.46 But it is also fair to note, although this cannot be a full excuse given the Fund’s acknowledged responsibilities and store of international knowledge, that many officials in the relevant countries also allowed that they had not been aware of the importance of this issue. Some had thought that their financial systems were in good shape.
31. Looking ahead, a number of those interviewed thought that it was appropriate for the Fund to be charged with monitoring compliance with the code of good practices on monetary and financial policies that is now under discussion. But there was less consensus that the Fund should be in charge of its design. This task was best left in the hands of the Bank for International Settlements (BIS) or a committee of international financial regulatory authorities, some suggested. For one thing, rule making (regulation) is a conceptually different exercise from rule enforcement (supervision). For another, such a code would not be useful unless there was a feeling of “ownership” on the part of those who would actually have to live by it. The Fund was, in their view, unlikely to supply that “ownership” feeling.
32. In the same vein, there was also widespread agreement that while the Fund could well have a responsibility for monitoring the vulnerabilities of individual financial systems, this task could not be taken to imply a permanent in-depth analysis of all the microeconomic aspects of various financial systems. The Fund currently is short of this capability, and even though it has been trying hard to improve its skill set by recruiting new and experienced people, there is a general shortage of expertise in this area that cannot be remedied in the near term.
33. Providing technical assistance on financial issues and on bank supervision is considered vital if the new code of good practices is to be implemented effectively. Institutions such as the BIS, the World Bank, the OECD, and the Fund have supplied such assistance in the recent past, together with the help provided directly by individual central banks. However, there was a general feeling that this activity should not form part of the core activities of the Fund, since these needs could be better satisfied by other institutions. There was an underlying concern that the resources available in this area would be spread too thinly as different international institutions competed for them.
Interrelations, External Shocks, Spillover, and Contagion
34. A consistent theme in our discussions was that the Fund had failed to emphasize in its surveillance activities what many thought should be its main comparative advantages: its knowledge of the international macroeconomic environment; and its knowledge of the experiences of other countries in dealing with similar policy problems. It was very unusual for missions to ask questions such as: “Have you thought about the impact a measure implemented in another country (for example, a devaluation or an increase in interest rates) will have upon your country?” or “Have you evaluated the impact that the measures you are planning to implement will have upon other countries?” Similarly, it was unusual for an Article IV mission—unless specifically requested—to provide an analysis of how other countries in roughly similar situations had dealt with a specific policy problem, and of the pros and cons of different approaches. But, we were told, when such analyses were prepared, they were found to be helpful and of high quality.
35. Some observers considered that bilateral surveillance needed to be looked at more broadly and more preemptively in the light of changed global circumstances. There was a specific suggestion that the Fund be given a mandate to assess a country’s vulnerabilities to specific shocks, and that it should discuss with the authorities what, if any, contingency plans the authorities had if the worst did indeed occur. While country officials saw the conceptual merit of such an approach, it was not clear how willing they were themselves to engage in such open-ended discussions.
36. Nonetheless, many of those interviewed emphasized that one of the most useful aspects of surveillance is the information that they got about other countries through the consultation material that the Fund distributes to its members. In this regard, the consistent approach and format of the Fund was seen as a plus.
37. While many thought that a greater role for regional surveillance might contribute to improving the effectiveness of surveillance, as noted in the previous chapter neither in Asia nor in Latin America did we find anyone who thought that recent Fund participation in (or organization of) meetings in those regions had constituted any kind of breakthrough. It was also noted that contagion is not just a regional problem. As a senior central banker emphasized, “Russia affected Brazil, which in turn affected Hong Kong.”
Early Warning Indicators
38. There is, naturally enough, basic agreement that the development of macroeconomic and financial indicators, as part of an early warning system, would be useful for surveillance. It was considered that such indicators would be useful not only in current surveillance, but even more so if the proposed Contingent Credit Line were to be introduced. Our conversations with Fund staff, however, indicated that they were yet to be convinced that such indicators were sufficiently reliable to bear much policy weight. The Board has recently reviewed work on an early warning system (EWS) and found that “it could constitute an additional tool … [of surveillance]” and that it might, in particular “usefully supplement the current discussions on WEMD.” The Board cautioned, however, against any use of EWS in publicly available documents because of the market- sensitive information contained in the analysis and, particularly, the risk of generating self-fulfilling expectations of crises. These issues are discussed further in Box 3.1.
39. One important strand in the current debate on the international financial architecture has been whether and how to apply internationally agreed standards in various areas. The most important of these so far have been the Basle capital adequacy standards, but there has been discussion of standards in quite a few other areas, such as accounting systems, bankruptcy codes, corporate governance, as well as the code of good practices in the financial and monetary area that was referred to earlier. It was put to us that this general approach constituted a potentially major expansion of surveillance. Thus, the current system is based on giving advice on each nation’s particular macroeconomic policies and, increasingly, on other policies as well, with the hope that consistent policy advice across countries would lead to a generally better outcome. The “new philosophy” is based more upon assuring the observance of internationally agreed standards in a range of areas, with the Fund playing a—perhaps the—leading role in monitoring them. It would mark a shift toward greater emphasis on rules and sound institutional designs to accompany the more traditional macroeconomic surveillance.
40. While this development has yet to have much of an impact on surveillance, we did ask a number of our interlocutors for their views. Some argued that it was impossible to apply universal standards to countries without taking into consideration their culture, history, and structural characteristics. Others—including many staff members—were not opposed in principle, but pointed out that the monitoring of standards in a wide range of specialized areas called for a very different set of talents from that needed for macroeconomic policy analysis, and that it was far from clear that the Fund was well equipped at present for this type of surveillance, or when it would be. We return to this challenge in Chapter V.
Scope and Coverage
41. There was a consensus that Article IV consultations should continue to focus on asking basic hard questions about key macroeconomic policies—monetary and fiscal policies and their implications for the balance of payments and the exchange rate. And as the discussion above shows, it was felt that the Fund does this quite well. At the same time, the Fund has gradually incorporated new issues to be examined in surveillance. In addition to the traditional macroeconomic demand-side topics, and above and beyond its more recent addition of financial sector and capital account issues, it has gradually become involved in more microeconomic and supply-side matters such as trade liberalization, labor markets, offshore banking supervision, tax reforms, expenditure streamlining, income distribution, poverty, land reform, environment, and so forth.
42. It was generally acknowledged that the expansion of Fund surveillance into financial sector and capital account issues was inevitable. These issues were seen as increasingly intertwined with the Fund’s traditional analysis of macroeconomic and external sector issues. However, most observers drew a sharp distinction between such matters and the varied structural or microeconomic issues into which the Fund has recently been expanding. There was a fairly strong consensus—extending over government officials, present and former Fund staff, and academics—that this expansion detracted from the effectiveness of surveillance, for three reasons.
- It diluted the focus on basic macroeconomic issues, to the detriment of those issues, although they were still central to the Fund’s mandate.
- Staff did not really have much to contribute on many of the issues that they were now expected to discuss, because they had neither the training nor the experience. Views varied about the quality of the advice provided in these new areas, but in general its quality was perceived to compare unfavorably with that offered in the more traditional macroeconomic areas. It also did not measure up well against the advice given by other international institutions that specialized more on the microeconomic side, such as the World Bank or—for most industrial countries—the OECD. Even apart from the question of quality, there was a general feeling that the Fund should rely more on the microeconomic work done by institutions with more specialized expertise.
- Some interviewees simply thought that as a matter of propriety or national sovereignty, the Fund was getting into issues—for example, military outlays, income distribution—that were “none of its business,” taking into account its stated purposes and legal standing.47
43. Many interviewees felt that surveillance, insofar as it was useful, was useful because it was relatively narrow and focused. They felt that it should be kept a “limited-purpose vehicle,” with the basic function of “bouncing off ideas” in a frank exchange, and that the Fund should function as a “clearinghouse” of ideas and experiences. As put by a highly experienced former minister of finance, “the essence of surveillance is not to try to verify the quality of government, but to have positive exchanges of views with the authorities.”
Box 3.1.Early Warning Systems
Much academic interest has recently focused on the production of early warning systems (EWS) that might give early indications of looming balance-of-payments trouble based on selected economic indicators.1
In the current literature on early warning systems, two different approaches have been taken. The first is based on standard econometric estimations of qualitative response models (e.g., probit models) explaining the probability of the binary occurrence of a crisis with a set of explanatory variables. The second is a “signals approach.” in which those indicators mat typically show exceptional behavior preceding a crisis are singled out. Optimal thresholds for these variables are then estimated, so dial indicators issue crisis signals when they surpass the threshold, and these signals arc then combined into a composite crisis index. Finally, Berg and Pattillo (1998) suggest a combination of the two approaches by embedding the step-function of the signals approach in a probit model using several variables as regressors.
Variables that have typically been included in the EWS investigations include the real exchange rate, credit growth, and the ratio of M2 to reserves. Some studies have also included export growth, the government budget deficit, and the ratio of the stock of foreign direct investment to total external debt.
The essential test for the EWS, of course, is their ability to predict out-of-sample crises. That is, given the information available prior to a particular crisis, would an early warning system estimated prior to the crisis in fact have predicted it? In that connection. Berg and Pattillo (1998) test the out-of-sample properties of some of the recently proposed models during the 1997 currency crisis by excluding the recent observations from the complete data sample and then calculating the predicted crisis probabilities based on the in-sample estimations.
Here, the signals approach suggested by Kaminsky, Lizondo, and Reinhart (1998) seems to produce better out-of-sample predictions than the proposed standard probit models, but the combination of the two models as well as revised probit models, both suggested by Berg and Pattillo. also seem successful. The general performance of these models, however, still leaves much to be desired—even though some crises are indeed correctly predicted, the number of false alarm signals is high and most often outnumbers the true warnings. This current state, of course, stresses the fundamental dilemma between overlooking actual crises (“Type I errors”) and wrongly predicting crises (“Type II errors”). The mixed results were also confirmed in EWS investigations carried out for this report based on the probit approach. The out-of-sample properties of the model were tested using periods of crisis for 7 of the 12 countries chosen as case studies.
One reason for the relatively limited success of the EWS is that the implicit assumption in these estimations of identical behavioral relationships across countries and across time periods does not always hold. If this is not the case, the resulting estimations and predicted crisis probabilities may of course be misleading. For example, macroeconomic external and internal imbalances may previously have been more important, whereas variables capturing a country’s vulnerability toward sudden capital outflows have become more important in recent years.
Another problem with the EWS is that potentially important variables are missing, for example, variables capturing financial sector weaknesses and more political and institutional variables such as poor corporate governance and the general level of administrative competence. In an attempt to address the potential importance of financial sector distress, a variable capturing the occurrence of banking crises was included in the EWS analysis carried out for this report. This inclusion did indeed correctly increase the predicted crisis probability in some cases (most noteworthy, Chile in 1982 and Sweden in 1992/93), but at the cost of increasing it wrongly in other cases. More analytical work on this issue, including attempts to improve the quality and availability of data, would clearly be desirable in future Fund work on EWS.
A possible explanation for the quite low ability to predict the timing of a crisis may also be the presence of self-fulfilling attacks as a consequence of the existence of multiple equilibria for certain ranges of fundamentals. It is clear that if this is the case, EWS can only give an indication of a country’s potential vulnerability to an attack. Related to this issue, the likely presence of international contagion has so far largely been ignored in the EWS literature, which has primarily focused on country-specific indicators. It would be useful to focus more on this issue in future work.
EWS would obviously be helpful in surveillance if they could improve crisis projections as compared with the informed predictions by staff members. However, even if the EWS were neither systematically better nor worse than normal staff predictions, their objective nature could still make them a valuable contribution to surveillance. Our recommendations on this score are discussed in the text of Chapter V.
Some obvious objections to the likely success of implementing EWS, however, seem worth mentioning. First, the existence of a successful early warning system is almost a contradiction in terms, since corrective policy changes following an early warning signal might prevent the crisis and thus prove the original crisis prediction wrong. On the other hand, the publication of results from an early warning system might prove to be self-fulfilling if international investors were to use them as focal points of speculative attacks. Finally, if it is assumed that financial markets are efficient, a natural question seems to be why the private sector has not already acquired the knowledge assumed present in the EWS.
There is understandable concern, not least in the Board, that the current high number of both Type I and Type II errors could damage both countries’ and the Fund’s credibility if they were to be published. However, it is clear that the private sector will—and already does—provide this type of analysis. We see no reason why the Fund should not present the data it uses internally to evaluate vulnerabilities in WEO and ICMR. in the form of tables and graphs, without necessarily offering written analysis or conclusions.
44. One partial exception to the above range of views came from our meetings with NGO and trade union representatives, who considered it essential that the Fund understand fully the likely social and environmental ramifications of its advice, especially in the area of fiscal policy. For example, the NGO officials we spoke to were particularly concerned about the short time horizons that prevailed in the decision-making processes of governments, suggesting that good economics often called for a longer-term view. For example, they noted, environmental sustainability needed to be judged over a particularly extended horizon. They thought that such considerations were a very appropriate part of surveillance, given that under Article I of the Articles of Agreement the Fund is committed in their view to “the proper use of natural resources.”48
45. However, these representatives too recognized that the Fund had little expertise in such areas, and they preferred to see it make more use of the work done by other institutions such as the International Labor Organization (ILO), the OECD, and the World Bank, rather than producing work itself that was unlikely to be as thorough. That did not mean the Fund should not have ideas of its own—and indeed the Fund should be asking “awkward questions” in these areas—but that it should be more prepared to incorporate the analysis of others in its work.
46. In general, the views of small states were not radically different from those of others. Their representatives generally thought that surveillance had been reasonably good and that policy debates were generally useful. However, they emphasized particularly that the Fund did tend to take a “one-size-fits-all” approach to policy. Among the particular concerns raised were the following.
- There was some concern that too much time was devoted to data issues rather than policy advice. (On the other hand, some staff members felt that this data work was, simply put, an essential prerequisite for any sensible discussions of policy issues.)
- Often it was necessary to educate the staff in the particular needs of small states—such issues as why social services tend to be expensive in archipelagos, or the difficulties in introducing a VAT in countries that do not have the requisite administrative capacities.
- Some representatives of small states thought that the Fund was excessively enthusiastic about exchange rate depreciation, failing in particular to appreciate sufficiently all the implications of a devaluation in a very small open economy.
- On fiscal issues, representatives expressed the view that general statements like “strong fiscal action is needed” were not very useful, since they required specific suggestions on tax issues and where cuts could be made. (Such states can of course receive technical assistance in the fiscal area. How well this assistance links up with the regular consultations we are not in a position to judge.)
47. The Fund’s published work on multilateral surveillance is widely recognized as being of high quality. We learned of many instances where the WEO was a basic source document and building block for officials engaged in monitoring and forecasting international developments. The ICMR was also highly rated, though clearly less widely known and used. While it appealed to a narrower audience than the WEO, it was particularly appreciated by those interested and/or involved in assessing international financial developments as bringing more analytical substance to the review of issues than is typically found in other coverage available. The same seems true for WEO as well, although it has more direct competition from other public and private publications than does the ICMR.
48. At the same time, there is no doubt that the major errors in forecasts that marked, in particular, the Fund’s assessment of output trends in 1998 have diminished the reputation of the multilateral publications. As shown in recent Fund documents evaluating the Fund’s programs in the Asian countries most strongly affected by the financial crisis, the forecasts went from a sharp slowdown in the growth rate of output in five Asian countries to about 2 percent (forecast in late 1997) to more recent estimates, suggesting a decline in their aggregate 1998 output of 8 percent. The prospects for recovery in Japan in 1997 and 1998 were also evaluated far too optimistically. The magnitude of the reversal of short-term capital flows, obviously the most difficult element to assess in the macroeconomic outlook, was severely underestimated in a number of cases. Admittedly, these errors were only slightly larger than those of the so-called consensus. Nevertheless, it was put to us that the Fund, with its close contact with policymakers, privileged access to information, and substantial resources, must expect to live up to a higher standard than other forecasters.
49. We are aware that it may be unfair to attribute blame for the somewhat unsatisfactory state of the forecasting record of WEO—already documented in Artis (1996)49—only to the Research Department (RES), which is ostensibly responsible for its publication.50 Global forecasting in the Fund is built up from the country analysis supplied by the area departments—that is to say, it is more “bottom up” than “top down.” We were told of major disagreements in a number of cases in recent years, as well as earlier in the 1990s (when recessions in Japan and Germany were seriously underestimated) between area departments and the Research Department, which management seems to have resolved largely in favor of the former—with the consequence of worsening the forecast record further.51
50. Moving beyond forecasts, we also examined the Fund’s multilateral surveillance, both published and unpublished, in the run-up to the Asian crisis. We found that the Fund—in both bilateral and multilateral surveillance—largely failed to identify the vulnerabilities of the countries that subsequently found themselves at the center of the Asian financial crisis, except in the case of Thailand. In particular, it failed until rather late in the day to address a number of systemic issues. Moreover, to the extent that surveillance did identify these vulnerabilities, the tone of published Fund documents—notably WEO—was excessively bland prior to the December 1997 update of WEO/ICMR, after the crisis had erupted.
51. WEO eventually did express concern over financial vulnerabilities in emerging market countries. But this was done in language that was not sufficiently clear to raise the attention level in the countries concerned or in the Executive Board. More pointed analysis was contained in unpublished material, especially on Korea, where the analysis of the Research Department was more hard-hitting than that of the area department. But this analysis, which was unpublished (and indeed not communicated to the Board) did not lead to a sharpening of country surveillance. We offer a more detailed assessment in Box 3.2
52. Mention must also be made here of the concept of equilibrium exchange rates. The Fund is charged above all with “exercising firm surveillance over the exchange-rate policies of members.”52 This puts a premium on using the best possible methods for assessing departures from sustainable exchange rates. Within the Fund, work in this area takes place primarily through the Coordinating Group on Exchange Rates (CGER), a working group established in 1995 by PDR and RES. Recent work by the CGER has been presented in an Occasional Paper53 and some use is made of the analysis in bilateral surveillance of the major industrial countries, in WEO, and in the Economic Counsellor’s presentations to the WEMD sessions (for a discussion of these sessions, see below).54
Box 3.2.Surveillance, Capital Flows, and Financial Crises
We looked at how Fund multilateral surveillance, especially as expressed through the Fund’s two main publications, the World Economic Outlook (WEO) and the International Capital Markets report (ICMR). dealt with the issue of capital flows and financial crises since 1993; and at bilateral surveillance of Korea in the same period. Three main impressions emerge: the first two suggest an underestimation of the risks involved, while the third makes it difficult to avoid the conclusion that the risk of contagion was barely perceived prior to the crisis.
First, there was generally a positive evaluation of the sustainability of sizable inflows to emerging markets. Net capital inflows were projected to remain at high levels or even to increase. This evaluation was linked to the general euphoria surrounding growth prospects in emerging markets. International capital markets were perceived as well-functioning with the risk of reversals largely linked to the occurrence of excessively expansionary policies resulting in domestic overheating and external imbalances. Despite the Mexican experience, an optimistic perspective on the beneficial effects of capital inflows soon reemerged.
Second, when doubts about the sustainability of capital flows were voiced-—more frequently in the ICMR than in the WEO—they did not put much emphasis on weakening financial systems in the capital-importing countries during the process of simultaneous liberalization of the domestic financial sector and of capital flows (both with the strong encouragement of the Fund).1 As a corollary of this relative lack of priority to financial fragility, the interrelationships of the latter and balance of payments crises did not figure prominently in policy advice, particularly in the early period. This is surprising, since economists at the Fund have contributed significantly to the development of analytical insight into the area of the “twin crises.”
Third, the importance of potential regional and international contagion of currency crises was given very little attention prior to the crisis, even in internal analyses. For example, after a mission from the Capital Markets group visited Asia in April 1997, an incisive memo to management warned accurately of the dangers of crises in Thailand and Korea, resulting, respectively, from an attack on the currency and from deepening problems in the banking sector. However, the same memo explicitly dismissed the risk of regional contagion. Indeed, even in late August 1997, well after the floating of the baht, a(broadly upbeat(memorandum to the Board on the risks of contagion in Asia did not even mention Korea.
The overall impression of Fund multilateral surveillance as expressed through the WEO and the ICMR is that while these documents did make a number of pertinent observations on capital flows and financial crises that are helpful in understanding subsequent developments in Asia and elsewhere, the risks were not forcefully presented. Nor did the documents draw the appropriate conclusions and present them in pointed, country-specific policy advice.2
Inadequate as the attention to risks appears to have been in multilateral surveillance, the evaluation of crisis signals was—with the benefit of hindsight—further divorced from reality in some cases of bilateral surveillance. We have looked at Korea as a prime case study.
In bilateral surveillance of Korea, the potential dangers of a rapid process of capital account and domestic financial liberalization were not properly assessed. In particular, little attention was paid to the vulnerability of Korean banks to a potential weakening of the won. Furthermore, it was only in early 1997 that critical views were first expressed as to the credibility of official estimates of nonperforming loans. More generally, overoptimistic forecasts of output growth in Korea—even in September 1997, the staff’s judgment was that the authorities would weather the gathering storm and that a worst-case scenario was a deceleration of output growth to 4 percent for the year—delayed a realistic evaluation of financial weakness.
As regards exchange rate policies. Fund recommendations advocated a more flexible regime, clearly on the assumption that the currency would appreciate under the pressure of continuing sizable net inflows. This analysis was based strictly on traditional macro-economic fundamentals without reference to micro-economic weaknesses in the financial and corporate sectors—even though such weaknesses had been identified by the capital markets mission referred to above.
Accordingly, almost up to the outbreak of crisis in November 1997, a low probability was attached to the occurrence of an external crisis. The real effective exchange rate measures did not show signs of overvaluation, the current account deficit had narrowed, available external debt indicators were not alarming, and remaining capital controls prevented foreigners from shorting the won. The recognition that domestic events, notably a banking crisis, could trigger pressures on the exchange rate, was not totally absent, but it was certainly not given much weight.
53. It is notable that while this work is both central to the Fund’s mandate, and of high analytical quality, relatively little is published. Some observers suggested that regular publication of Fund estimates of equilibrium exchange rates would be useful to both policymakers and market participants.
54. Although much less known than WEO and the ICMR, the WEMD sessions also constitute an important element within the process of multilateral surveillance. In these sessions, Executive Board members, selected staff members, and management engage in relatively open and informal discussions of issues. These can range from the most recent developments in the international monetary system to an assessment of vulnerabilities in different countries around the world. Many participants have rated these informal meetings among the most interesting and important of Board meetings, and those eligible to attend are keen to do so.
55. Unfortunately, since the main goal of these sessions is to generate open and free discussions among participants, neither the background documents nor the thrust of the discussions are known much beyond the meeting room—except to the extent that they are relayed back to the home countries. We assume that these discussions do exert some influence upon what happens on other occasions in the institution, at least by ensuring that Executive Directors are well informed about the latest relevant developments.
56. The Fund is also charged with preparing background material for the Ministerial and Deputies’ G-7 meetings. These notes are not openly distributed but they do receive attention at a high level prior to G-7 meetings. The Fund devotes considerable attention to this material, and some participants consider it one of the more important Fund instruments of multilateral surveillance. However, the material serves as useful background to the G-7 meetings rather than as a policy agenda.
Other Sources of Surveillance
57. There are a number of other organizations whose functions overlap with the Fund’s surveillance role, although not all of them describe their functions as surveillance (in addition to regional surveillance institutions, which were discussed in an earlier chapter). We should first note that with just a few exceptions, there was a consensus among our interlocutors that, in the macroeconomic sphere, Fund surveillance taken as a whole, both published and unpublished (essentially Article IV consultation reports), compared favorably in terms of quality, focus, and coverage.
58. The “country examinations” of the OECD, representing presently 29 countries (overwhelmingly advanced economies), is perhaps the closest analogue to the Article IV process, and this was the organization to which the Fund was most often compared by interviewees. The OECD staff produces “country reports” (on a roughly biennial cycle) on each of its members, which are then discussed by country representatives. The key differences from the Article IV process are the following.
- The reports focus more on structural issues and less on macroeconomic conditions. As a consequence, they are rather longer and more detailed.
- The examination takes the form of an discussion, in which high officials from one or two other countries, on the basis of the country report, examine (candidly) the country’s policies; the country representatives then present a defense.
- The country report is published, with the consent and after negotiation in a drafting session with the authorities of the country concerned. However, the record of the discussion is not published.
59. The OECD also has a multilateral surveillance role: it produces a semiannual Economic Outlook, which covers some similar ground to the WEO, and it hosts Working Party 3, at which senior officials from member countries discuss current global economic developments in quarterly sessions. Given the level of representation, this forum, in the view of experienced officials, tends to generate more interesting and lively exchanges than are possible in the Executive Board. No report on those discussions is issued.
60. In some of the larger countries, the OECD (which, as just noted, publishes its regular country reports and also issues a press release alongside) has been generally more in the public eye, while the IMF has had a lower profile. Most of those who spoke to these matters thought that both the Fund and the OECD did a good job in their respective surveillance processes, and that these processes were at least in part complementary. While most considered that the Fund’s output was more useful overall, as noted earlier there was general agreement that the OECD did a better job in analyzing structural issues.
61. The World Bank, in countries where it is a major lender, occasionally produces a “Country Economic Memorandum,” which gives a comprehensive overview—both macroeconomic and structural—of the country’s economic situation. The World Bank also produces “Country Assistance Strategies” for borrowing countries. While these are not specifically designed as instruments of macroeconomic surveillance, lending operations clearly need to be seen in the context of the general macroeconomic situation. This is particularly the case for “structural adjustment” lending, which has become an increasingly important part of the Bank’s operations in recent years. However, the general feeling was that it did not really engage in surveillance as such, at least not in the way practiced by the Fund, or by the OECD for that matter. Its focus appeared to our interlocutors to be much more on sectoral issues.
62. The BIS has a largely informal, but increasingly important, role in financial sector surveillance. As well as keeping an eye on industrial country situations, it monitors emerging markets and coordinates a network in developed countries of central bank economists who share information about developing countries. This process provides input to meetings of central bank governors hosted by the BIS—in particular when BIS management feels that a country presents a potential problem. The BIS will also provide the secretariat for the new Financial Stability Forum, which will effectively formalize these arrangements and also involve nonbank regulators (securities markets, accounting standards), as well as other international financial institutions, including the Fund. It also provides technical assistance, and is expanding its activities through a new Financial Stability Institute and a Hong Kong office.
63. In the private sector, rating agencies (and to a lesser extent investment banks) perform a surveillance function for the benefit of private investors. The methodology used by rating agencies is not dissimilar to that of the Fund, involving analysis of much the same data and a country visit to talk to the authorities. As a result, reports produced by rating agencies often cover similar ground to that of an Article IV consultation, albeit usually in less depth. The Institute of International Finance (IIF), an association of financial institutions, produces “Country Reports” that are closer both in form and function to Article IV staff reports.
64. The main difference is that private sector institutions generally take a much narrower view, guided by the changing thrust of market interest. Rating agencies, for instance, are concerned above all with the issue of whether a country would be able to service its debts, not with the general quality of its overall macroeconomic policies. This point was also emphasized to us by the rating agencies themselves, who made it very clear that they did not regard themselves in any sense as substitutes for the Fund.
65. However, it is also apparent that private sector institutions do a more explicit job of the assessment of risks, including political risks. While it is quite rare for the Fund explicitly to examine vulnerabilities, or to look at adverse-case scenarios, private agencies regard this as fundamental. More than one observer expressed the belief that the private sector was able to produce more up-to-date and precise information than the Fund for countries that have the market’s attention, with the added advantage of not having to be as polite to governments. At the same time, it was also noted by some that information put out by private financial institutions was in principle less trustworthy, inasmuch as it could reflect the financial interest of the institution. This would not, of course, be true of the IIF, which does surveillance work for private financial institutions in general. Its surveys were thought to be more narrowly focused than those of the Fund, and in any event, they do not have the same country coverage. Those who saw both the IIF country studies and the Fund staff reports considered that the IIF material was far from being a substitute for that of the Fund.