Chapter

Evaluation Summary

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
August 2006
Share
  • ShareShare
Show Summary Details

Highlights of the IEO Evaluation of Multilateral Surveillance

Surveillance is a core function of the IMF, a critical element of its toolkit to promote global financial stability. Multilateral surveillance brings into analysis economic linkages and policy spillovers between countries, as well as international economic and market developments. It complements bilateral surveillance by adding global and cross-country perspectives to the analysis of developments in individual countries. And by exploring options to deal with policy spillovers in a global context, it can enhance the policy advice that the IMF gives to its members.

The IMF is not alone in analyzing the world economy; a number of other government bodies and private entities do so as well. What is special about the IMF is that its near-universal membership gives it a uniquely broad and in-depth perspective. Moreover, all IMF member governments have committed themselves to be part of a system of peer review and oversight. To be influential in this environment, IMF multilateral surveillance must bring to bear analytical rigor, clear policy prescriptions, and an active engagement with senior national policymakers and relevant international forums.

Owing to the substantial cost to the IMF of multilateral surveillance, and the key role that the IMF plays in ensuring a more stable world economy, the IEO conducted an evaluation of the IMF’s multilateral surveillance activities for the period 2000–05. Its evaluation report commends the analytical quality of the individual components of multilateral surveillance but sees considerable scope for better coordinating these components. The IEO report offers several suggestions for improving the effectiveness of multilateral surveillance.

Multilateral Surveillance: Less Than the Sum of Its Parts

Multilateral surveillance is carried out by several IMF departments and interdepartmental committees. IMF staff uses many instruments to analyze economic developments, and it conveys the resulting policy messages in a wide range of outputs or products (see box).

The IEO evaluation addresses a number of questions about multilateral surveillance:

  • Do the multilateral surveillance issues analyzed correspond to the IMF’s unique perspective with respect to economic linkages between countries?

  • Are the issues examined relevant and timely?

  • Does multilateral surveillance enhance the policy advice provided by bilateral surveillance?

  • How well are macroeconomic and capital market surveillance combined in the analysis of relevant issues?

  • Do the messages of multilateral surveillance reach the intended audience?

  • Are the messages being presented in a way that maximizes their impact?

The Main Outputs of IMF Multilateral Surveillance

IMF multilateral surveillance is disseminated to various audiences through a number of outputs, including:

  • Semiannual “flagship” publications: the World Economic Outlook and the Global Financial Stability Report;

  • Semiannual regional outlooks produced and published by four IMF area departments; and

  • Regular contributions to intergovernmental forums and committees, such as the Group of Seven (G-7), the Group of Twenty (G-20), and the Financial Stability Forum.

In addition, IMF staff prepares several documents for internal use as well as for IMF management and the Executive Board.

To answer these questions, the IEO reviewed public and internal documents drafted between 2000 and 2005. It also surveyed or interviewed IMF staff, officials from several governments and international organizations, market participants, and academics at universities and research institutes.

Most of the components of multilateral surveillance products were found to be well crafted and to feature high-quality analysis; they were also useful to particular audiences and for satisfying particular needs. The World Economic Outlook (WEO) is especially well regarded. The evaluation report, however, cites the lack of a clear and comprehensive strategy to guide the integration of the components and the delivery of outputs. Because of this, multilateral surveillance is falling short of its full potential.

The absence of an overall strategy has meant that IMF multilateral surveillance as a whole is less than the sum of its parts. Outputs give too much weight to descriptive information on economic developments and prospects, for which the IMF is increasingly only one of many sources. They give too little weight to analyzing economic policy linkages—in which the IMF has a comparative advantage—and to proactively identifying scope for collective, international action. In addition, the current setup for involving the Executive Board in multilateral surveillance limits the Board’s contribution.

The IMF’s failure to clarify the operational goals of multilateral surveillance and to define the mechanisms to best meet them has resulted in:

  • A predominantly “bottom-up,” or country-based, approach to policy advice;

  • A “silo” structure, in which different IMF departments produce different outputs without adequate coordination. This hinders the fuller integration of macroeconomic and capital market approaches;

  • A proliferation of publications that lack focus; and

  • An insufficiently proactive engagement with various high-level groups of national policymakers.

Content and Quality

Selecting issues for analysis. The products of multilateral surveillance—especially the WEO—have been largely successful in selecting for analysis issues that reflect the IMF’s comparative advantage. The WEO gives roughly equal weight to issues that deal with the spillovers of policies in individual countries and to analyzing and comparing the experience of different countries. Dedicated analysis of exchange rate issues and related spillover effects, however, does not appear frequently—which is surprising given the IMF’s mandate to oversee the international monetary system and the exchange rate policies of its member countries.

Identifying relevant issues and global risks. The WEO has also succeeded in identifying in a timely way relevant issues for analysis, as measured against the issues subsequently picked up on G-7 and G-20 agendas. In terms of identifying relevant global macroeconomic and financial risks, both the WEO and the Global Financial Stability Report (GFSR)—which appear semiannually—also compare favorably with similar publications of other international and national bodies. This assessment, however, is based on evidence gathered during the relatively calm period of 2000–05, when no major crisis tested the IMF’s “early warning” mechanisms.

Integrating multilateral and bilateral surveillance. The IEO evaluation confirms the finding of a 1999 external evaluation of surveillance1 that IMF surveillance has a strong bilateral (or country) orientation. As a result, policy advice and economic forecasts predominantly reflect the views of IMF area departments (the departments—grouped by geographic region—that carry out bilateral surveillance). Because of this country orientation, multilateral surveillance has not sufficiently examined options to deal with policy spillovers in a global context. Moreover, the language of multilateral advice is no more based on explicit consideration of economic linkages and policy spillovers than that of bilateral advice. In addition, the dominant bottom-up (or individual country) approach also tends to yield consistently optimistic macroeconomic forecasts for certain regions.

Integrating macroeconomic and capital market analysis. The insufficient integration between the WEO and the GFSR noted in the IEO evaluation report is also emphasized in the recent McDonough Report.2 The evaluation identifies areas where integration could have been desirable but did not take place, largely owing to the “silo” structure of multilateral surveillance.

Use and Delivery

Use of the WEO and GFSR in bilateral surveillance. The evaluation finds that most area department economists do not make much use of the WEO in their country work (other than the WEO’s quantitative forecasts of major economic variables). Only 14 percent of IMF senior staff surveyed said that WEO topics were discussed with national authorities. Substantial scope thus exists for IMF staff to increase its use of multilateral surveillance outputs in its bilateral surveillance.

Similarly, few area department economists (4 percent) use the GFSR on a regular basis in their country work. Although the GFSR has raised some important longer-term issues, it has not in practice added value to bilateral surveillance beyond the information already available in the markets. And it has not adequately distilled the implications of market developments for the IMF’s day-to-day country work.

Presenting the message. Given the variety of tasks assigned to multilateral surveillance products, the reports have become too long and unfocused. In the case of the main surveillance chapter of the WEO (Chapter I), for example, each component of the analysis may be useful to a particular audience or meet a particular need, but the effort to meet all the varying demands has expanded the chapter unduly. Indeed, the full WEO document could benefit from considerable streamlining to highlight its critical messages. As to surveillance notes, which the IMF prepares as input for G-7 and G-20 meetings, these should concentrate on spelling out the consequences of policy spillovers and the options for addressing them.

Reaching the intended audience. Attempts to reach multiple audiences through the same publications have complicated the task of communicating the WEO and GFSR messages. The wide press coverage enjoyed, particularly by the WEO, indicates that IMF multilateral surveillance messages have a significant potential for influencing public debate. Yet, the IEO evaluation team was struck by the low readership (both internally and externally) of the full WEO report, with most readers relying only on summaries. This underscores the need for the products of IMF multilateral surveillance to have a more explicit “customer” focus, with a range of well-communicated products aimed at meeting the diverse needs of various IMF audiences.

Exploiting the potential for peer pressure. The potential for multilateral surveillance to exert peer pressure on individual country policies is not fully realized. First, the IMF is not proactively engaged with the G-7, the G-20, or in other forums to which it has unique access. Second, the current structure for involving the Executive Board limits the contributions that it, and the International Monetary and Financial Committee, can make to multilateral surveillance.

Evidence of Effectiveness

The ultimate test of the effectiveness of the IMF’s multilateral surveillance is its impact on policies in member countries. Are such policies ever modified as a result of IMF advice about linkages and spillover effects, or as a result of discussions or peer pressure in international forums to which the IMF provides analysis and advice? Answering this question is difficult for several reasons, including the multiple factors that influence a country’s policy choices. The evaluation report identifies a few specific instances where policy debate or policymaking was influenced by IMF multilateral surveillance:

  • Risk transfer to household balance sheets. The IMF introduced this issue at a meeting of the Financial Stability Forum. Several officials told the IEO evaluation team that the IMF material influenced their preparation for the meeting.

  • Global imbalances and oil prices. Officials in several countries cited recent IMF analyses on these issues as having informed internal debate in their countries. Policymakers said that the IMF work was for the most part timely and valuable in developing their own thinking.

  • House prices. A WEO analysis of house prices spurred debate in at least one large European country about house price rises across Europe.

  • Foreign direct investment in financial services. One G-7 official said that an IMF staff member contributed a great deal of detailed knowledge to a meeting of the Basel-based Committee on the Global Financial System. Another official called the IMF’s contributions useful and timely.

More generally, the IEO evaluation team found that many of the policy-makers interviewed agreed that IMF multilateral surveillance could be a valuable input into national and global economic policymaking if it is conducted and communicated effectively.

Recommendations

To improve the effectiveness of multilateral surveillance, the evaluation report recommends that the IMF:

  • Strengthen the IMF’s role at the center of a more robust global peer review system by establishing a more proactive engagement with relevant intergovernmental groups.

  • Enhance the roles of the Executive Board and the International Monetary and Financial Committee in multilateral surveillance.

  • Streamline and better focus the products of multilateral surveillance, present shorter and clearer messages, and deliver them more strategically to target groups.

  • Define more clearly the goals of multilateral surveillance and the mechanisms to achieve them. Particular effort should also go into better integrating multilateral perspectives into bilateral surveillance.

IMF Board Urges Better Integration of Multilateral and Bilateral Surveillance

In discussing the IEO evaluation report on March 24, 2006,3 Executive Directors welcomed the broadly positive assessment of the analysis in the IMF’s key multilateral surveillance outputs. They also noted that multilateral surveillance products had been largely successful in identifying relevant issues and related risks in a timely manner. The wide and diverse public interest in these outputs, documented by the IEO’s report, is a testament to this success. Directors discussed ways to improve the effectiveness of multilateral surveillance, based on the IEO’s four recommendations.

The Board agreed that it would be beneficial to clarify the operational goals of multilateral surveillance, but it was not persuaded about the need for broad organizational changes. Priority should be given to strengthening the integration between multilateral and bilateral surveillance, particularly for countries that have an impact on global financial stability.

Directors noted that multilateral surveillance outputs would have a larger effect on the global policy debate if they were better targeted to their core audiences, streamlined, and focused on key issues.

Most Directors concurred that, while the Executive Board and the International Monetary and Financial Committee remain the most appropriate forums for discussing policy spillovers and possible responses, the IMF should also participate more actively in other forums—such as, but not limited to, the G-7 and the G-20—which provide opportunities for a frank exchange of views on multilateral issues.

The External Review of IMF Surveillance, commissioned by the Chairman of the Executive Board’s Evaluation Committee Thomas Bernes, was undertaken by a team of outside experts, including Ricardo Arriazu, a former alternate IMF Executive Director; John Crow, former Governor of the Bank of Canada and the Chairman of the External Review Team; and Niels Thygesen, the Danske Bank Professor of International Economics at the University of Copenhagen. The report is available on the IMF website (www.imf.org).

The McDonough Report, titled “Report of the Review Group on the Organization of Financial Sector and Capital Markets Work at the Fund,” provided the IMF with an outside and independent perspective on how it should organize its financial sector and capital market analysis and surveillance activities. The review was led by William J. McDonough, former President of the Federal Reserve Bank of New York. The report has not been made public.

A full Summing Up of the Board discussion, as well as the responses by IMF management and staff to the evaluation report, can be seen on the IEO website (www.imf.org/ieo) and are also included in the print version of the report.

IEO Evaluation Reports

2006 Multilateral Surveillance

2006 Financial Sector Assessment Program

2005 IMF Support to Jordan, 1989–2004

2005 The IMF’s Approach to Capital Account Liberalization

2005 IMF Technical Assistance

2004 The IMF and Argentina, 1991–2001

2004 The IMF’s Role in Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility

2003 Fiscal Adjustment in IMF-Supported Programs

2003 The IMF and Recent Capital Account Crises: Indonesia, Korea, Brazil

2002 Prolonged Use of IMF Resources

All IEO evaluation reports are available on the IEO website at www.imf.org/ieo. For further information on the IEO and its work program please see its website or contact the IEO at (202) 623-7312 or at ieo@imf.org.

Printed copies of the evaluation reports ($25.00 a copy) may also be obtained from:

International Monetary Fund, Publication Services

700 19th Street, N.W., Washington, D.C. 20431, U.S.A.

Tel.: (202) 623-7430 Telefax: (202) 623-7201

E-mail: publications@imf.org

Internet: http://www.imf.org

    Other Resources Citing This Publication