CHAPTER 2 Evaluation Projects Undertaken in FY2002–03
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- December 2003
This chapter provides a brief summary of the main findings and recommendations of the first three evaluation reports and the conclusions reached in the Executive Board discussion on each of these reports.
Prolonged Use of IMF Resources
The IEO’s first evaluation report focused on the phenomenon of countries having repeated access to IMF financing, which appears to conflict with the traditional perception of the role of the IMF primarily as a source of short-term financing with a revolving use of resources. The report examined what factors account for prolonged use, whether this is necessarily a problem for either the country itself or the IMF, and if so what can be done to ameliorate the situation. A wide variety of analytical techniques were used, including cross-country analysis, detailed case studies of three prolonged users (Philippines, Pakistan, and Senegal), and desk reviews of two countries (Jamaica and Morocco) that had ceased using IMF resources after periods of prolonged use.
“Prolonged Use of IMF Resources Prevents Their Proper Use”
—Asia Pulse, September 26, 2002
“IMF Lends Too Much, Too Long, soys Monitor”
—Financial Times, September 25, 2002
How extensive is prolonged use and who are the prolonged users?
Although the issue of prolonged use has been considered earlier on several occasions, the IMF had never adopted an official definition of prolonged use. The report adopted a definition whereby any country engaged in an IMF-supported program for at least seven of the previous ten years would be classified as a prolonged user in that year. Under this definition as many as 51 of the 128 countries that borrowed in the period 1971–2000 were prolonged users at some time.
The report also found that prolonged use has increased over time. While most of the increase involved low-income countries eligible for the IMF’s concessional financing, the bulk of the financial commitments to prolonged users came from the IMF’s nonconcessional resources (see Figure 1). Prolonged use has also been persistent in the sense that relatively few countries “graduate” from such use. On average, prolonged users have IMF resources outstanding for 21 years (see table for a list of some of the most prolonged users).
Figure 1.Prolonged Use Is on the Rise
Countries eligible for the IMF’s low-interest (concessional) loans accounted for most of the increase in the number of prolonged users. However, nonconcessional loans accounted for most of the increase in outstanding exposure.
Sources: IMF Finance Department; and IEO staff calculations.
PRGF = Poverty Reduction and Growth Facility.
What causes prolonged use?
The report identified five factors which led to prolonged use, some of which represent a natural expansion of the role of the IMF in response to changing circumstances.
(1) Broadening of the rationale for IMF program involvement. Over time, it was accepted that many balance of payments problems, especially in low-income countries, arose from deep-seated structural problems that required more time for adjustment. This realization, which also led to the establishment of the concessional facilities in the mid-1980s, implied that IMF financing would be provided over a longer period, especially in low-income countries. However, some of the consequences of prolonged use were not explicitly recognized. As a result, a mismatch developed between the new tasks the IMF was being asked to perform and its operational approach, which is still focused on achieving a restoration of external viability within a relatively short time frame.
(2) IMF lending as a “seal of approval” for other sources of financing. Many donors and creditors require an IMF-supported program as a seal of approval for other aid flows such as adjustment loans from international financial institutions (IFIs), or grants from bilateral donors, or debt relief, and this contributes to expansion of prolonged use. The evaluation concluded that linking aid to IMF-supported programs can compromise the quality of programs and hence, the quality of the seal of approval. This is because this linkage raises the stakes of program negotiations to the point of putting strong pressure on both country authorities and the IMF to reach an agreement, even though both parties may have doubts about the program’s feasibility.
(3) Programs as substitutes for surveillance. Program involvement has at times been extended because of the judgment that surveillance was not a strong enough vehicle to achieve desired results, either in terms of signaling the soundness of the macroeconomic framework, or promoting desired policy changes. Recent efforts to make surveillance assessments more transparent, to focus more sharply on vulnerability issues, and to promote the observance of internationally agreed standards and codes help to strengthen surveillance compared to the situations for much of the evaluation period, but additional steps could be taken in this direction.
(4) Weaknesses in program design and implementation. Prolonged use is sometimes a reflection of program failure, possibly on account of weaknesses in program design, which leads to repeat programs. Many of the reasons why programs were ineffective were not unique to prolonged users:
Programs are often overoptimistic in projecting real GDP growth, partly because insufficient attention was paid to analyzing how the real economy would respond to key policy measures or to assessing the expected sources of growth.
Many programs had difficulty dealing with uncertainty, in part because there was limited ex ante discussion of the major risks and how policies would broadly respond to those risks.
The risk to programs of weak ownership and lack of political commitment was often understated.
There was insufficient political commitment to core policy adjustments, which is often much more important than the extent and structure of specific policy conditions in IMF-supported programs.
Conditionality on structural policies was often exceedingly broad, without a clear order of priority among conditions, and its specification did not always ensure a good integration with program design (particularly as far as prior actions are concerned). As a result, compliance with a subset of these conditions was often sufficient for continued access to IMF resources even though it did not ensure that the most critical problems were being addressed.
(5) IMF governance and other internal institutional factors. Although the Executive Board has on various occasions approved the elements of a strategy to reduce prolonged use, including strengthened analytical and assessment efforts, more explicit “exit strategies” spelling out the circumstances under which IMF support would no longer be provided, and a more proactive use of conditionality, these have not been implemented systematically. One reason for this was the absence of an explicit definition of prolonged use, which made it difficult to enforce due diligence requirements for identified cases. In addition, a number of factors related to IMF governance and procedures also contributed to prolonged use:
Implementation capacity constraints that a program might face, related to political feasibility and ownership or to administrative capacity, are not given sufficient attention, leading to the failure of programs and repeat programs. Efforts were made to take account of these constraints in best practice cases, but there are insufficient systemic incentives to ensure that such an approach is followed generally.
|Years Under IMF Arrangements||Number of Arrangements (of Which Precautionary Arrangements)2||Years with Outstanding Obligations Greater Than 100 Percent of Quota|
Fifty-one countries met the definition of prolonged users at some point during the period 1971–2000. The table lists the 20 most prolonged users.
Precautionary arrangements are arrangements where the member country’s government has stated in its letter to IMF management that it does not intend to make drawings.
Fifty-one countries met the definition of prolonged users at some point during the period 1971–2000. The table lists the 20 most prolonged users.
Precautionary arrangements are arrangements where the member country’s government has stated in its letter to IMF management that it does not intend to make drawings.
Surveillance tends to be “crowded out” by program-related issues in prolonged use situations and this means there is insufficient opportunity to take a critical look at the adequacy of program design and to draw lessons from the experience of past programs. This leads to poor design of programs, contributing to repeat programs.
Political considerations are also at the root of prolonged use in some cases. Political considerations are bound to enter into the decisions made by an institution where ultimate approval rests with shareholder governments, but the blurring of technical judgments and political considerations in such cases contributed to the dilution of accountability and lower credibility of those programs.
Is prolonged use a problem?
The report recognizes that prolonged use can be justified in cases where the simultaneous challenges of macroeconomic stabilization, institutional development, and structural reform take considerable time to resolve. That was the case, for example, in many transition countries that have now “graduated” from IMF financial support and may also be true of many low-income countries. However, the evaluation suggests that prolonged use does involve significant costs that need to be borne in mind when considering the IMF’s long-term role. Specifically:
Prolonged use may hinder the development of robust domestic policy formulation processes over time;
There is an inherent tension between the quasi-permanent conditionality implicit in prolonged use and country “ownership”;
Surveillance activities, and the identification of longer-term vulnerabilities, may be “crowded out” by short-term program issues;
The perception that IMF resources would be available over the long term, despite policy slippages, may have weakened incentives to take decisive action to deal with the underlying adjustment problems; and
Prolonged use in some cases reflects pressures on the IMF to agree to a series of weak programs. As a result, the effectiveness of these programs will be reduced and the credibility of all IMF-supported programs may suffer.
The evaluation report made a number of recommendations covering program design, surveillance, and internal IMF governance issues which would help reduce prolonged use.
Institutional arrangements and rationale for IMF involvement
The IMF should adopt an explicit definition of prolonged use as a trigger for enhanced “due diligence” (i.e., systematic ex post assessments and forward-looking consideration of “exit” strategies). Greater efforts should be made to identify whether countries are ready to implement credible programs, and the IMF should be more selective in extending financial support where there is insufficient evidence of commitment to a credible program. To mitigate the “seal of approval” pressures for continued programs, the IMF should aim to provide the international community with credible alternatives to IMF lending arrangements as a condition for other official flows. A differentiated rate of charge for prolonged users should be introduced as a signaling device.
Specific operational procedures should be developed to maximize ownership, such as: (1) give the authorities the initial responsibility for proposing a reform program; (2) encourage a process whereby core program elements are subject first to a policy debate within the member’s own political institutions; (3) use surveillance to create a better understanding of what would be expected if a program should become necessary; and (4) undertake more explicit discussion of major uncertainties and how policies would be adapted if things turn out differently. Programs should place greater emphasis on key institutional changes and strengthening implementation capacity. This will require greater selectivity in program content and greater efforts to tailor the time frame of program design to the foreseeable length of a country’s adjustment needs.
Steps should be taken to ensure that surveillance is not crowded out by programs in prolonged user cases. A case exists for greater operational separation between surveillance and program activities for prolonged users in the context of prolonged use.
Internal governance issues
The ability of staff to analyze political economy issues should be strengthened and a review of internal incentives facing staff should be undertaken with a view to minimizing staff turnover on countries and to foster increased candor and accountability.
Executive Board response
The IMF’s Executive Board agreed with most of the IEO’s recommendations and the Managing Director established a staff Task Force to make specific proposals on how the issues raised could be addressed. The Task Force report was discussed by the Board in March 2003 and the broad strategy proposed by the Task Force was endorsed.11 This entails implementation of the IEO’s recommendations to improve surveillance, conditionality, and program design—many of which were already under way through ongoing initiatives—and additional measures to strengthen “due diligence” for prolonged users. The latter will be triggered by an explicit definition of prolonged use (with a clear understanding that such use is not necessarily undesirable) and will involve ex post assessments of past IMF involvement and forward-looking consideration of the longer-term strategy for the IMF’s role in the country concerned.
While endorsing the approach recommended by the Task Force, Executive Directors stressed the importance of systematic implementation and asked for a timely monitoring of progress in implementing the recommendations.
Institutional arrangements and rationale for IMF involvement
Directors saw merit in an explicit definition of prolonged use as a trigger for greater due diligence, but emphasized that prolonged use per se was not necessarily a problem and that the definition should not be interpreted as creating a new classification of member countries. The Board approved the definition proposed by the staff Task Force (for cases involving general IMF resources, seven or more of the last ten years under Stand-By or Extended Arrangements; for low-income countries, after a country has gone through two multiyear arrangements under concessional facilities). A semiannual report on the incidence of prolonged use will be prepared.
Directors also supported the call for greater selectivity in extending IMF financial support, based, inter alia, on assessments of implementation capacity and ownership. However, a number of Directors stressed that greater selectivity should not imply giving up on difficult cases. They endorsed the call for more explicit “exit” strategies in the case of prolonged users, which the staff Task Force proposed would be prepared as part of periodic ex post assessment and strategic planning exercises. Directors encouraged staff to continue to explore the scope for alternatives to IMF lending arrangements as signaling mechanisms. The broader role of signaling in low-income cases is being addressed as part of the ongoing review of the IMF’s role in low-income countries.
The Board did not support the proposal for a differentiated rate of charge for prolonged users.
Directors broadly agreed with the IEO’s recommendations and underscored that they should be seen as part of a continuing effort at improving program design to ensure greater effectiveness. Many of the recommendations are incorporated in the recently approved revised conditionality guidelines which provide the appropriate vehicle for putting them into practice. Further work on program design, including research effort, will also be needed, focusing in particular on the link between structural reforms/institutional capacity strengthening and program objectives. The regular reviews of conditionality by the Board will be the main vehicle for monitoring progress on these issues.
Directors agreed that priority should be given to increasing the effectiveness of surveillance, including the need to combine clarity and candor with recognition of social and political realities. They highlighted the importance of efforts to ensure that Article IV consultations in program countries “step back” from program context. Progress in strengthening surveillance, including by introduction of a fresh perspective, will also be addressed as part of the Board’s work program for FY2004.
Internal governance issues
Most Directors encouraged the staff to enhance its analysis and reporting of political economy issues in staff reports. Some Directors cautioned that the IMF should be careful in venturing into this area, given its comparative advantage in technical analysis and the need to avoid intruding on internal political matters. Directors also underscored the importance of distinguishing clearly between technical and political judgments and that staff should be candid in its assessment of risks.
On internal incentives facing staff, the Task Force took the view that while overall personnel policies do not need to be changed, management should consider guidelines and incentives to reduce excess mobility in country teams.
The Role of the IMF in Recent Capital Account Crises
The IEO’s second evaluation report examined the role of the IMF in three recent capital account crises, in Indonesia (1997-98), Korea (1997-98), and Brazil (1998-99), focusing on the effectiveness of IMF surveillance in identifying vulnerabilities in the precrisis period and the effectiveness of program design in restoring confidence. These crises have been extensively studied both inside and outside the IMF and a number of corrective measures have already been taken. The report made a number of additional recommendations building on steps already taken to further enhance the effectiveness of the IMF in both crisis prevention and crisis resolution.
“...the IMF’s response …’was… inadequate in many respects,’ the report says“
— “Post Mortems of Indonesia, Korea and Brazil Crises,” Business Times, Singapore August 4, 2003
“An Unusual Review of Mistakes Made by the IMF”
— Valor Economico July 30, 2003
Similarities and differences
The three country cases shared several features common to capital account crises. In each case, the crisis was precipitated very quickly, followed by a sharp currency depreciation; contagion was a factor; and the initial IMF-supported program failed to achieve its objectives. However, there were also important differences. The Indonesian crisis was the most severe of the three, with GDP declining by 13 percent in 1998, accompanied by a large increase in poverty and only a slow return to normalcy thereafter. Korea also experienced a contraction, with GDP falling by 6.7 percent in 1998, but in this case growth rebounded to 10.9 percent in 1999. Brazil’s growth experience was better than expected, with GDP growing by 0.8 percent in 1999 instead of a decline of 1 percent as projected in the original program.
An important difference across the country cases concerned the degree of political commitment to the program. Indonesia suffered most in this regard, with an evident lack of political commitment not only to the original November 1997 program but also to the revised January 1998 program. This, combined with a high degree of political uncertainty and social conflict, led to an environment that diminished investor confidence. In sharp contrast, the authorities in Korea quickly displayed a high degree of political commitment following the presidential election. The situation in Brazil was somewhat mixed. There were initial setbacks to the program, but this was soon followed by a greater commitment to the revised program.
The manner in which vulnerabilities developed and the ensuing crises manifested themselves also differed across the country cases. In Brazil, vulnerabilities were mainly in the public sector, arising from the mix of loose fiscal policy and tight monetary policy. This led to a steady rise in public sector debt and a progressive overvaluation of the real. In contrast, in Indonesia and Korea, the private sector was the primary source of vulnerabilities, in the form of weak banking and corporate sector balance sheets. Because of these weaknesses, the crisis in these countries was a “twin” crisis in both the external and the banking sectors. Brazil’s crisis was not a twin crisis, reflecting the fact that its banking system was relatively sound, and this explains to a large extent its macroeconomic performance following the sharp currency depreciation.
The evaluation identified a number of specific weaknesses in surveillance, program design, and internal governance.
IMF surveillance was generally successful in identifying macroeconomic vulnerabilities where they existed. In Brazil, for example, the IMF spotted the increasing vulnerability of the crawling peg exchange rate regime, especially in the face of fiscal weaknesses. However, in Indonesia, it underestimated the risks arising from financial sector and corporate balance sheet weaknesses and from the governance problems that contributed to those weaknesses. It also failed to recognize the macroeconomic impact of financial sector weaknesses in Korea, even though markets had begun to express concern on this aspect. The evaluation concludes that even when key vulnerabilities were correctly identified, the IMF’s role as confidential advisor was not very effective and suggests that the IMF probably could have been more effective at influencing policy if it had made its analyses public, thereby contributing to a wider policy debate.
Macroeconomic framework and projections
In all three cases, macroeconomic outcomes turned out to be very different from program projections. In Indonesia and Korea, the initial projections were overly optimistic, leading to a design of macro-economic policies in the programs that turned out to be too tight. In contrast, the initial projections for Brazil in 1999 were too pessimistic, perhaps in reaction to the earlier overoptimism in East Asia. However, this meant that the fiscal adjustment achieved in Brazil, welcome though it was, was less than it should have been, given the more favorable growth outcome. In retrospect, in light of Brazil’s continuing adverse public debt situation, this represented a missed opportunity. To some extent, these problems arose because macroeconomic projections in an IMF-supported program are necessarily the outcome of negotiation. However, there were also analytical weaknesses associated with the failure to take sufficient account of the large currency depreciation which might occur in view of the possibility of multiple equilibria, and the severe balance sheet effects that might result in view of currency mismatches.
All three programs initially involved fiscal tightening, albeit to different degrees: mild in Indonesia and Korea and fairly strong in Brazil. The evaluation does not support the claim that the fiscal tightening in Indonesia and Korea was the cause of the output collapse. The initial intention to tighten fiscal policy in Korea was probably not warranted, in view of the very favorable debt situation and the unexpected downturn in output, and the same consideration holds for Indonesia, although to a lesser extent. However, in both countries fiscal targets were relaxed quickly when the extent of output collapse became evident. The output collapse can be better explained by severe balance sheet effects arising from the currency collapse and (in the case of Indonesia) the negative effect of political developments on domestic investment. Brazil saw much stronger fiscal tightening, as was appropriate because fiscal sustainability was a major issue driving the evolution of the crisis. Sharply higher primary fiscal surpluses were achieved in line with program targets, but this turned out to be insufficient to achieve the objective of reducing the debt-to-GDP ratio.
The stance of monetary policy in all three countries was initially tight and there was an explicit recognition of the trade-off between higher interest rates and a weaker exchange rate. The outcome differed considerably across the three countries. In Indonesia, the maintenance of tight monetary policy envisaged by the program was initially not implemented because open-ended provision of liquidity to distressed banks led to a loss of monetary control. In Korea, the tight monetary policy envisioned in the initial program was implemented but this was not by itself sufficient to stabilize the exchange rate. It was only after the rollover of external debt was achieved that conditions stabilized and interest rates were gradually lowered in early 1998. In Brazil, there was an easing of interest rates immediately after the program was approved—over the IMF’s objections—and this may have contributed to the timing of the collapse of the crawling peg. However, after the currency was floated in January 1999, there was a decisive tightening of monetary policy in March 1999 and this led to the restoration of stability in the foreign exchange market. The experience of the three countries does not provide a definitive answer to the ongoing debate on the effectiveness of high interest rates in stabilizing the exchange rate. However, it is clear that an interest rate defense on its own is unlikely to be sufficient, particularly when balance sheets are weak.
Official financing and private sector involvement
The inadequacy of the official financing package was a major weakness of the initial Korean program announced in early December 1997. The IMF was aware of the problem and brought it to the attention of the major shareholders, even suggesting that if financing was not available then a rollover of debt should be orchestrated. However, faced with reluctance on the part of the major shareholders to take an initiative on this matter, the IMF arbitrarily resorted to unrealistically high assumptions about rollover rates in order to match the programmed financing gap to the available financing. It sought to bolster the financing available with bilateral credit in the form of a “second line of defense” which would be available if needed. However, the availability of this financing was visibly uncertain to the markets, and the program quickly collapsed when the markets reacted adversely. In Indonesia, the size of the package was not relevant—the problem really arose because of weaknesses in program design, the lack of political commitment to the program, and the complicating factor of political uncertainty and instability and its impact on domestic investment. In Brazil again the problem was not the size of financing but the fundamental unviability of the crawling peg. In Korea and Brazil, the IMF’s role as crisis coordinator in organizing private sector involvement was limited by the initial reluctance of major shareholder governments to use nonmarket instruments to influence the behavior of private sector institutions and concerns that such action might precipitate an exodus of capital from emerging markets. However, in Korea, once a decision was made by the major shareholders to ask international banks to maintain exposure, the IMF played a useful role in facilitating information exchange and setting up systems for monitoring compliance.
Bank closure and restructuring
The experiences of Indonesia and Korea suggest that programs for bank closure and restructuring to deal with a distressed banking sector must be based on a comprehensive and well-communicated strategy in which transparent rules are consistently applied. The lack of such a strategy in Indonesia was a major cause of the collapse of the November 1997 program. The issue of whether a blanket guarantee for deposits should have been introduced in Indonesia in November 1997, instead of the partial guarantee actually offered, deserves careful consideration. The evaluation suggests that Indonesia’s banking crisis was not yet systemic in November, so that the partial guarantee was appropriate. The system collapsed for other reasons and as it happened, when a blanket guarantee was introduced in January 1998, it was subject to abuse and consequently raised the fiscal cost of bank restructuring.
While all three programs involved structural conditionality, the Indonesian and Korean programs (especially the January 1998 Indonesian program) contained a proliferation of structural reform measures. In contrast, the scope of structural conditionality in the Brazilian program was limited to structural fiscal reform and prudential regulation. This difference is partly due to the absence in Brazil of many of the distortions present in the two Asian cases. While measures to rehabilitate and reform the financial sector were necessary in Indonesia and Korea, many of the nonfinancial structural reform measures were not necessary to restore confidence, although they may well have been beneficial in the long run. The evaluation suggests that, particularly in Indonesia, the proliferation of nonfinancial structural conditionality led to a loss of focus on critical reforms in the banking sector, and also contributed to the lack of ownership at the highest political level.
A program for restoring confidence must include a strategy to communicate the logic of the program to the public and the markets, in order to enhance country ownership and credibility. None of the three programs contained such a strategy initially.
Human resource management
The effectiveness of surveillance was reduced by the lack of sufficient incentives to make judgments that were frank and potentially unpopular with country authorities, resulting in a tendency for sharper elements of a diagnosis to be diluted in final Board papers. In crisis management, the quality of IMF response was compromised by a delay in the appropriate reallocation of staff resources and an insufficient utilization of available internal knowledge.
The role of the Executive Board and the IMF’s major shareholders
The Board and major shareholders interacted closely with management at all stages. The evaluation recognizes that such close involvement is necessary when major decisions need to be made quickly, though contacts at multiple layers can be counterproductive. However, there is evidence that some decisions were subject to micromanagement and political pressure, contributing to a blurring of technical and political judgments.
Collaboration with other international financial institutions
Tensions sometimes developed over the role that different IFIs should play in an IMF-supported program, particularly in matters relating to the financial sector. While a good working relationship eventually developed, it depended too much on personalities, and not on a well-defined procedure.
The report recognizes that a great deal of learning has already taken place within the IMF and that new guidelines have been issued, or are being discussed, to incorporate that learning into policies and operational procedures. Nevertheless, the report makes a number of recommendations which would further enhance the initiatives already taken.
Surveillance should take a stress-testing approach to analyzing exposure to a potential capital account crisis. In particular, staff should itemize risks and assess their potential impact, and develop greater understanding of political constraints on policymaking and of market perspectives on policy, in part through wider dialogue.
Increased transparency and incentives for candor
To increase the impact of surveillance, staff assessments should be made more candid and more accessible to the public. In particular, there should be a presumption of publication for Article IV staff reports. Management could develop modalities for escalated signaling to provide the Executive Board with a vehicle for signaling when failures to address identified vulnerabilities have become an increasing source of concern. Escalated signaling would give member countries enough time to address underlying vulnerabilities, while also progressing toward greater candor as a means of increasing the effectiveness and impact of surveillance. Moreover, management and the Board should explore the possibility of seeking “second opinions” from outside the IMF as part of the surveillance process when the authorities disagree with the staff’s assessment on issues that are judged to be of systemic importance. This would improve the degree of objectivity with which contentious issues are handled in the surveillance process and may enhance the impact of surveillance.
A comprehensive review of the IMF’s approach to program design in capital account crises should be undertaken. In particular (1) greater attention should be paid to balance sheet effects; (2) program design should allow for a flexible response, in case unfavorable outcomes materialize; (3) conventional financial programming-based conditionality should be reviewed and possibly adapted for capital account crisis circumstances; (4) parsimony and focus should be the basic principles of structural conditionality, and crises should not be used for pushing reforms that are not critical to crisis resolution, however desirable they may be in the long run; and (5) there should be an agreed communications strategy, characterized by a high degree of transparency.
The IMF as crisis coordinator
The IMF should ensure that the size of the financing package is sufficient to generate confidence and that its quality is such as to make it credible. In particular (1) packages should not rely on parallel official financing unless the terms of access are transparent and credible; and (2) terms for the involvement of other institutions providing parallel financing should be specified at the outset. The IMF should be proactive in its role as crisis coordinator. In particular, management should provide a candid assessment of the probability of success to the Executive Board and shareholders, and ensure that the technical judgment of staff is protected from excessive political interference. While the nature of private sector involvement will have to be determined on a case-by-case basis, the IMF should play a central role in identifying circumstances where concerted efforts can help overcome “collective action” problems.
Human resource management
Human resource management should be adapted to develop and better utilize country expertise, including political economy skills, and to establish “centers of expertise” on crisis management issues. In particular (1) the length of staff assignments should be monitored to ensure continuity of staff expertise, and a critical mass of expertise on each systemically important emerging market economy should be developed; (2) Resident Representatives should play a more central role in surveillance and program design; and (3) internal procedures should be made to protect those who raise uncomfortable issues through proper channels, but consequently attract complaints from the authorities.
Executive Board response
The Executive Board discussed the report in May 2003. Executive Directors welcomed the report as offering a comprehensive and thoughtful analysis of the Fund’s role in capital account crises in the three cases and of the lessons to be learned. They considered it a useful complement to previous studies undertaken both within and outside the Fund (see Appendix 6 for more details).
Surveillance, transparency, and candor
Directors concurred with the overriding message of the report: to strengthen the effectiveness of IMF surveillance by extending and systematizing current guidelines for assessing vulnerabilities. They supported the call to itemize major potential shocks while emphasizing that stress-testing should not be mechanical but should focus on key risks facing a particular country. Most agreed that the IMF should develop greater understanding of political constraints on policymaking, although some cautioned that this could be counterproductive if it was seen as interference in domestic affairs. Most Directors saw great value in systematic discussions with the international financial community. Many Directors were not in favor of shifting from voluntary to presumed publication of staff reports, but some strongly supported presumed publication. Directors agreed with the need for greater emphasis on candor in staff reports. Reactions to the proposal for escalated signaling were mixed, with many Directors considering it an idea worth pursuing while a few felt it would undermine the IMF’s role as a confidential adviser. On the related issue of second opinions, many Directors were not in favor, but some felt it may provide a fresh perspective.
Directors endorsed the IEO’s recommendations on program design, and hoped that forthcoming staff papers on program design and balance sheet effects would give due attention to them.
The IMF as crisis coordinator
Directors generally agreed with the recommendations on financing packages, while noting limitations on the IMF’s influence on other sources of financing. The Board stressed that the recently revised access policy must be observed and emphasized the importance of program credibility, not large financing packages, as the heart of IMF involvement. The Board stressed that political judgments and decisions should be its exclusive domain and that it should be involved early in program discussions but without micromanaging operational details. Directors attached particular importance to the early involvement of the private sector.
Human resource management
Directors generally agreed on the need for institutional change to ensure that the IMF is in a position to respond rapidly to member countries facing crises. Many Directors supported the creation of centers of expertise; longer country desk assignments; a greater role for Resident Representatives; and modification of internal guidelines and human resource procedures. They also noted that human resource issues are management’s responsibility.
It was noted that ongoing Board discussions would revisit specific operational aspects of the recommendations, particularly those endorsed by the Board. Biennial review of surveillance will be the main vehicle for assessing what has been done to address surveillance-related recommendations. Many of these and other issues would be discussed again by the Board in the context of discussions on transparency, financial soundness indicators, the balance sheet approach, reporting requirements, data standards, and sustainability assessments, and operational decisions will be taken by the Board in the context of those discussions.
Specific actions have already been taken by management in some areas addressed by the report, including the setting up of an Internal Task Force to review broad strategic issues relating to the IMF’s Resident Representative program and the recent reorganization of the Monetary and Financial Systems (formerly Monetary and Exchange Affairs) Department.
The Role of Fiscal Adjustment in IMF-Supported Programs
The third evaluation undertaken by the IEO in FY2003 dealt with fiscal adjustment in IMF-supported programs and focused on two sets of controversial issues. The first concerns the quantitative dimension of fiscal adjustment: whether, as some critics contend, fiscal adjustment in IMF-supported programs (1) suffers from a “one-size-fits-all” approach; (2) is insufficiently flexible; and (3) is often unnecessarily contractionary. The second is the qualitative dimension: whether the efficiency, sustainability, and equity of fiscal adjustment could have been improved by using a different sequence and composition of policy measures on the revenue and expenditure side. To address these questions, the evaluation used a large crosscountry sample of up to 169 programs in the 1993–2001 period, supplemented by more detailed desk studies of 15 specific IMF-supported programs including analysis by local experts in four cases.
“IMF ‘Too Optimistic’ on Growth, says Watchdog Evaluation Report”
—Financial Times, September 10,2003
Quantitative aspects of fiscal adjustment
Is there a “one-size-fits-all” approach?
The evidence does not support the view that IMF-supported programs typically adopt a one-size-fits-all approach to fiscal adjustment, nor the perception that programs always involve austerity by targeting reductions in current account deficits, fiscal deficits, or public spending as a percent of GDP. In fact, 40 percent of the programs examined targeted a widening of the current account deficit as a percentage of GDP, while slightly over a third planned an increase in the primary fiscal deficit and primary spending as a percentage of GDP.
Variation across countries obviously does not establish that the targeted fiscal adjustment is appropriate in each case. The evaluation used cross-section analysis to examine factors underlying the observed variation and this revealed some patterns: the targeted adjustment in the fiscal deficit is higher when initial fiscal deficits are higher, when expenditure levels relative to GDP are high, and when proposed current account adjustments are large. However, these issues could not be pursued in the individual country programs examined because program documents do not indicate how fiscal adjustment targets are determined and how they are related to the specific assumptions made about the pace of recovery in private sector demand and short-term growth prospects. In the absence of such information, an evaluation of the appropriateness of the adjustment attempted in individual countries could only have been attempted by undertaking a detailed study, specifying an appropriate (counterfactual) program for each situation, and comparing the fiscal adjustment in that program with the targeted adjustment in the IMF-supported program. This was beyond the scope of the evaluation.
How flexible are targets?
The cross-country evidence does not support the view that IMF-supported programs are insufficiently flexible in an ex post sense, forcing a rigid pattern of fiscal adjustment that is insensitive to changes in circumstances. In fact, many programs fail to meet initial targets, and these targets are revised to reflect reality.
On average, IMF-supported programs achieved only about one-half of the targeted improvement in overall and primary fiscal balances. However, there is significant variation around this average; about 60 percent of programs underperformed, with SBA/EFF-supported programs in nontransition countries registering the highest incidence of shortfalls, while SBA/EFF arrangements in transition countries had the smallest shortfalls. Moreover, almost all fiscal adjustment achieved in the first two years occurs in the first year of the program. Except in the transition country arrangements, programs were on average unable to achieve further fiscal gains during the second year of the program, in spite of relatively more ambitious fiscal targets for the second year.
As many as two-thirds of the programs studied incorporated revisions to their initial fiscal deficit targets by the time of the completion of the second program review. The revisions are asymmetrical—fiscal targets are often revised downward when growth is below expectations, but less often revised upward when growth is higher than originally projected. Interestingly, the evaluation found that program documents do not explain clearly the extent to which the fiscal shortfall observable at the review stage is due to a weaker policy effort or to the effect of exogenous developments.
Is fiscal adjustment unnecessarily contractionary?
This is one of the most controversial issues related to the design of IMF-supported programs and the evaluation revealed some interesting patterns. There is no evidence that growth rates in program years were systematically lower than the trend in the preceding decade. However, there is considerable cross-country variation around this average behavior. There were a number of cases, particularly the group of capital account crisis cases, where growth did slow down and in some cases even turned negative.
There is clear evidence of overoptimism in projecting investment rates and GDP growth. Actual investment rates in the second year of the program were below projections in 60 percent of cases and 5 percentage points of GDP or more below projections in about one-fourth of cases. Growth optimism is particularly evident in programs that started from an adverse situation. When growth was negative during the first year of the program, growth projections for the second year were on average twice as high as in reality. Programs were also generally reluctant to project a slowdown in growth and very rarely projected negative growth. This “reluctance to predict a downturn” has potentially significant implications for program design, since it means that the need for countercyclical fiscal policy and its appropriate role is rarely discussed explicitly.
A tighter fiscal stance by itself cannot be called contractionary since it may be needed in situations where private investment demand is potentially buoyant and fiscal contraction is necessary to create room for financing private investment. However, it may not be appropriate when there is a sharp downward shift in the investment function, or when the level of private demand responds more sluggishly to the program than originally projected. In such cases, a demand stimulus may well be appropriate, calling for a less contractionary fiscal stance.
The lack of clear explanation of the rationale of the extent of fiscal adjustment in individual cases makes it difficult to determine whether the fiscal stance is indeed unnecessarily contractionary. A contractionary bias is most likely when both output and investment are below projections, the current account deficit is lower than programmed, and reserve accumulation is above the program target (indicating that external financing was not a constraint). In addition to the findings regarding output and investment optimism, the evaluation found overperformance on the external side in about one quarter of cases. Under these situations, it is possible that the adjustment planned was unnecessarily contractionary. However, even this conclusion needs to be qualified, because it focuses only on the impact of fiscal adjustment on aggregate demand. Where debt sustainability is an issue, a loose fiscal stance justified on countercyclical grounds may actually prove to be destabilizing if it is misread by markets as indicating a lack of commitment to macro stability, leading to a delayed return of confidence. Weighing these different considerations is difficult, but these issues need to be explicitly discussed and explained in program documentation, which is generally not done.
Projections of aid flows in concessional programs
Concerns have been raised that IMF-supported programs in low-income countries unnecessarily squeeze spending because the aid projections are too conservative and assumed to “taper out” too quickly relative to what donors may be willing to provide. The evaluation shows that program projections of aid do tend to decline over the medium term, albeit at a moderate pace in most cases, but there is no evidence that projections have systematically underestimated actual aid flows for the outer years of programs. However, the evaluation did not address the issue of the scope for mobilizing additional aid flows in support of more ambitious public spending linked to interventions to reduce poverty.
Effect of IMF-supported programs on the level of social spending
Econometric analysis of 146 countries observed over several years with and without an IMF-supported program finds no evidence that the mere presence of an IMF-supported program leads to lower public spending in either health or education below the level that would have prevailed in the aftermath of a crisis if there were no program. Indeed, the evaluation found that public spending on health and education was actually about 0.3 to 0.4 percentage points of GDP higher compared to what it would have been without the program. This increase is sustained beyond the end of the program but diminishes over time.
The fact that social spending is higher than it would have been in a nonprogram situation does not establish that the most vulnerable groups are effectively protected from economic shocks during program years. Unless programs and budgetary mechanisms that provide such protection are effectively in place, it is difficult to ensure that additional resources devoted to this sector will reach target groups. IMF-supported programs generally operate in a time frame which is too short to create such instruments if they do not already exist, and in any case the IMF lacks the necessary expertise. An alternative framework is clearly required to address such issues.
Social concerns in program design
At present, the treatment of social protection in non-PRGF countries is governed by the 1997 Guidelines on Social Expenditures, which call for the IMF staff to track health and education spending and encourage authorities to incorporate spending targets for these sectors in program objectives by relying on work by the World Bank. However, a detailed examination of the 15 sample programs shows substantial variation in how social expenditure issues are treated in practice. Trends in broad categories of expenditures such as education and health are noted, but only one-third of programs analyze these trends and identify priority social expenditures that need protection. In a more recent group of programs, half of the program documents analyze these changes, but few programs (outside the PRSP/PRGF countries) discuss how explicit monitoring and feedback systems could be established, or how to integrate these aspects with the work program of the World Bank.
Preserving budgetary allocations in critical areas to protect the most vulnerable groups from these shocks is not too costly, especially in middle-income countries. However, this objective cannot be achieved simply by monitoring trends in broad social spending categories. Spending categories that are most critical to vulnerable groups are often squeezed out by other expenditures in the same broad category (in education, for example, basic medical or primary school supplies can be preempted by personnel expenditures).
Fiscal reforms under IMF-supported programs
Implementing structural reform on the fiscal side is crucial because, in the absence of progress in this area, much of the fiscal adjustment that may be achieved in the short term is not sustainable in the longer term and does not strengthen the ability of the fiscal system to meet future shocks.
The evaluation found that the fiscal designs of programs make only limited gains in the structural area. They have generally focused more on the revenue side than on spending reallocations and reforms and the focus on the revenue side has been on introducing a value-added tax (VAT) or increasing rates, with relatively little attention paid to policy efforts aimed at reducing exemptions and evasion.
The VAT should remain as the cornerstone of a modern tax system. However, considerable improvements in collections could be achieved by curtailing discretionary exemptions and reducing tax evasion, particularly in customs duties and direct taxes (personal and corporate); even in the short run, these efforts could yield important revenue increases. Targeted efforts focusing on well-known groups or categories of taxpayers believed to be significantly under-paying can help generate revenues relatively quickly while avoiding the need for large increases in tax rates. The evaluation finds that efforts by the IMF in this area have not been forceful enough, particularly if they affect powerful vested interests. Tax administration reforms in IMF-supported programs have often focused on the technology side (such as computerization, training, etc.) rather than on the politically more difficult actions, such as legislation to empower tax agencies to pursue tax evasion forcefully or to make the system less prone to political interference.
On the expenditure side, conditionality has been concentrated on short-term quantitative targets to reduce public employment, or cap public sector wage increases, rather than focusing on the reorientation of public spending and medium-term civil service reform. As a result, the process has been inefficient and reversals have often occurred.
Progress in implementing fiscal reforms
The sample of 15 programs showed that progress in implementing fiscal reform was limited. In no given reform area was implementation rated satisfactory in more than 40 percent of cases. Measures to reduce the public sector wage bill, achieve civil service reform, and reform the social security system have been particularly difficult to implement. The lack of progress in implementation of reform measures is also the result of a mismatch of time frames: the short horizon of programs relative to the longer time needed to complete institutional reforms.
Learning and the process of surveillance
The evaluation found that program requests do not make sufficient efforts to evaluate past fiscal performance or analyze policy failures under prior arrangements. Typically, programs tend to focus on fiscal performance during the last year prior to the program, with little attention to policy failures under previous arrangements.
The evaluation also examined surveillance during the three-year period prior to the program and its link to program design. Although there is variability across programs, in general efforts during surveillance to flag forcefully the need to accelerate reform have been limited. Surveillance is drawing too few lessons from past failures, and not contributing to defining the future paths for more complex reforms.
Internal review process
An examination of the internal review process revealed that the scope and detail of review departments was greater at the stage of program implementation than at the stage of initial program design. The process tends to be reactive—with reviewers commenting increasingly as the programs unfold, instead of at the design stage. A comprehensive internal debate would have the greatest value added at an early stage of program formulation involving an exploration of alternative policy options.
The evaluation made recommendations in five areas.
Program documentation should provide a clearer justification for the magnitude and pace of targeted fiscal adjustment and how it is linked with assumptions about the recovery of private sector activity and growth.
The internal review mechanism should place more emphasis on the early stages of the process, and the key briefing paper for program negotiations should also articulate more clearly the basis for the fiscal program and its links with debt sustainability issues.
Formulation and implementation of key institutional reforms
Programs should make stronger efforts to specify those structural reforms which should be carried out during the program horizon as part of a broader road map of priority reforms that ideally should have emerged in the course of surveillance.
The surveillance process should be used to provide a domestically owned longer-term road map for fiscal reforms and also to assess progress achieved. The identification in advance of areas considered critical will foster greater domestic debate and encourage homegrown solutions and greater ownership while allowing the authorities flexibility in the timing and packaging of reforms.
The analysis should assess why important distortions were not addressed and what lessons were learned from past experience. This should include an effort to identify and unbundle the various constraints to critical reforms. Surveillance should include more systematic efforts to estimate the extent of tax evasion and tax exemptions, including the use of cross-country comparisons. Public debt sustainability could help anchor the road map of fiscal reform priorities proposed above and to assess trade-offs over time.
Role of the IMF in social protection
The IMF should clearly delineate the operational framework in which social issues will be addressed within program design in non-PRGF countries. This should include a clear indication of the IMF’s responsibilities and activities in this area, which should include alerting the authorities about the need, in crisis situations, to protect those social programs and spending categories that are critical to the most vulnerable groups. Since this requires preparatory work, the IMF could invite the authorities, regularly during Article IV consultations, to suggest what existing critical social programs and social services they would like to see protected in the event of adverse shocks. On the basis of the priorities identified by the authorities, the World Bank and the IMF could agree with them on reform of public expenditure management (PEM) systems, specifically geared to protect the specified programs and spending categories. This would be a concrete application of the ongoing collaborative initiative by the IMF and the World Bank to strengthen PEM systems according to priorities set by the countries themselves.
Executive Board response
Executive Directors welcomed the report and agreed that the report has a number of constructive recommendations whose implementation would enhance the Fund’s advice on programs in the fiscal area. Directors commented on the specific recommendations of the IEO report (see Appendix 7 for more details).
Program design and internal review process
Directors supported the recommendation that program documents should provide a more in-depth justification for the magnitude of the fiscal adjustment, and deemed that this initiative would instill greater discipline in program design, enhance transparency, and provide the public and the private sectors with a more convincing rationale for the program, thereby helping to overcome political obstacles to implementation. Nevertheless, they recognize that uncertainties regarding key macroeconomic variables particularly under crisis situations complicates this task. A few Directors cautioned against spurious precision in such justification. Directors looked forward to further staff analysis of the issue of growth projections in the context of the program design discussions.
Directors supported the recommendation that the internal review process should place relatively more emphasis on the early stages. They welcomed management’s recent initiative in this area.
Enhancing key institutional reforms and their links with surveillance
Directors agreed that these reforms can be more critical for fiscal sustainability than short-term expenditure and revenue measures. However, they recognized that short-term measures are hard to avoid if the immediate objective is economic stabilization. Medium-term institutional reforms are particularly relevant in countries that have achieved macroeconomic stability and where “second generation” reforms are necessary. Some Directors agreed with the report’s suggestion that reforms should be broken down into those that require executive action, legislation, and capacity building. Directors, however, pointed out that in crisis situations the pressing need to resolve a crisis may pose serious constraints on a medium-term approach. Careful judgment will continue to be needed to focus conditionality on reforms judged critical while at the same time ensuring that progress is made in addressing vulnerabilities and achieving programs’ goals.
Most Directors agreed that Article IV consultations should play a stronger role in identifying long-term reform priorities and the causes of past failures, and that this analysis should inform subsequent program design. Nevertheless, they underscored that the ultimate responsibility to develop a fiscal reform agenda resides with the individual country authorities, while the Fund should stand ready to provide advice.
Role of the IMF in social protection
Directors agreed that an important aim of program design should be to protect critical social expenditures. However, they stressed, as recognized in the report, that the Fund should not become involved in the detailed selection and design of social policy; this task is outside both the Fund’s mandate and its expertise. A number of Directors supported the IEO’s call for updating of the 1997 guidelines that direct IMF work in the social area, in order to improve their clarity and effectiveness as an operational tool in protecting the most vulnerable from economic shocks and budgetary retrenchment. Other Directors, however, viewed the existing guidelines as adequate, and a few considered that the annual and medium-term budgets of non-PRGF countries already adequately identify critical social sector programs. These Directors recalled that the new framework for Bank-Fund collaboration on public expenditure issues should enhance countries’ public expenditure reform strategies, including measures to protect critical social spending. Many Directors agreed with the recommendation that staff should inquire, during Article IV consultations, whether the authorities have identified social programs that they would like to protect in the event of a crisis, as they believed this would help dispel the criticism that Fund programs unduly curtail social spending. A few others considered this recommendation impractical, as it would create significant costs and pressures for the authorities with little benefit.
See International Monetary Fund, “Conclusions of the Task Force on Prolonged Use of Fund Resources,” February 4, 2003 as well as PIN 03/49, April 9, 2003, “IMF Concludes Discussion on Prolonged Use of Fund Resources”—both available at www.imf.org. An internal operational guidance note for assessments of countries with a longer-term program engagement has also been issued.