Chapter 5 The Process of TA Delivery
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- October 2005
This chapter examines the process of TA delivery—ranging from the design of TOR, the formulation of recommendations, the selection of modalities for providing TA, to the articulation of a dissemination strategy. We also look at how IMF experts interact with local counterparts and authorities. The discussion is based on the field work of the six country studies. A summary of major findings by country may be found in Table 5.1.
Formulating Requests for TA and Drafting the Associated Terms of Reference
Although we find some variability, the case studies generally showed a passive involvement of the authorities in preparing TA requests and in drafting the TOR for specific TA activities. The TORs were usually prepared by IMF staff and approved by the authorities with little discussion, which tended to reduce the authorities’ ownership of the activity and frequently masked important differences in expectations between staff and the authorities on the speed and direction of reform. This was particularly problematic when the officials in the agencies directly receiving the TA were not involved in the process.
For example, in Yemen, a number of officials, particularly those outside the central agencies, expressed concern that they were not adequately involved in drafting the TOR for activities in which they were directly involved. As a result, they felt that the TOR did not reflect their specific needs and priorities. These officials suggested that greater ownership could have been achieved if the authorities were asked to prepare the initial draft TOR on which the IMF would then comment. In their opinion, this would have increased the likelihood that the direct recipients of the TA were consulted.
This was also the case in Cambodia, where senior officials in the Budget Department—in charge of implementing TA recommendations—were not involved in setting the reform agenda being supported by IMF TA. This not only reflected weak communications within the Ministry of Finance, but also insufficient efforts by IMF staff to interact with officials in the budget areas other than the most senior officials. Several past internal and external evaluations of IMF TA have detected similar problems.
We also found cases where weaknesses in institutional capacity were a serious handicap to putting together concrete requests for TA. This was the case for the central statistical office in Zambia, where there was a dearth of resources (both human and financial). The problems appear to have been mitigated by a two-stage approach using a diagnostic mission in the context of the GDDS initiative to help the authorities define more precisely TA needs in various areas.
In Niger, on the other hand, the drafting of the TOR for the advisor in tax administration was interactive, and based on the results of a workshop that was organized with senior officials in the tax department during the mission to install the expert. The authorities participated actively in these deliberations and agreements were reached on ways to track progress, with reports to be submitted both to the IMF and the authorities. Partly as a result of this, the work of the expert had a significant positive impact.
We found improvements in the extent of the authorities’ involvement in the preparation of TORs over the course of successive TA interventions. For example, in the case of TA on PEM in Zambia, the authorities were little involved in setting the initial agenda for the first resident expert in October 1997. In contrast, subsequently they were active participants in setting and agreeing to a detailed action plan, from which the TOR for the most recent resident expert was derived. FAD made determined efforts to have high- and middle-level officials actively involved in the discussions to increase ownership and establish clear understandings concerning the expectations of both the authorities and the resident expert.
Terms of reference for TA activities should also be based on previous diagnostic work. This was the case for TA on tax administration in Honduras. The TOR for the long-term expert was anchored in the findings of a previous mission that issued a major report on reforms in tax systems and tax administration with specific and clearly prioritized actions. A similar experience was seen in Zambia, where the TA in statistics was guided by a preliminary diagnostic work under the GDDS initiative. To be sure, having diagnostic material as a stepping-stone for preparing TORs should not be construed as removing the need to consult with the authorities on the TOR. This is particularly the case if the extent of the authorities’ agreement with the findings and recommendations of the diagnostic mission is not clear.
|Banking Supervision and Bank Restructuring||Tax and Customs Administration||Budget, Treasury, and Public Expenditure||Multisector Statistics|
|Cambodia||Officials at the National Bank deemed that the process worked efficiently combining short-term peripatetic missions and the LTE. Whenever problems were encountered in implementation (in between missions), officials could consult with the LTE.|
Authorities expressed their strong preference for hands-on experts.
|Lack of funding constrained the follow-up of TA recommendations. In tax and customs departments the ability to satisfy their computerization needs, in terms of hardware and software, is limited or nonexistent.||Contrasting performances of the different LTEs.|
The initial design did not fully identify the nature and dimension of the payment arrears problem.
Officials in charge of TA implementation were not included in the design of the reforms, which undermined ownership.
|As acknowledged by the authorities and donors, an LTE proved to be the adequate mode of TA delivery given the low capacity of the civil service.|
TA recommendations in statistics are usually not very controversial politically, and thus are more likely to be implemented.
LTE praised for leadership and hands-on approach.
|Honduras||Short-term missions often did not take sufficiently into account the inputs of the authorities.|
The LTE made an outstanding contribution, characterized by a strong leadership, hands-on style, and accessibility.
|Comprehensive background work and analysis by staff prior to the start of the resident expert.|
LTE was praised for his leadership and hands-on style.
|Poor coordination with other donors.||TA recommendations were the product of a close consultation process with the authorities.|
Country counterparts stressed that follow-up often requires additional resources that are not available.
|More candid and timely report—by experts and backstoppers—on the lack of commitment by senior staff, would have been beneficial particularly when decisions regarding the yearly renewal of TA had to be made.||Good tracking of progress, except in auditing taxpayers.|
Poor coordination with the Inter-American Development Bank.
|Niger||There was not enough dissemination of the TA reports.||Resident expert assigned to Niger in light of delay in establishing West AFRITAC; would likely not have received LTE otherwise.|
Authorities closely involved in drafting of TORs for LTE.
Regular reporting by LTE to IMF staff and head of Tax Authority.
Accessible and effective expert; appropriate modality for Niger.
|Authorities actively involved in formulating TORs for at least some short-term missions.|
Authorities regretted lack of time to discuss recommendations during short-term missions.
TORs for LTE refocused at time of inspection mission to assessing need to extend contract.
Very accessible expert; appropriate modality for Niger.
|Very little TA in provided by IMF staff between 1996 and 2003; problems with GDP data series not addressed by TA.|
Initial shortage of French-speaking experts.
STA contracted AFRISTAT to provide real sector TA; however, ability to monitor progress undermined by little detailed reporting back to headquarters and almost no oversight or backstopping of expert.
TA in government financial statistics delivered through regional advisor.
|Heavy emphasis on training and hands-on support.|
Unclear division of responsibility between West AFRITAC and headquarters staff for providing TA.
|Heavy emphasis on training and teaching; used team approach.|
Unclear division of responsibility between West AFRITAC and headquarters staff for providing TA.
|Reluctance of staff to change GDP data in midst of program.|
Relatively few training opportunities for staff of statistical agency.
|Ukraine||Tensions between officials and resident experts, particularly in restructuring and closure of banks.|
Work of experts in this area became less welcomed, and they were diverted to the area of laws and regulations.
Some of the tensions occurred because actions were part of program conditionality.
|Design based on initial pilot approach, which was then generalized. Some observers felt approach was too ambitious given initial conditions, and that a less ambitious program could have been explored.|
LTEs judged well by authorities.
|Progress was slow at beginning and generated tension between the IMF and authorities.|
Officials are of the view that the purpose of the Treasury Single Account was not well explained and different options should have been put forward.
Officials thought that Ukraine had appropriate arrangements in place that were not fully appreciated by the staff.
|Officials extremely happy with technical expertise and collegial approaches of resident experts and ability to establish links and help cooperation across agencies.|
Translation of documents and reports a problem.
|Yemen||Central Bank of Yemen (CBY) designated counterparts to expert to ensure continuity.|
Efforts were made to send counterparts on courses to enhance ability to absorb TA and disseminate it within CBY.
Continuity between missions facilitated though maintenance of communications with peripatetic expert.
Expert focused on knowledge transfer; emphasized classroom and on-the-job training and helping draft supervision manuals; also provided recommendations for a long-range career development curriculum for banking supervisors.
|Weak up-front effort to assess ownership and extent to which authorities were prepared to commit their own resources to making effective use of TA; lack of detail in requests often masked weak commitment to effective use of TA.|
Little direct assessment by IMF of likelihood of parliamentary passage of general sales tax legislation before commitment of significant TA resources.
Recipient agency (i.e., Tax Authority) not involved in formulating TORs.
|Authorities did not clearly or substantively justify requests for TA, thereby masking lack of commitment to making effective use of TA.|
Lack of specificity in TORs, including with respect to expectations of coordination with other TA providers.
No up-front agreement with authorities on role of training.
Inability to speak Arabic undermined effectiveness of one LTE.
Little feedback of experience to subsequent activities.
|Difficulties in translating many of the materials developed by expert.||Coordination problems between World Bank and IMF.|
|Assignment of responsibilities to specific individuals encouraged accountability and “ownership.”|
|Zambia||TA focused on major concerns of PRGF-supported program.|
Peripatetic modality a good fit for Bank of Zambia’s institutional capacity.
Although expressing reservations with some TA recommendations, authorities commended experts and quality of discussions.
|Not much involvement of the authorities in design of the early TA interventions, thus undermining ownership; later corrected.|
TA interventions too short-term focused on supporting IMF program, including decisions to renew TA.
Weak coordination with other donors in related areas.
Some advisors were not effective in relating with the authorities.
|TA guided by a preliminary diagnostic work under the GDDS. TORs anchored to this diagnosis.|
TA reports and recommendations prepared in a collaborative fashion.
Early assessment of institutional weaknesses that undermine the effectiveness of TA
A consistent theme in many of our case studies is the extent to which the effectiveness of IMF TA is undermined by a lack of awareness of institutional, organizational, or managerial particularities in recipient countries. This can take the form of a lack of clarity on decision-making authority and accountability (e.g., Yemen) or difficulties in generating ownership in one part of a government department of work done elsewhere in the department (e.g., Ukraine), or in different parts of the government (e.g., Zambia).
It is difficult, if not unrealistic, to expect each IMF TA intervention (particularly short-term missions) to understand and anticipate such obstacles. However, failure to do so can diminish greatly the benefit from IMF TA. This can be particularly important when there is a sudden increase in TA activities in a new environment, as happened with respect to PEM in Cambodia, Yemen, and Zambia.
In the case of Cambodia, the effectiveness of TA in budget execution and treasury was initially undermined because the design did not fully identify the nature and dimension of the payment arrears problem. In Zambia, some government institutions were deliberately entering into arrears to obtain resources beyond their budgetary allocation. An awareness of this behavior was key to the assessment of the expenditure control problem and to making recommendations.
In the case of Ukraine—a country with good levels of education and well-established institutions, albeit still geared to the previous centralized system—the challenge was to explain the reasons why specific TA and institutional changes were needed. In our discussions, Ukrainian counterparts thought that the tensions that emerged with the IMF in the area of treasury systems could have been avoided if more time had been taken at the start to explain why and what was required. This is consistent with the findings of an FAD assessment of the TA experience in introducing treasuries in the Baltic countries and other countries of the former Soviet Union. It is candidly acknowledged in that assessment that more time should have been spent in explaining to the authorities and officials the merits of introducing such a system.
One possible way to address this problem would be for IMF staff periodically to prepare an internal assessment of the managerial and organizational challenges in key government departments and agencies in a particular country.1 In addition to identifying these challenges, staff could draw on past experience to suggest proactive strategies for dealing with many of these problems. Given the rotation of IMF staff among countries and the use of outside experts, this could help ensure that the particularities of working in one part of a government administration are well understood and integrated into TA. For such an effort to be worthwhile, these assessments would need to be frank and clear and should—where warranted—draw on input from other major TA providers in a particular country. We do not have in mind a complex system of guidelines requiring detailed formal reports and clearance procedures, but rather a greater emphasis on asking up-front, what are likely to be the main obstacles to progress and how can they be tackled. Such efforts could focus on selected countries that are expected to receive large amounts of TA from the IMF over a given period of time and where the IMF is less familiar with the institutional environment of the country (e.g., in post-conflict economies). Resident representatives should play a key role in preparing such assessments.
Short-Term Versus Long-Term Modalities for Providing TA
The 1999 Fund-wide evaluation suggested that greater emphasis should be given to shorter-term modalities for providing TA rather than to resident experts. It was thought that the former were a more cost-effective means of providing TA. This principle was therefore integrated into the filters contained in the 2001 TA policy statement.
The evidence from our case studies does not support such a definitive conclusion. Our case studies showed that where institutional capacity was particularly weak, long-term experts (LTEs) could sometimes play a critical role, suggesting that the most effective modality depended on the particular circumstances.
In Niger, the impact of the resident tax advisor was judged positively by local officials—one senior official even directly attributed the increased tax collection from large enterprises to his work. Government officials noted his significant contribution to capacity building through his “hands-on” approach and emphasis on training, which could not easily be provided by short-term missions. There was a general sense that the “hands-on” approach taken by resident experts was particularly valuable for Niger, where weak capacity often meant that it was difficult for the authorities to spare qualified staff to attend courses abroad. Obviously, this modality must evolve as institutional capacity improves. In contrast, TA on customs administration delivered through short-term missions was significantly less effective. Most recommendations were not endorsed by officials, because they considered that these recommendations were either based on an inadequate understanding of Niger’s context or were unrealistic.
A similar situation was found in Cambodia. Resident experts were involved not only in basic training, but also played a key role in coordinating efforts within and among ministries. Because of the low level of counterpart skills, experts had to follow up closely the progress in implementation and improvement in the learning process. Sometimes, these experts played a key role in adapting TA recommendations left by short-term missions. For example, in the area of budget execution, the dimension and nature of the payment arrears problem became clear only after the TA resident advisor had spent some time there. A somewhat similar role was played by resident experts in statistics and at the Central Bank. TA was being delivered by peripatetic missions and specific tasks were left to be implemented by the authorities between missions. Whenever they ran into trouble, the timely support of the resident expert allowed the authorities to continue to progress more quickly than otherwise.
In Honduras, many officials expressed strong preferences for having LTEs at the first stage of building institutional capacity.2 It was felt that these experts had more time to adapt their recommendations to the country’s reality and then to follow up on implementation. The officials recognized that once a minimum institutional capacity had been reached, the ideal course was to have more short-term TA missions. This conclusion is supported by the experience of IMF TA to the Bank of Zambia. In the mid-1990s, the IMF provided intensive capacity-building TA through a series of resident experts. Once the institutional foundation was laid, missions and peripatetic experts provided subsequent TA. These means of delivery were considered appropriate to the level of institutional development of the bank by Zambian officials and experts interviewed in the field.
Even in Ukraine, officials recognized the important contribution that LTEs made to the transfer of knowledge. In particular, they helped in collaboration and dialogue across agencies, a challenging task during periods of radical redefinition in the relative role of agencies. This was particularly true of TA in the areas of fiscal management and statistics.
Our conclusion is that the most effective mix between resident experts and short-term missions depends on the level of institutional capacity. Moreover, appropriate modalities for TA delivery should follow a sequence adapted to the evolving level of institutional development (Box 5.1). Neither modality is inherently better than the other. Rather, the most effective modality will largely depend on the specific country context and the IMF should be prepared to adapt the delivery mode to that which appears to work best in each case.
Attributes of a Good Long-Term Expert
Generally, local counterparts and officials were satisfied with the majority of resident experts. In Niger, LTEs in revenue and in PEM were considered particularly effective because of their accessibility, their emphasis on training and teaching, their good communications skills, and their highly interactive, team-based approach to working with officials. Similarly, Cambodian officials from diverse agencies and ministries consistently praised most LTEs, particularly because of their hands-on approach that facilitated knowledge transfer. It was found that when these experts had civil service experience, they were more successful in conveying their messages to local authorities.
In the case of Honduras, there was a unanimous view among officials that the LTEs in banking supervision and tax administration had made critical contributions in their areas of expertise. They were praised for their hands-on approach, their leadership qualities, and their ability to coach and explain. Many examples were given of the experts going well beyond their terms of reference in order to respond to unforeseen problems.
In the case of Ukraine, feedback was more varied. Some LTEs were praised because they were able to improve collaboration across agencies when the relative roles of agencies were being redefined. However, there were differences on how the same resident experts were judged by different officials. One gains the impression that these differences were associated with how politically and bureaucratically sensitive were the recommendations of the specific TA, particularly in the areas of banking supervision and bank restructuring, tax administration, and treasury reforms. Tensions in the area of banking may have been compounded because the same resident expert was in charge of both, providing advice on capacity building as well as following up on program conditionally (Box 5.2).
Box 5.1.A “Life-Cycle Model” of Capacity Building?
The May 1999 Review of IMF Technical Assistance concluded that “in general, TA provided by staff missions appears more likely to be successful than that provided by LTEs” and called for “greater emphasis [to] be placed on the assignment of short-term and peripatetic experts.” However, that evaluation did not clearly link the duration and modality of TA with country characteristics like the stage of development.
At the same time, there has been an acknowledgment within the IMF that—particularly in low-income countries—TA in support of capacity building is best provided in the context of a longer-term framework.1Where a sustained effort is required, IMF staff have called for a careful assessment of the magnitude and types of TA inputs to ensure meaningful and lasting results. Such an approach would need to manage what the staff refers to as the “tensions between the need to show short-term results in the context of an IMF-supported program and the need to provide sufficient time for institution building to take root.”
As a general principle, countries and institutions in the earlier stages of development might benefit from a more intensive approach, with a particular emphasis on “hands-on” training. Among the evaluation’s case studies, Niger, Honduras, and Cambodia provide good examples of countries that have benefited from such an approach. This has implications for the evolution of regional TA centers like the AFRITACs, which provide TA to low-income countries but that may not currently be equipped to provide more than a few weeks of TA to individual countries in a particular area during the course of a year.
Our case studies suggest that as institutional and civil service capacities develop, the approach to delivering TA should also evolve, with less need for an intensive, day-to-day interaction between TA provider and recipient. But since overall capacity would likely still have significant weaknesses, perhaps exacerbated by high turnover of civil servants, a more selective use of LTEs rather than an abrupt shift to short-term missions might still be appropriate.
The recent reviews by FAD and MFD on TA experiences in post-conflict countries also recognize that the first phases of assistance may have to rely heavily on LTE. As time passes, and local capacities increase, these modalities can evolve toward short-term missions or peripatetic visits.2
For countries with relatively well-educated civil services and easy access to a range of technical expertise, IMF TA could be considerably more targeted, drawing to the extent possible on the emerging expertise of local officials, including in the formulation of recommendations. Greater use of short-term activities like seminars and workshops might also be appropriate. This suggests that any strategy to have countries “exit” from IMF TA (as suggested by Executive Directors during the March 2004 Board discussion of the IMF’s TA program) may require a coherent evolution of TA modalities, something that is not often made explicit in discussions between staff and the authorities.1IMF (2004d).2IMF (2004J and 2004k).
In the case of Zambia, the authorities’ appraisal of LTEs varied significantly. They were positive when these experts made special efforts to provide training and hands-on involvement in solving problems. In contrast, they were critical when experts were reluctant to go beyond mere criticisms of existing practices.
A good resident expert seems to be the one able to adapt to the environment of the country in question. Where institutional capacity is particularly weak, the LTE may need to perform the role of coach and teacher. When skills and institutions are more developed, the LTE should be able to explain clearly the rationale for the recommendations and its link to the state of the art on the subject. The ability of the expert to gain and maintain the trust of the authorities is a key quality for effectiveness; for example, the authorities need to feel that the LTE is a “partner” and not a “task master” or a headquarters’ “spy.” However, in seeking to build such a relationship, LTEs need to straddle a fine line and not become too involved in day-to-day operations that will distract them from their ultimate TA responsibilities.
The success of a TA activity delivered through a resident expert depends also on the quality of oversight by the relevant staff at IMF headquarters. Failure to consistently “backstop” an expert—particularly one engaged from outside the IMF—can undermine the effectiveness of TA.3 A number of suggestions have been made to explain why backstopping from IMF headquarters has sometimes fallen short. First, the rotation of staff among tasks and country assignments can undermine the consistency of the oversight of experts in the field. Second, backstoppers generally have their own TA assignments that have tended to take priority in their work since these tasks are seen to contribute more to professional growth and individual performance appraisals. As such, backstopping can easily be neglected because of other pressures on staff time. Related to this, a number of staff with whom we met suggested that the management in functional departments assigns a greater priority to TA delivery than to management and oversight of TA activities in the field.
Box 5.2.Involvement of LTEs in Monitoring Conditionality
A question arises whether it is advisable for resident TA experts—whose focus is usually on strengthening institutional capacity—to be in charge of following up and monitoring conditionality of programs in their specific areas.
This was the case in Ukraine in the area of treasury management and banking sector restructuring, where specific actions were part of the 1998 EFF. Indeed, the terms of reference for the resident expert in banking explicitly refer to his responsibilities in assisting the authorities in meeting their commitments to the 1998 EFF program. Some officials in Kiev were of the opinion that such a role was not useful in developing trust in the resident expert. Sometimes, the perception arose that these experts were assigned to a specific agency not because of the needs of the authorities, but as a response to the IMF’s drive for more and better information of what was happening in that agency and how it might affect the fulfillment of conditionality.
A similar case seems to have emerged in the area of PEM in Zambia. As felt by donors and some staff, the involvement of a resident expert in monitoring conditionality may have reduced the authorities* trust in the expert, thereby undermining his effectiveness in the capacity-building area.
In assessing the appropriateness of having LTEs monitor compliance with IMF conditionality, it should be kept in mind that—from a legal standpoint—only certain categories of experts can serve this function. In particular, LTEs placed by the IMF in central banks are normally considered employees of the central banks themselves. For reasons related to central bank rules on confidentiality of information or conflict of interest, it may not, therefore, be possible for these LTEs to assist the IMF in monitoring the observance of conditionality.
The relevance of backstopping activities is likely to grow if the IMF becomes more actively involved in capacity building in low-income countries. It is, therefore, important that departmental planning makes provision for sufficient backstopping resources and adequate incentives are given to headquarters staff to better manage TA in the field.
Enhancing Ownership of TA Recommendations
A critical step in TA delivery is the formulating of recommendations. A key challenge is to balance technical soundness with implementability while ensuring a high degree of country ownership. Country officials and counterparts suggested that more informal and iterative discussions on a broader set of options prior to the wrap-up meeting at the end of the TA mission would contribute significantly to enhancing ownership.
In Ukraine, counterparts suggested that a range of “models” and alternatives be put on the table prior to deciding on a specific modality or course of action. A similar reaction was seen in Yemen in the area of tax administration, where options based on regional experiences closer to those of Yemen were viewed as useful. In the context of a tax policy mission in Zambia, the authorities felt that some of the recommendations should have been supported with reference to success stories of countries in the region.
A consistent comment from all countries was the need for a more interactive approach, prior to the drafting of recommendations. Many officials suggested that local experts should work closely with IMF experts as recommendations were being formulated that would provide the authorities with the opportunity to discuss and consider alternatives. For example, this would have improved the recommendations in the area of customs administration in Niger, where officials in the customs department argued that TA recommendations did not take sufficient account of constraints on the ground. Cambodian officials felt that short missions did not give them enough opportunity to discuss the appropriateness or practicality of the recommendations before they were finalized. They suggested a better result would have been achieved if the mission had remained in the country longer to allow for more detailed discussions of specific recommendations. Similar concerns were expressed by officials in Honduras and Zambia. The latter pointed to IMF-supported programs that attached conditionality based on TA recommendations that had not been sufficiently discussed at the technical level.
A review of other external and internal evaluations of IMF TA (referred to in Chapter 1, “Methodology and Empirical Basis”) yielded similar findings. Several of these evaluations found that a diagnostic mission of a few weeks was too short to grasp fully all the important institutional nuances and constraints. Implementation difficulties may be underestimated. When the missions are short, very little time is left for the authorities to comment on the recommendations prior to the departure of the mission. TA recommendations need to be discussed extensively and early on to avoid unexpected surprises at the last minute. While the benefits from following the recommendations may seem obvious to the TA experts, they may seem less clear to the country counterpart. Such discussion is critical if broader institutional resistance to the reforms is expected.
Efforts at a Wider Dissemination of TA Recommendations
The evaluation found wide variations in the extent to which the results of TA activities were made available within governments. For example, in Yemen, the dissemination of IMF TA reports within and across relevant agencies appears to have been quite limited. Many interlocutors within the government were unaware of the results of IMF TA activities in areas of relevance to their work. In the case of Zambia, the end-1996 mission report on treasury management, which laid the broad outlines of future TA on PEM, had very restrictive circulation even though its implementation would eventually have involved various departments and administrative levels within the government.
These problems of dissemination were compounded by the fact that many TA reports produced by the IMF were classified as either confidential or strictly confidential, which complicated sharing the reports with development partners. This evaluation finds that the current policy of treating TA reports as confidential and limiting their distribution both within governments and among partner agencies providing assistance in similar areas should be relaxed wherever possible.4 If publication is not possible, at least there should be some up-front agreement with the authorities that direct beneficiaries of TA (within government and relevant partners) be provided with TA reports unless there is a compelling reason for confidentiality (such as assessments of the soundness of individual banks).5
Coordination with Other Providers of TA
Given the scarcity of IMF TA resources and the IMF’s limited expertise in many areas critical to capacity building, coordination with other TA providers (including the World Bank) is important for enhancing the effectiveness of IMF TA. That said, we found many instances in our case studies of weak coordination between IMF and other TA providers working in similar areas. Shortcomings took various forms, including a lack of systematic information sharing and/or communication between the IMF and other TA providers, vague guidance to IMF resident experts and representatives on the parameters of donor relations, and a lack of clarity on the role IMF TA was to play relative to that of other TA providers. These weaknesses were evident even in Cambodia and Yemen, where the existence of TCAPs should have promoted better coordination. A list of examples of these and other coordination problems from our case studies includes:
Yemen, where the TORs for IMF resident experts were not explicit on the extent of, or modalities for, coordination with other TA providers. Reference to coordination rarely went beyond exhortations to “closely coordinate with other donors” but no clear guidance was given on what this entailed. In the absence of an explicit requirement for substantive outreach and communication, the extent of coordination was left to individual experts. A review of reports to headquarters by LTEs showed little discussion of the activities of other providers.
Cambodia, where the lack of coordination among TA providers was frequently noted by local authorities as an obstacle to implementation. The lack of coordination sometimes produced contradictory advice that confused the authorities—and provided the authorities with an opportunity to play the advice of one expert off against the other.
Honduras, where stronger coordination from the start would have increased the complementarity of the activities of the Inter-American Development Bank and the IMF in the area of tax administration.
Niger, where communication between West AFRITAC and donors on the ground was weak, resulting in considerable confusion about the nature of, and expectations for, West AFRITAC’s efforts in the country.
Zambia, where donors, government officials, and experts voiced concern over poor coordination of TA on PEM, including on information sharing with donors working on closely related topics. There was also a lack of agreement as to the extent of IMF’s leadership in shaping a strategic vision for the overhaul of PEM.
Experience in coordinating TA between the IMF and World Bank suggests an important qualification to the above discussion. In our case studies, we generally found better communication between IMF TA providers and the World Bank, with frequent meetings between IMF experts and World Bank staff before and during TA missions. That said, this relatively good communication did not always lead to effective coordination.6 There could be several reasons for this, but particularly notable are differences in mandate, staff incentives, and institutional priorities between the IMF and World Bank that were difficult to overcome in the absence of a clear overall framework on which both institutions would align.
It is widely acknowledged that coordination of aid (including TA) is ultimately the responsibility of the authorities. However, in practice, capacity constraints and scarce human resources often mean that the authorities are not always able to assume this role. (Similar problems were identified in several of the TA evaluation studies mentioned in Chapter 2.) As a result, and particularly in the least developed countries, the burden of donor coordination frequently falls to a single major donor or multilateral institution, with a leading role for the authorities being a somewhat longer-term goal.
This leads to the question of what model should guide IMF staff in coordinating with other TA providers, including the World Bank. For low-income countries, one would expect that the PRS process would provide the appropriate framework, with TA providers, including the IMF, aligning their comparative advantages to the needs emerging from the PRSP. However, as discussed in Chapter 4, there is strong evidence that in most cases the PRS process does not yet provide the necessary information, given its lack of specificity on country-specific capacity-building priorities.7 Indeed, discussions with IMF staff during the course of the evaluation made clear that there is still considerable doubt within some parts of the IMF that the PRS process could or should play such a role.8 At the very least, this suggests that much clearer guidance from management and the Board is needed on what the PRS process implies for the IMF to better align its TA to country needs.
A second possible model would build on the distinction between “upstream” and “downstream” TA. “Upstream” TA is that which sets out a strategic framework which would then guide “downstream” TA (i.e., “projectize” the framework, focusing on specific capacity-building activities necessary for implementation of the framework). Among functional departments within the IMF, FAD adopted three years ago an explicit strategy of focusing its TA resources on “upstream” activities.
At a conceptual level, the IMF’s widely acknowledged expertise in its core areas of specialization makes it a logical leader in setting the “upstream” (macroeconomic) agenda for more operational, “downstream” TA. However, in practice, the upstream/downstream distinction can be problematic and, in any event, can only complement, not substitute for a country-driven approach. Our case studies revealed a wide range of country situations, where different configurations of TA providers, with different levels of engagement, led to differing expectations of what the IMF should deliver in terms of TA. While virtually all TA providers with whom we met appreciated the IMF’s involvement in capacity-building efforts, they have not always been willing to cede the role of “agenda setter” to the IMF, even in areas where the IMF has significant expertise. There have also been situations where major donors have significantly different objectives or motivations from the IMF in providing TA to a particular country. While many of these providers were open to the IMF playing more of an “upstream” role, they expected to be meaningfully involved in the formulation of the overall framework. They often saw value in having some of the “downstream” TA provided directly by the IMF.9 In effect, while there appeared to be an openness to coordination with the IMF, there was often resistance to being coordinated by the IMF (either explicitly or implicitly).
These experiences suggest that a pragmatic approach is needed, recognizing that coordination itself is not without costs and that its potential benefits will depend on the extent and complexity of IMF TA and the degree to which its effectiveness depends on complementary capacity-building efforts. This suggests that the most effective model for guiding TA coordination will depend heavily on country-specific circumstances, including pre-existing donor relationships in areas of mutual interest. In any event, effective coordination would require up-front agreement, or at least clarity, on IMF’s role vis-à-vis other providers. We believe that our earlier suggestion of deriving TA priorities in the context of a medium-term policy framework, with the participation of the authorities and donors, could help early on to guide the discussion with other providers on an appropriate division of labor.
How proactive should the IMF be with respect to donor coordination when there is not yet an overall country-driven framework in place? At a minimum, we see value in routinely providing timely information to donors on TA mission schedules and on the purpose of each mission. This would give interested donors sufficient opportunity to signal to IMF experts their interest in the work of the mission and, if warranted, to prepare meaningful input for the TA mission. Our case studies suggest that while donors were routinely informed of area department missions, they were infrequently aware of visits by TA experts or missions in areas of relevance to their work. TOR for TA activities should, therefore, provide clear guidance on what is expected from IMF experts in this regard.
Conclusions and Recommendations
The major conclusions and recommendations from the earlier analysis are as follows:
Generally, local counterparts in the country have been satisfied with the majority of resident experts and valued particularly their hands-on role in training, coaching, and emphasis on team work.
The evidence does not support the conclusion of the 1999 evaluation that TA provided by short-term staff missions is a priori likely to be more effective than that provided by resident experts. The effectiveness of different delivery modalities depends on country circumstances, including a country’s stage of institutional development.
For large multiyear TA activities, and particularly in countries where the IMF is less familiar with the overall institutional environment, we recommend that the IMF spend more time up-front learning about the political and institutional environment. The IMF should avoid rushing into a massive TA program—for example, as a response to the sudden availability of externally financed TA—where there are serious doubts about the political and/or administrative absorptive capacity of the country in question.
It is recommended that the authorities and officials of the agencies involved be required to invest more of their own time and effort in articulating requests and formulating TOR, particularly for resource-intensive TA. This would signal their ownership and commitment to making effective use of the TA. IMF staff also needs to spend enough time interacting with the authorities early on to explain the rationale for the TA recommendations.
It is recommended that, for these large activities, the authorities be required to commit resources to sustain the activity after the financing by the IMF (and other donors participating) ceases. This is critical to encourage sustainability. Depending on the specific TA activity, the IMF staff may also request the authorities’ commitment to prepare critical legislation needed to implement the TA, some assurances of employment stability for key counterparts, and other measures.
We recommend that the IMF allow more time for TA experts to interact with authorities during missions including in formulating recommendations. This may require reassessing the adequacy of the time that the mission stays in the field, even if it results in fewer individual TA activities.
We recommend that IMF staff make more deliberate efforts to disseminate the recommendations of TA within relevant government departments and among development partners. This is important in order to maximize the potential benefit of their work, overcome poor internal communications, and counteract the loss of institutional memory because of high turnover of local counterparts. This will likely require permitting more liberal access to IMF reports within governments.
We suggest that the functional departments review the adequacy of staff resources dedicated to backstopping experts in the field and that they provide headquarters staff with sufficient incentives to ensure that actual time spent backstopping is consistent with resource needs.
An interesting and (very) recent example is the review by FAD and MFD on their experience in providing TA in post-conflict countries. An important part of the review is to derive lessons on sequencing TA activities in the respective areas of responsibility (IMF, 2004j and 2004k).
Clearly, such expressions of preferences are not based on specific cost comparisons and should not be regarded as providing a definitive conclusion on cost-benefit comparisons. Many authorities would probably opt for a more expensive LTE. However, in many cases, those interviewed pointed to specific contributions that could not have been made easily by short-term missions, whatever the cost.
Yemen presents one such example. It was found after three years that the first budget expert had deviated significantly from his terms of reference. Similarly, the failure to adequately supervise AFRISTAT’s assistance to Niger likely resulted in inadequate coordination with the area department and inadequate attention from the regional expert to particular capacity-building priorities.
It is the responsibility of the respective government, not the IMF, to decide on the appropriate distribution of a TA report.
STA has been particularly proactive in this regard, routinely requesting the authorities to agree to the dissemination of TA reports to other relevant government agencies and TA providers. This approach could be adopted as a “best practice.”
TA in PEM in Yemen provides a particularly good example of this phenomenon. Here, IMF and Bank approaches and objectives were not well integrated at the outset. This led IMF staff to express concern with the Bank approach in internal communications and resulted in each institution working relatively independently of each other.
The question of alignment between IMF activities and the PRSP is discussed more extensively in IEO (2004).
For example, a common argument we heard from some functional departments was that, since the PRSP is meant to be a country-authored document and many authorities lack the necessary technical expertise to articulate a specific TA strategy in support of their policy priorities, it is not realistic to expect the PRSP to play such a coordinating role. In our view, these arguments miss one of the central points of the PRS initiative—namely that it was designed precisely to improve the effectiveness of aid delivery by putting a country-driven process at the center of decisions on priorities, with the expectation that donors would align on those priorities. Technical expertise from the Bretton Woods institutions or other donors was meant to help inform, but not substitute for, that domestic process.
IMF involvement in these TA activities was often welcomed by government officials who saw the IMF as playing more of an honest broker during the operationalization of overall reform agendas.