Chapter 3. Enhancing the Follow-Up Process for IEO Evaluations
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- August 2014
Under the framework established following the 2006 external evaluation of the IEO, responsibility for follow-up on IEO evaluations was assigned to IMF Management. The IEO’s informal role has primarily been to assist the Executive Board to oversee this process, mostly by providing information. The process involves two main instruments: Management Implementation Plans (MIPs) and Periodic Monitoring Reports (PMRs). IMF Management is expected, soon after the Executive Board discussion of an IEO evaluation report, to present to the Board for its approval a forward-looking MIP to implement evaluation recommendations endorsed by the Board. The implementation status (and any necessary remedial or substitute actions) are then to be summarized in an annual PMR, until recently prepared by IMF staff, for Board review and assessment.
This chapter reports on the follow-up on IEO evaluations during FY2014, including approval of the MIP for the IEO evaluation of The Role of the IMF as Trusted Advisor, and new procedures and initiatives by the IMF and the IEO in response to concerns about the follow-up process raised by the 2013 external evaluation of the IEO.
Implementation Plans and Monitoring Reports
IMF staff issued a proposed MIP for the Trusted Advisor evaluation on April 26, 2013. Following a discussion by the Evaluation Committee, the Executive Board agreed to the plan on June 6, 2013. The implementation plan proposes measures to enhance the role of the Fund as a trusted advisor, building on ongoing reform initiatives that are aligned with the recommendations of the evaluation. The measures include: actions to enhance consultations with country authorities ahead of policy discussions; improvements to the management of human resources to strengthen relations with country authorities; and regular surveys of country authorities and Offices of IMF Executive Directors to assist in monitoring progress on these measures. A proposed MIP for the IMF Forecasts evaluation was pending as of the end of FY2014.
In deference to a new proposed process discussed further in the next section, no PMR was issued in FY2014. The last PMR was issued by staff in September 2012 and discussed by the Evaluation Committee in February 2013.
Implementing the Recommendations of the Second External Evaluation of the IEO
As noted in last year’s Annual Report, a key issue highlighted in the 2013 external evaluation of the IEO concerned the structure of the follow-up process established following the 2006 external evaluation. The 2013 external evaluation report identified four major problems with the follow-up process: a lack of strong ownership by the Board; conflicts of interest for Management; lack of capacity to respond to the broader, more substantive recommendations in IEO evaluations; and its bureaucratic nature.
When the Board discussed the external evaluation report in March 2013, Directors endorsed many of the report’s recommendations for further enhancing the effectiveness of the IEO, including steps to facilitate a more accurate recording of the outcome of Board discussions of IEO evaluations and to monitor actions taken to implement Board-endorsed recommendations. On February 10, 2014, the Board approved staff proposals to implement these recommendations.
Application of the rule of silence during Board meetings on IEO evaluations. The 2013 external evaluation report noted a lack of common understanding on how an Executive Director’s silence on specific IEO recommendations was interpreted in recording the outcome of the Board discussion. To address this concern, the Board approved a new procedure whereby the IEO evaluation report would list recommendations that require Board endorsement; the Management Statement issued ahead of the Board discussion of the evaluation report would list all of the recommendations as written in the IEO report and state Management’s support or reservations clearly; and Executive Directors’ silence would then be interpreted as supporting the chair’s position outlined in the Management Statement.
Preparation of Periodic Monitoring Reports. The 2013 external evaluation report pointed out that the approach of having IMF staff prepare PMRs could involve a conflict of interest and argued for separating IMF Management’s implementation and monitoring functions. Accordingly, the Board agreed to task the Office of Internal Audit (OIA) with preparing future PMRs, given OIA’s mandates, its experience in monitoring and assessment of reforms, its arm’s-length relationship with Management, and the capacity and skills base of its staff.
The Board also approved proposals to increase interactions between the IEO and the International Monetary and Financial Committee during the Annual and Spring Meetings and on minor amendments to the IEO terms of reference. All approved changes are expected to be reviewed within two years.
One of the criticisms of the follow-up process made by the 2013 external evaluation was that the monitoring of implementation of IEO recommendations had become a very bureaucratic (“box-ticking”) exercise and lacked any independent assessment. The external evaluation saw a need for two types of monitoring reports. In addition to the PMR, which monitors the implementation of MIP actions, a separate report would be needed to monitor the more generic and substantive concerns raised by IEO evaluations and shared by the Board but which are often not encapsulated in specific recommendations. The external evaluation suggested that the latter report—an “issues-oriented review of the extent to which IEO recommendations have been implemented”—be prepared for the Board periodically by the IEO.
In response to this suggestion, the IEO prepared a report on Recurring Issues from a Decade of Evaluation, which identifies generic and substantive issues from the IEO’s first twenty evaluations and assesses where they stand. As noted in Chapter 2, this report was submitted to the Board Evaluation Committee in April 2014.
Revisiting Past IEO Evaluations
The IEO has launched a new series of reports that revisit past IEO evaluations five to ten years after they were first issued. The initial impetus for this new series came from inquiries from Executive Directors and member country authorities on the status of issues raised in past evaluations. The 2013 external evaluation of the IEO also found a strong case for revisiting some of the key issues that were the subject of past evaluations.
In FY2013 the IEO issued two pilot reports revisiting, respectively, the 2002 evaluation of prolonged use of IMF resources, and the 2003 evaluation of fiscal adjustment in IMF-supported programs. Based on this experience, the IEO decided to continue this initiative and in FY2014 issued an update on the 2005 evaluation of IMF technical assistance; a summary of this report is contained in Chapter 2. These evaluation updates are brief stock taking exercises, more modest in scope than full IEO evaluations but broader in coverage than PMRs. They summarize the original IEO evaluation, describe relevant developments that have taken place since the evaluation (including, but not exclusively, with respect to the implementation of the IEO’s recommendations), and identify outstanding and new issues related to the evaluation topic that merit continued attention. All three reports are available on the IEO website. Two more updates are in the pipeline, as discussed in Chapter 4.