United Nations Office on Drugs and Crime

Evaluation Function

The Independent Evaluation Unit (IEU) is responsible for leading or managing evaluations in UNODC, contributing thereby to UNODC’s accountability framework. While evaluations are core to IEU’s mandate, emphasis is also on offering innovative evaluation knowledge products, as well as follow up on the GA resolution on building national evaluation capacity. The UNODC evaluation policy can be accessed at UNODC's evaluation webpage. The policy was first adopted in 2004, updated in 2007 in light of the UNEG Norms and Standards and a fully revised policy was published in 2015.

The two types of evaluations at UNODC are In-Depth Evaluations and Independent Project Evaluations, carried out by independent external evaluators who work closely with  IEU, as the unit provides clearance at all steps of the evaluation cycle. In-Depth Evaluations are strategic evaluations of relevance to the organisation, such as country, regional, thematic, global programmes, cross-cutting issues, for example gender, human rights, etc., or corporate policies and are led by IEU’s evaluation officers.  

According to UNODC evaluation policy, handbook and guidelines, all projects and programmes in UNODC are to be evaluated at least every 4 years or 6 months before the project or programme is completed. IEU monitors the portfolio of UNODC from an evaluation cycle perspective and has to this effect invested significantly in building and maintaining IT tools. The work of IEU is based on three main pillars, namely 1) Evaluation Capacity Building which involves external, especially Member States and International Organisations, as well as internal capacity building; 2) Evaluation Knowledge Products, which encompasses strategic analyses based on conducted evaluations (for example the first UNODC Evaluation Meta Analysis 2011-2014) as well as cooperation mechanisms for sharing best practices; and 3) Evaluation Results  that include independent project, programme as well as  strategic evaluations. IEU works with an external company, which independently assesses the quality of all evaluation reports. Further areas of work of the Independent Evaluation Unit include continuously developing its normative tools, providing quality assurance for the whole process of Independent Project Evaluations, managing and participating in In-Depth Evaluations conducted by external consultants, as well as developing an evaluation culture through the design of training and capacity development initiatives within and outside of the organization.

Promoting a Culture of Evaluation within the UN System

As part of its objective to institutionalize an evaluation culture throughout UNODC, the IEU is continuously developing and revising tools and guidance on evaluation. The knowledge produced in form of the evaluation reports, in particular the recommendations and lessons learned, is compiled, synthesised and disseminated by IEU in order to inform Senior Management, UNODC’s stakeholders, Member States and the United Nations Organisations at large. This, in addition to training workshops both at headquarters and in the field, fosters continuous learning and development. Furthermore, the unit is also currently developing and implementing a  capacity building strategy for national evaluation capacity.  Moreover, IEU is coordinating its evaluations, to the extent feasible,  with UN-wide Oversight Bodies (OIOS, JIU as well as BoA). 

Snapshot

• Evaluation Policy

  • UNODC's Evaluation Policy (fully revised in 2015) is available online through the UNEG website and the UNODC evaluation webpage.

• Priorities

  • Utilization-focused evaluations

  • Transparency and participatory evaluations

  • Evaluation conforms to internationally accepted standards

  • Quality assurance

• Human Resources

  • Chief of IEU: F

  • Evaluator : M=1

  • Associate Expert: F=1

  • Support staff : F=2, M=1

  • No decentralized evaluation staff in the field

• Evaluation expenditure in 2015 (excluding staff)

  • The respective evaluation budget has to be reserved within the projects/ programmes to be evaluated. The UNEG peer review concluded that “although an accurate full analysis is missing of the total amount that UNODC spends on evaluations, i.e. staff costs plus the direct costs for IPEs and IDEs, the Peer Review Penal analysis shows that UNODC dedicates approximately 0.5% of its total financial resources to evaluation”.

  • Approx. US$24,700 extrabudgetary funding for IEU travel, consultancies, etc.

• Evaluations produced in 2015

  • 24 (13 Independent Project Evaluations + 11 in-depth evaluations)

• Key resource: web link/key document here

  • Evaluation step by step, UNODC Evaluation Handbook 2011 (in the course of being updated, to be published in 2017)

  • Homepage at http://www.unodc.org/unodc/en/evaluation/index.html?ref=menutop

Independence

The IEU is located independently from the management structure under the Office of the Executive Director. The head of IEU reports directly and simultaneously to the Executive Director of UNODC, Senior Management and to Member States. The head of evaluation also has full authority to issue evaluation reports and  advises project managers on the budget to be reserved under a specific budget line for evaluation. Evaluators are screened for potential conflict of interest but  there is no mechanism in place that would protect evaluators who report cases of wrongdoing.

Agenda Setting & Evaluation Planning

In line with its evaluation policy, the IEU prepares an annual evaluation work plan (non-costed), which is presented to the Executive Director and Member States. The plan outlines the type, timing and budget of respective evaluations. It provides an overall framework and allows scheduling and prioritisation of evaluations, while allowing flexibility and responsiveness to evolving needs with provision for changes when required.

Stakeholder Involvement and Promoting National Ownership

According to the IEU evaluation policy and handbook, stakeholders are invited to participate in interviews and surveys and include the so-called Core Learning Partners (CLP). The CLPs are the main stakeholders , i.e. a limited number of  those deemed as particularly relevant to be involved throughout the evaluation process. The CLP members are engaged from the start of an evaluation i.e. in reviewing and commenting on the draft TOR and the evaluation questions, reviewing and commenting on the draft evaluation report, as well as facilitating the dissemination and application of the results and other follow-up action. Overall, UNODC's evaluations strive to pay particular attention to gender dimensions and inclusion of vulnerable groups in the evaluation process. Furthermore, since 2015 and in line with the GA Resolution (A/RES/69/237) as well as the Follow-up Mechanism to the SDGs, IEU is actively contributing to building national evaluation capacities, especially in the specific fields of UNODC’s mandates.

Quality Assurance

UNODC's Independent Evaluation Unit (IEU) assures the quality of its evaluations through developing guidelines and templates  to be mandatorily used in the evaluation process, further ensuring that all evaluations are undertaken in line with UNEG and UNODC evaluation norms and standards. IEU commissions, manages and implements In-Depth Evaluations in consultation with respective project managers, ensuring the overall quality, validity, relevance and usefulness of the evaluation deliverables. The role of IEU for Independent Project Evaluations is one of quality assurance, where the entire process is backstopped, IEU reviews, comments on and clears all the steps and deliverables of the evaluation, in line with a quality criteria checklist. The IEU has developed various guidance materials as well as an on-line evaluation application to facilitate evaluation management, linked to the Programme and Financial Information Management system (ProFi) of the Organization The Evaluation Handbook (undergoing a full revision in 2016) refers to the OECD-DAC criteria, UNODC's own adapted criteria and criteria related to gender and human rights in evaluation. 

Furthermore, IEU’s work is regularly monitored and assessed through OIOS and JIU. In addition to these official monitoring entities, IEU hires on a regular basis an external consultancy firm in order to have an independent, rigorous assessment of all published evaluation reports. This means, once an evaluation is finalized, it is also submitted to an external quality control by an external company, which independently assesses the quality of the report.  Moreover, IEU takes part in the SWAP reporting led by UN WOMEN in order to assess its compliance and progress in mainstreaming gender equality in all evaluations.

Use of Evaluation

UNODC's evaluation unit uses a Management Response, and an Evaluation Follow-up Plan as part of its follow-up to evaluations. Specifically, an evaluation follow-up plan (EFP) is prepared for implementation of recommendations. It contains the entity to which the recommendation is addressed, the management response, actions taken, contact details of the responsible person, the timing, and status of follow-up. The project manager has the responsibility to fill in the EFP  in the on-line evaluation application. While Senior Management makes sure that recommendations are implemented, the IEU is responsible for monitoring the status of follow-up on an annual basis. Monitoring reports on the implementation of recommendations are sent to Senior Management and the Executive Director. Moreover, IEU is monitoring the use of evaluation results in project and programme documents and revisions as well as through annual feedback questionnaires to Project Managers as well as Senior Management.

Evaluation reports are made available to the public by IEU through a repository of evaluations on IEU's website, fully remodelled in 2011 as well as through the  on-line evaluation application. Other means of dissemination include preparing Evaluation briefs as well as conducting workshops. In parallel, project managers may distribute evaluation reports to their stakeholders. Overall, in disseminating evaluation reports to Member States, Senior Management and other stakeholders, the information needs of different users are considered. 

Joint Evaluation

UNODC has been a founding member of UNEG and engages actively in joint evaluation activities and exchange of good practices, especially in collaboration with other Vienna-based agencies.