International Civil Aviation Organization

Evaluation Function

The Evaluation and Internal Audit Office (EAO) is located in the Office of the Secretary General, and is responsible for the systematic evaluation and audit of ICAO's programmes, projects and activities, as well as investigations into allegations of financial wrongdoing. EAO is also mandated to act as focal point for the Joint Inspection Unit (JIU) and in relation to the audits of the External Auditor. Additionally, EAO provides secretariat services to the Evaluation and Audit Advisory Committee (EAAC).

The purpose, authority and responsibilities of EAO are defined in the Charter for the Evaluation and Internal Audit Office. The ICAO Evaluation Policy, approved by the ICAO Council in 2014, complements the Charter, and further defines the aims, principles, and planning process of the evaluation function, as well as its role in decentralized evaluations and quality assurance.

In 2015, the evaluation function is composed of the Chief, EAO (part-time), one Evaluation Officer, and a support staff (25%). EAO's evaluation expenditure not related to staffing (consultancy and mission travel) during 2014 was approximately CAD 38,000. There are no dedicated decentralized evaluation staff. Nevertheless, some organizational units attempt to conduct decentralized evaluations. EAO intends to offer support to such evaluations, within the limits of its capacity, in order to improve their quality. 

Promoting a culture of evaluation in-house

EAO staff members participate in outreach activities in order to raise awareness among staff on the role of evaluation function at ICAO. EAO also provides informal briefings to Council members and senior managers to highlight and promote the value of the evaluation function.

Snapshot

Evaluation policy

• The Charter for the Evaluation and Internal Audit Office was approved in 2009 and revised in 2012. It is subject to periodic review by the Evaluation and Audit Advisory Committee. The ICAO Evaluation Policy, approved by the ICAO Council in 2014, complements the Charter, and further defines the aims, principles, and planning process of the evaluation function, as well as its role in decentralized evaluations and quality assurance.

Priorities:

• Implementation of the ICAO Evaluation Policy

• Development and monitoring of key performance indicators for the evaluation function

• Preparation of an Evaluation Manual

• Provision of quality assurance and support to decentralized evaluations, within the limits of EAO capacity

Human Resources (for 2015)

• Director: 1 (M)

• Evaluation Officer: 1 (F)

• Support staff (Evaluation and Audit Associate): 1 (M)

• Decentralized evaluation staff : None

Evaluations produced per year (2014) by central unit and by decentralized units

• 3 evaluations produced by central unit, of which :

  • 2 thematic evaluations

  • 1 project evaluation

Key resources:

• ICAO Website (www.icao.int)
• Charter for the Evaluation and Internal Audit Office
• ICAO Evaluation Policy

Independence

The evaluation function of ICAO is co-located with the audit function in the Evaluation and Internal Audit Office (EAO).

The office Head reports directly to the ICAO Secretary General and has free access to the Chair of the Evaluation and Audit Advisory Committee (EAAC).

Agenda Setting & Evaluation Planning

EAO maintains a register of potential evaluation topics, and prepares an annual evaluation work plan on the basis of an assessment of risk, evaluability and strategic importance and consultations with the Full Senior Management Group, the External Auditor, the EAAC and Representatives on the ICAO Council. The work plan is submitted to the ICAO Secretary General for his/her consideration, and presented to the Council in an information paper. The Secretary General, the President, and/or the Council can, at any time, request EAO to carry out specific evaluations in areas of particular concern, for which appropriate resources are available. The Chief, EAO may decide to accept or decline such requests.

Stakeholder involvement

Stakeholders, including the national counterparts, where appropriate, are systematically consulted throughout the evaluation process (planning and design, conduct and follow up). Key informants are invited to participate in interviews, peer reviews and/or reference groups during the evaluation process. Important internal stakeholders are often invited to form an Evaluation Reference Group to oversee an evaluation, to provide technical input and quality assurance of the evaluation analysis, findings and recommendations. External expertise may be sought for the purpose of quality assurance, where technical knowledge is required.

Quality Assurance

EAO adheres to the UNEG Norms and Standards for Evaluation in the UN System. Where advanced technical knowledge is required relating to civil aviation, EAO may engage temporary external expertise to support evaluations. The ICAO Evaluation Policy provides the evaluation function with a mandate for providing quality assurance services to decentralized evaluations, within the limits of its capacity.

Use of Evaluation

Upon completion of data collection and analysis, EAO prepares a written report describing the objectives, findings and conclusions of the evaluation and recommendations for improvement. The report is reviewed by the manager of the unit evaluated, and for completion of an action plan to implement the recommendations. The final report and action plan are submitted to the Secretary General for endorsement and subsequently they are presented to the ICAO Council by the Chief, EAO.

Managers are required to systematically follow-up on the implementation of recommendations that have been accepted by the Secretariat. EAO monitors the status of the implementation of recommendations and reports on this periodically to the ICAO Secretary General and Council.

Evaluation results are disseminated within the Organization and lessons are systematically extracted and communicated through meetings with senior management and operations management, and via annual reports of EAO.

Joint Evaluation

EAO is not involved in joint evaluations, primarily due to the technical nature of the work of the Organization.