Ethiopia: Final Evaluation Consultancy Service

Project “Universal and equal access to quality health services, to meet the health needs of women and children in the Liben Zone (Somali Region, Ethiopia)” AID010595/CCM/ETH

Location: Filtu and Dekha Suftu Woreda (Somali Region)

Consultancy period: Between 15^th August to 15^th of September

Consultant service: 8.000€ (Euros) including flight, business visa and insurance

CCM provides accommodation and transport

Application documents: CV, Cover letter, methodology and final proposal

Deadline for application: 24^th July 2019 atrecruitmentpvs@ccm-italia.org

ONLY short listed candidate will be contacted

Background information

CCM, Comitato Collaborazione Medica, is a non for profit nongovernmental organisation founded in 1968 in Turin, by a group of medical doctors from Piedmont Region in Italy.

Our key mission is to promote the right to health and ensure access to essential health care.

· We are a lay and independent organisation, guided by the values of solidarity, equity, non-discrimination and cultural respect

· We work through long term development projects, believing that health can be really promoted only with stable and durable interventions

· In case of natural disasters or epidemics in the areas where we work, we implement emergency interventions

In regards to Ethiopia, CCM is registered in the country since 1998. CCM projects in Ethiopia aim at contributing to the promotion of a good health and quality of life for the poor and vulnerable people in the most remote areas of the country.

CCM is currently implementing the project “Universal and equal access to quality health services, to meet the health needs of women and children in the Liben Zone (Somali Region, Ethiopia)” started the 1^st January 2016 and planned to be concluded on 31^st October 2019.

The project is funded by AICS – Agenzia Italiana per la Cooperazione e lo Sviluppo (Italian Agency for the Development Cooperation), CCM own funds and other donors.

The project aims at ensuring women and children (extremely vulnerable groups in a semi-nomadic context, with a patriarchal tradition) access to quality primary health care services, through their active involvement. The intervention meets the needs identified by CCM, Filtu and Dekha Suftu Woreda HealthOffices and recognized by the Somali Regional Health Bureau (SRHB), which put maternal, child and reproductive health among their priority actions for sustainable development in the area of intervention. In agreement with the institutional counterparts, the action proposes an integrated approach that strengthen service delivery at facility level and increase the demand of care at community level. The project focuses on both health services supply and demand, strongly involving the civil society and making beneficiaries and local organizations active protagonist of the action, in order to maximize efficiency and impact and ensure sustainability for the action.

The Overall Objective is to contribute to the improvement of health in Filtu and Dekha Suftu districts trough Primary Health Care integrated strategy to ensure continuity in treatments at both community and health facility level.

The Specific Objective is to improve the access to integrated maternal, child and reproductive health services (RMNCH) for women and children of Filtu and Dekha Suftu districts, and to make it universal and equal in order to increase the number of people benefiting from health services, favouring vulnerable groups and disadvantaged persons.

In order to achieve the above-mentioned objectives and make women active in their search for preventive and curative services at health facilities, the project seeks to:

· strengthen the prevention and care system at primary level, improving the quality of services in the health facilities and in the community;

· increasewomen’s trust in the health system and in health workers, involving Community Influential Leaders in the promotion of maternal health;

· identify sustainable strategies to increase the access to services, by creating community health networks formed by community influential leaders and health workers and through mass awareness campaigns;

· reinforce the capacities oflocal health authorities (WoHOs), by increasing their participation to the supervision of primary services and to the community coordination.

The specific objective is meant to be achieved through three Expected Results (ERs):

· ER 1) The maternal, child and reproductive health services (MCRHS) are improved for the benefit of the whole population of Filtu and Dekha Suftu districts, without discrimination and with a special attention to the most vulnerable persons (pregnant women, women in the reproductive age and children, displaced persons and pastoral communities);

· ER 2) The maternal, child and reproductive health integrated services medical coverage is expanded trough the strengthening of the reference system and the active involvement of the communities;

· ER 3) Local health authorities are trained and able to programme, supervise and manage the services provided by the HFs.

The expected results and the project objectives are planned to be achieved through several activities, including a preliminary needs assessment to ensure a proper planning of project activities; the training of health workers in maternal and child health and in the management of complicated pregnancies and deliveries; the purchase and distribution of essential drugs and medical equipment; the rehabilitation of health facilities (minor rehabilitations), the creation of community network groups involving influential leader, Health Extension Workers, religious and kebele leaders, TBAs; the implementation of awareness raising campaigns on MCH and the continuous capacity development of local stakeholders to ensure proper ownership of the project. The proposal, designed and developed in close collaboration with the local health authorities, is based on the results achieved during previous projects implemented in the same area, also financed by AICS and the European Union.

Scope of the service

The service required is meant at producing the FINAL evaluation of the project “Universal and equal access to quality health services, to meet the health needs of women and children in the Liben Zone (Somali Region, Ethiopia)”.

The objective of the evaluation is to fully review and assess the results achieved by the project during the period of implementation, as well as its impact and sustainability.

Moreover, the evaluation shall become a valuable exercise to generally assess CCM strategy and approach in the area of intervention, with specific reference to maternal and child health care. Recommendations and findings from the final evaluation will serve to support the planning and designing of any future actions in the same and similar context.

The external evaluation aims at getting a critical analysis on the project life cycle. It will therefore analyze and elaborate deeply on the strengths, weaknesses, opportunities, constraints and lessons learnt from the project, in order to:

• Evaluate the relevance of the action, compared to project objective and assessed needs;

• Evaluate the efficiency in the utilization of the resources availed by the donor;

• Evaluate the effectiveness of the action carried out;

• Evaluate the impact of the project in the catchment area;

• Evaluate the sustainability of the project;

• Evaluate lessons learnt and best practices, to be capitalized and replicated elsewhere.

Suggested methodology

The evaluation process should be impartial, independent must be as open as possible with respect to the results achieved and made available.

It is suggested to use a mix of qualitative and quantitative techniques internationally accepted and scientific tools based on verifiable data and information collection.

The evaluation must involve the beneficiaries, the local counterparts and institutions, donors and all the stakeholders able to provide learning that might be used to lead to a better efficiency and effectiveness in achieving shared results. It is recommended that the methodological tools are designed and adapted according to the features of each beneficiary group involved. In particular, the methodology must be gender-sensitive and ensure a balanced representation and participation by all community groups.

Data and secondary information shall be both provided by the contracting authority upon consultant/firm’s request and directly collected during the field visits through the interviews with any responsible bureau and/or community group. The reliability of these sources can be subject to verification with any concerned entity, including the donor, local authorities, health facilities, other stakeholders and service provider agencies. Despite the shortcomings often identified in the system, data from the existing Health Management Information System (HMIS), adopted by the MOH at national and regional level, may also give an overview of the health services status in the area. These are the data used to set the baseline reported in the logical frame work of the project document and monitor any progress.

The evaluation techniques may include structured interviews, open-ended questionnaires, direct field observations, analysis and review of reference material and documents supplied by CCM in Filtu (Somali Region), Addis Ababa and Torino (Italy). Secondary data sources may also be consulted when appropriate in the interest of this final evaluation.

Other methodologies might be employed if considered necessary. Nevertheless, the final methodology designed by consultant should be discussed and agreed with CCM and, if needed, with the donor, before commencement of the final evaluation.

In any case, the methodology should at least include:

• In-depth review of the project documents; progress technical, financial and activity reports including monitoring reports by the NGO, AICS and any other actor;
• Field visits (direct observations) to project site including focus group discussions and participatory assessment with the targeted beneficiaries, project staff, stakeholders, bureaus and local authorities, and collection of additional material through semi-structured interviews;
• Briefing/debriefing meetings with all project stakeholders, the implementing NGO country office/the project team, local government partners at regional and district (“woreda”) levels, leaders of the respective beneficiary communities, other actors including NGOs implementing similar actions in the intervention area.

The final evaluation will place greater emphasis on the outcomes and impact, rather than the output and processes, of project implementation.

The final evaluation shall, at least, focus on the following aspects:

Relevance and quality of project design

What is the present level of relevance of the project?

Has the project overall and specific objectives consistent with, and supportive of Partner Government policies?

How appropriate have the project objectives been?

Have the project activities been the best way to achieve the objectives?

If not, which could have been the alternative options?

Did the project respond to the needs of the target groups?

Had key stakeholders been involved in the design process?

Have the project’s objectives and outcomes been clear, practical and feasible within its time frame?

Efficiency of implementation

How well is the availability/usage of means/inputs managed?

How have the resources and budget been used?

Had the project the appropriate financial controls, including reporting and planning, that allow management to make informed decisions regarding the budget and allow for timely flow of funds?

To what degree have been inputs provided / available on time to implement activities from all parties involved?

Have project resources been managed in a transparent and accountable manner?

How are the monitoring tools?

How are the implementing parties’ capacities in the project implementation?

Are the parties able to perform the responsibilities entrusted to it?

Details required: - How? Where? When? Which Strategies adopted? Interventions? Supervision? Monitoring & evaluation?

Effectiveness

How well has the project achieved its planned results during this time?

What extent of the expected results has been achieved?

What is the quality of the results/services available?

Are all planned target groups accessing to / using project results available?

To what extent has the project adapted to changing external conditions (risks and assumptions) in order to ensure benefits for the target groups?

Are the activities carried out timely and effectively?

How the project fits within the general CCM strategy for the area and the maternal and child health care sector.

Impact

What are the direct impact prospects of the project at Overall Objectives level?

What is the overall impact of activities undertaken to target specific objective highlighting the major contributory factors for positive and negative aspects?

Are there any social or political risks that may jeopardize the sustainability of the project outcomes?

Do the various key stakeholders see that it is in their interest that the project benefits continue to flow?

What were the threats? How have these been addressed? Could these have been addressed in a different way?

To what extent does the project have any indirect positive and/or negative impacts?

Have there been any unplanned positive impacts on the planned target groups or other non-targeted communities arising from the project? How did this affect the impact.

Sustainability

What is the level of ownership of the project by target groups and will it continue after the end of external support?

How far the project is embedded in local structures?

What is the likelihood that target groups will continue to make use of relevant results?

What is the risk that the level of stakeholder ownership will be insufficient to allow for the project outcomes/benefits to be sustained?

What is the level of policy support provided and the degree of interaction between project and policy level?

Is there sufficient public/stakeholder awareness in support of the long term objectives of the project?

Are there any obstacles hindering project sustainability?

How well is the project contributing to institutional and management capacity?

How far is the project embedded in institutional structures that are likely to survive beyond the life of the project?

Are project partners being properly developed (technically, financially and managerially) for continuing to deliver the project’s benefits/services?

Financial / economic viability?

Are the services/results affordable for the target groups at this point of the project?

Lessons Learnt & Best Practices

What are the best practices and lessons to be learned from the project implementation?

What should have been done differently? What could be improved for similar projects in similar contexts?

How did the context and local dynamics influence the implementation of the community activities?

What recommendations could be given to boost the action and the results in similar contexts in the future?

Gender

To what extent the project took into account gender dimension?

What was the share of women and men as direct beneficiaries of project?

Do project reports incorporate gender dimensions, progress and results?

Describe how differences, needs, roles and priorities of women, men and other relevant groups are considered

Expected Outputs

• Short presentation on the study approach, sampling technique, size and location and organization of field work, to be discussed and validated with the project management team before the beginning of field work;
• Brief updates (PPT presentation) about the preliminary findings of the evaluation mission in order to share it with the team and to gather inputs from the staff and from the different implementing partners before leaving the project intervention site.
• Final consistent high-quality evaluation report along with the copy of data gathering tools used, significant abstract of interviews highlighting the most important concepts of the evaluation and pictures collected during the evaluation;
• Soft copy of raw data, as collected during the field visits;
• All documents must be provided in soft copy in English.

Users of the evaluation report

Main users of the evaluation will be:

· CCM and its counterparts;

· Italian Agency for Development Cooperation – AICS.

Timeframe

The evaluation shall be conducted during the period between the 15^th of August to the 15^th of September and must be submitted and approved by CCM within 45 days from the completion of the field visit

Proposal submission

In response to these terms of reference, potential consultants are requested to submit a detailed technical and financial proposal outlining how they propose to address the evaluation objectives to the commissioning manager at CCM. The proposal should outline the following:

• Detailed evaluation methodology and approach;
• Report outline and suggested content, i.e. key section and areas under that sections that should be addressed by this evaluation;
• Detailed Work Plan and calendar;

• Financial proposal split into main headings, taking into consideration that the total amount of budget for the consultant serviceis 8.000 €, including flight, business visa and insurance. Please consider thatCCM will provide the following:**

• Transfer from Addis Ababa to Filtu (Somali Region);

• Transport in Filtu and Dekha Suftu Woreda (Somali Region);

• Accommodation in Filtu (Somali Region) and in Addis Ababa;

• CVs of evaluators, detailing qualifications and experience appropriate to the objectives above;

• 3 References of previous employers for similar work;

· Portfolio of previous evaluations, preferable similar to the project presented.

Required Qualifications/Expertise

The individual or team should have the following specific experiences and qualification:

• In depth understanding of health issues in developing countries, with a special focus on maternal and child health;
• At least 3-5 years of experience in the development field;
• Consistent experience in conducting project evaluations for development and humanitarian projects (minimum 3 evaluations of NGOs projects, preferably in Sub-Saharan countries);
• Experience in conducting qualitative and quantitative research;
• Advanced degree in social sciences;
• Excellent reporting and communication skills;
• Capability to liaise among different relevant stakeholders at national and local level;
• Good communication skills in English; Italian and Somali will be considered as an asset;
• Working experience in Ethiopia and specifically in Somali Region is an asset.

Evaluation criteria

The consultant proposal shall be evaluated against the following criteria

a)Technical proposal

• Adequacy and technical quality of the proposal in meeting TORs, proposed scope and focus;
• Background and experience of the consultant firm and/or individuals.

b)Financial proposal

• The total financial requirement of the consultant to carry out the task is in accordance with the available budget.

Terms of payment

The consultants will receive remuneration under the following terms of payment which will be based on the output of the work and not on the duration that it might take:

• First tranche: 30 % of the total shall be paid upon signing of the contractual agreement;

• Second tranche: 30% after the submission of the first draft of the report;

• Third tranche: 40% after the submission and the approval by CCM of the final report

Management structure

The Contracting Authority shall be represented by CCM project coordinator, also responsible for the provision of general technical support.

Interested candidates should submit to recruitmentpvs@ccm-italia.orgno later than 24th July 2019 (European calendar):

• CV detailing qualifications and experience appropriate to the consultancy. Maximum 3 pages**. For European applicants is compulsory to complete the CV with the Privacy European Authorization 679/2016**

• Cover letter maximum 1 page, with three professional references and contacts

• Technical/methodological and financial proposal

Please, not ONLY the short listed candidates will be contacted