Terms of Reference
Consultancy for Midterm & Final Programme Evaluations
DFID–funded Consortium project titled “Emergency assistance to crisis-affected communities of Central African Republic, Phase 2” implemented by 4 INGOs: The MENTOR Initiative, International Medical Corps, Cordaid, and Oxfam (Oxfam phase out as from February 1, 2018) between October 2016-September 2018.
- BACKGROUND OF THE PROGRAMME
The MENTOR Initiative is a discrete and agile humanitarian organization purpose built to relieve human suffering from tropical diseases. MENTOR has been present in C.A.R. since 2008. From the very start of the mission, core activities have focused on supporting community-based healthcare services and to date, MENTOR is the only organization in the country to implement this ‘grassroots’ approach on a large scale. This approach has proven to be effective and fully appropriate in a context where standard health system structures are virtually non-existent outside of main towns and where malaria, a disease that is easily diagnosed and treated if caught on time, is endemic. MENTOR is currently operational in eight sub-prefectures in north-western C.A.R.
More than 200 curative community-based healthcare workers are now, with support from MENTOR, operational and trained to diagnose and treat essential diseases, including uncomplicated malaria. Complex cases beyond the capacity of the Community Health Worker are referred to healthcare facilities supported by MENTOR or partner organizations. Community sensitization forms an essential component of the programme, provided through IEC/BCC activities. As part of these activities, Mass Drug Administration campaigns are also implemented, mainly to prevent soil transmitted helminth infections and vitamin A deficiency. Furthermore, access to improved maternal healthcare is provided through Intermittent Preventative Treatment for malaria, iron supplementation at community level and extensive sensitization on safe pregnancy practices.
In 2015, The MENTOR Initiative, International Medical Corps, Save the Children (phase out to Cordaid in 2016), and Oxfam entered into a Consortium partnership for the first Phase of the current programme, with International Medical Corps as lead of the Consortium. Phase 1 aimed to implement a health, nutrition, and WASH programme response reaching an estimated 432,665 persons across four prefectures of Central African Republic (Ouham, Ouham Pendé, Haute-Kotto, and Basse-Kotto) and aimed at preventing and reducing excess morbidity and mortality of the population affected by the ongoing crisis. A final programme evaluation from Phase 1 was undertaken and is available.
The 4 INGOs entered into Phase 2, with The MENTOR Initiative as lead, on October 1, 2016 for a period of 2 years. Oxfam, as the partners specialized in WASH activities, has been phased out as from February 1, 2018. The health, nutrition, and WASH programmes are currently implemented in the same 4 prefectures and aim to build upon what took place during Phase 1 while harmonizing on best practices. However, the security situation in the C.A.R. has severely regressed since the beginning of Phase 2 and Consortium partners have adapted and revised the programme to be able to implement pertinent emergency activities where necessary.
- GOAL AND OBJECTIVES OF THE EVALUATIONS
There is to be 2 evaluations undertaken several months apart: an immediate Midterm Evaluation to be started in mid May 2018 and subsequent Final Evaluation to be conducted in November/December 2018. The goal of the two evaluations is to assess and learn from the process and achievements of the programme in the four prefectures, evaluate progress towards the program objectives and results, implementation strategies, assess the extent to which the programme demonstrates good “Value for Money” (using DFID’s Value for Money approach), document lessons learned, and to inform the partners, DFID, beneficiaries and other relevant stakeholders about results and findings. Results will influence a Phase 3 design and proposal.
Specific aspects for consideration
- The evaluation will be guided by the evaluation criteria of relevance, effectiveness, efficiency, impact, and sustainability.
The evaluation should specifically assess:
- Relevance: Assess to what extent local needs and priorities have been addressed, activities and output are consistent with the 4 intended outcomes of the programme
- Effectiveness: Assess what has been accomplished in relation to expected outcomes and results set in the logframe in each area of intervention and from a central level, specifying the major factors which have contributed to the achievement or not of the intended objectives; assess the extent to which the Consortium coordinates with other relevant actors in each zone and identify where this could be improved upon; assess the Consortium’s ability to adapt to evolving security and political contexts in specific intervention zones and C.A.R. in general; assess how effective the Consortium was in handling the phasing out of Oxfam as WASH partner of the Consortium in early 2018
- Efficiency: Assess how inputs (human, financial, material resources) have been translated into results and if results have been achieved at an acceptable cost – see Value for Money below; assess to what extent the Consortium approach and the management of the Consortium programme has provided added value in attaining achievements or has hindered it
- Impact: Assess to what extent the programme has contributed to reducing excess morbidity and mortality of the targeted population affected by ongoing crises; assess the impact of different health financing systems in place (targeted gratuity, total gratuity)
- Sustainability: Assess to what extent the programme will produce benefits in the long-term to the communities and the national Health System and identify gaps that could be filled; identify advantages and disadvantages of the “Performance based Financing” system (financed by the World Bank) that is also in place in certain Consortium intervention zones alongside the more traditional “input” support given by Consortium partners and identify adaptations that would best ensure sustainable access to quality healthcare including as policy and programme level
- b) Accountability How each INGO has been accountable to beneficiaries, national and local authorities, and the donor.
“Value for Money”: In line with DFID’s policy, the evaluation will analyse the achievement of the Consortium towards identified “Value for Money” indicators and evidence that partners have effectively assessed and considered value for money considerations in the management of the programme. The evaluation should identify areas where improved synergies could translate into better Value for Money in the future.
Due to chronic insecurity in certain intervention zones, Consortium partners are developing and implementing remote management tools. The evaluation should identify to what extent these tools and processes impede or promote accountability.
- c) Accessibility How access to health services affected the various population in the 4 Prefectures specifically pertaining to health, nutrition, and WASH aspects. Identify if there have been factors preventing or restricting access of certain groups or minorities to healthcare services and potential strategies to address them.
- d) Quality Evaluate quality of interventions in terms of indicators chosen and reported against, implementation methodology, competence of staff, team building, gender, and if health care provided through the programme is socially and ethically acceptable.
- e) Lessons learned Evaluate the extent to which the Consortium has thus far acted upon lessons learned during Phase 1 (including evidence and data gaps) and identify further lessons learned, both positive and negative, that can feed into a Phase 3.
- TIMEFRAME, METHODOLOGY, AND DELIVERABLES
Design and methodology used in the evaluation will be proposed by the consultant, but should include quantitative and qualitative methods, a desk review of relevant documents, interviews of relevant stakeholders, interviews with beneficiaries, and observation. Bids for the consultancy should provide a clear description of the design and methodology the consultant will use to answer the key questions, including recognized evaluation methods, proposed counterfactuals if/where appropriate, data collection methods, analytical methods, and approach to synthesis.
Visits to field sites are expected; however, security will be assessed before each field mission and the consultant would be expected to travel from Bangui to field sites only when partners have deemed security is sufficient.
The consultant will be provided with previous M&E documents including logframe, databases of medical data since the beginning of the programme, quarterly reports, etc. The consultant will be responsible for the identification and provision of any new primary data needed for the purposes of the independent evaluation. The consultant will need to determine which arrangements would be most cost-effective overall and least burdensome on beneficiaries or programme implementers.
MENTOR requests that all databases containing qualitative and quantitative data collected during the consultancy are to be shared in a commonly used format, together with clear metadata, and which is anonymized and safeguards confidentiality.
The timeframe for the immediate Midterm Evaluation is proposed as follows:
|DESK REVIEW||The consultant will initially conduct a desk review of all materials related to this programme to familiarize her/himself with the programme background as well as its goals and objectives; The MENTOR Initiative will provide any relevant documents required by the consultant for this evaluation.
Methodology to be adopted, sampling strategy, work plan, list of persons to be interviewed, templates of data collection tools have to be shared with the partners through the Consortium Coordinator in an inception report produced after the review of the background materials.
|5 days (approximate dates: May 14-18 2018)||Inception report|
|FIELD WORK||The fieldwork will be conducted in the four areas of intervention (Ouham, Ouham Pendé, Haute-Kotto, Basse-Kotto) and at coordination level in Bangui to start approximately the first week of May 2018 for approximately 5 weeks, depending on flights availability and accessibility in the areas of intervention. The consultant will be responsible for managing the evaluation process in-country, but will be supported by the Consortium Coordinator who will provide technical assistance (e.g. make in-country travel arrangements, assist in making appointments, provide data, documents and information available, etc.) and staff of the 4 different organizations while conducting activities.
The external consultant is expected to debrief the Consortium Coordinator and the 4 Country Directors on initial findings. At this stage, the Consortium Coordinator will identify 1-2 key findings that the consultant is requested to elaborate on in separate short paper(s) to be presented with final evaluation report (for example, on health finance systems).
|25 days (approximate dates: May 21-June 22 2018)||Debriefing on initial findings/conclusions|
|REPORT||The evaluation will result in an initial draft evaluation report, which will be sent for comments to the Consortium Coordinator and all 4 Consortium partners. Ideally, this draft will be in French. The short papers will also ideally be provided initially in French.
After consultation, a final evaluation report is expected no later than 10 days after receipt of comments.
The final evaluation report (ideally provided in English and French by the consultant) should include a 1-page executive summary, background information on the programme and evaluation, the methodology adopted, presentation of key programme achievements, results and analysis of evaluation criteria clearly linked to evidence, summary of strengths and weaknesses, challenges encountered, conclusion, lessons learned and recommendations to respond to the objectives of the evaluation, as outlined above. The report should be a maximum of 50 pages with 11pt. single-spaced type.
|10 days (approximate dates: June 25-July 6 2018)||Draft evaluation report
Final evaluation report along with 1-2 short papers on key themes identified during debriefing on initial findings
Reports should communicate overall findings in an accessible way for non-technical readers, including presentation of data in visually appealing ways, highly structured and rigorous summaries of findings and robust and accessible syntheses of key lessons. Recommendations should be timely, realistic, prioritized, and evidenced-based.
The timeframe for the Final Evaluation to be completed in November/December 2018 will be proposed by the consultant, but is expected to be shorter (approximately 6 weeks) as the consultant will already be familiar with the programme. Relevant field visits to assess progress are expected where necessary and a draft and final evaluation report are expected.
- COMMUNICATION MANAGEMENT
The consultant’s primary contact in C.A.R. will be the Consortium Coordinator. Secondary contacts include the Consortium manager (based in Tanzania) and MENTOR’s Head of Mission (based in Bangui).
- CONSULTANT PROFILE
- Higher university degree in relevant field, i.e. Public Health, Development Studies, Humanitarian Response, etc.
- Demonstrable relevant practical experience in conducting evaluations in development and humanitarian settings using qualitative and quantitative research methodologies
- Proven experience in using participatory evaluation methods, conducting evaluations for complex multi-sectoral, health programs in challenging settings, experience with DFID-funded projects and/or Consortium programmes is an asset
- Strong understanding of C.A.R. context is an asset
- Strong computer skills including appropriate statistical analysis software
- Excellent spoken and written communication skills in English and French, knowledge of Sango is an asset
- Strong analytical and communication skills for complex, fast-paced environments
- A strong commitment to delivering timely and high-quality results
- Expected to sign and follow MENTOR’s Code of Conduct and the 4 principles and standards elaborated below
Principles and standards
- The evaluation should adhere to international best practice standards in evaluation, including the OECD DAC criteria for humanitarian evaluations and DFID’s Ethics Principles for Research and Evaluation. Bids should demonstrate how they will achieve this.
- In line with Paris Declaration principles, the consultant should draw on existing data where available, ensure new data collection is complementary to existing systems and that new data are made available to stakeholders as far as possible.
- Disaggregation of data, including by sex, gender, and disability will be useful throughout the evaluation.
- The consultant will need to comply with MENTOR’s policies on fraud and anti-corruption and cooperate with any checks required from MENTOR for the duration of the evaluation.
An appropriate budget including detailed expenditure estimated is to be submitted along with bids. Bidders are requested to be very clear about methodology providing a detailed breakdown of costs for the different significant activities to be undertaken during the evaluation. The budget range for the 2 evaluations is expected to be between $30,000 – $40,000, all included. Bidders are strongly encouraged to compete on the basis of their commercial proposal, demonstrating value for money, as well as technical proposal.
There will be three payments enumerated as follows:
20% of fees for Midterm Evaluation upon submission of the Midterm inception report
50% of fees for Midterm Evaluation upon presentation of the draft Midterm evaluation report
30% of fees for Midterm Evaluation upon submission of final Midterm report and all documentation as required by The MENTOR Initiative for final payment
*Payment for Final Evaluation will follow the same logic
Interested bidders should submit a CV, a well-developed methodology, and a financial proposal. The CV should clearly explain past evaluation experience, skills, and qualifications. Financial proposals should include detailed expenses.