Terms of Reference for Final Evaluation at Vision Aid Overseas Zambia

I.  BACKGROUND OF PROJECT

1. Project Description

Vision Aid Overseas (VAO) proposed to scale up the already tested Comprehensive School Eye Health Program in Zambia and test and document a scaling methodology. This proposed expansion of the School Eye Health Program in Zambia supports CBP’s thematic priorities to “Promote High Quality and Comprehensive Child Eye Health Services” and to “Expand and Treat Refractive Error”. We proposed a two-pronged approach that targets both vertical scale-up (policy and environment) and horizontal scale-up (expansion of services). Specifically, we proposed to:

1.            Expand the services to reach more children, specifically the rural poor. We will roll out the program in all Primary and Secondary schools in Petauke District, Eastern Province of Zambia, which serves a child population of 151,180. The district is predominantly rural, with most of the population working in agriculture. It is one of the poorest districts in Zambia, with a poverty rate of 69% and infant and under-five mortality rates of 80 and 127 per 1,000 live births, respectively (UNDP).

2.            Build the capacity of the Ministry of Health (MoH) and the Ministry of General Education (MoGE) to integrate the program into the existing school health and nutrition (SHN) program, and use the evidence and learning generated to advocate for increased resource allocation and commitment towards an appropriate scale-up plan.

Similar Jobs in Zambia

Learn more about Vision Aid Overseas Zambia

Vision Aid Overseas Zambia jobs in Zambia

2. Overall objective

To expand access to child eye health services in Petauke District using a tried-and-tested government-approved school-based eye health program model and to support the integration of eye health into the SHN program.

3. Specific objectives

1.            To increase the availability and accessibility of quality eye health services for school-aged children through (a) timely mass-screening by well-trained teachers, (b) follow-up care and management using the tried and tested Mobile Eye Clinic (MEC) model, and (c) improved community awareness and demand for eye health services.

2.            To document and support evidence-based advocacy for the government to support a nationwide scale-up of school eye screening and treatments as an integral component of the SHN Program.

II. EVALUATION OBJECTIVE, SCOPE, AND INTENDED USE

The general objective of the final evaluation is to:

1.            To evaluate the if program’s performance by assessing its outputs against set targets

2.            To determine the motivating and inhibiting factors to the success of the project

3.            To determine the short-term impact of eye health management (spectacles or/and eye medications) on the affected children

4.            To make recommendations for the future expansion (replication or scale-up) of the programme

This will be guided and supported by Queen’s University Belfast (QUB) to ensure a high level of academic rigour is applied. The Development Assistance Committee (DAC) criteria for programme evaluation will be used to assess the programme’s relevance, effectiveness, efficiency, impact and sustainability.

VAO conducted a baseline during the Mobile Eye Clinics with 200 learners. The evaluation process will include assessing the Quality of Life (QoL) of learners who received glasses. A questionnaire and a dataset from the QoL baseline will be provided.   The learners will also be interviewed by the consultant to understand the student experience after wearing the glasses. The goal is to assess the change in learners’ quality of life after receiving and wearing glasses.

2.1 Target audience and intended use

The evaluation will serve VAO and the partner organisation CBP as an important tool for learning, creating knowledge, and demonstrating accountability and transparency. Findings from the evaluation will also feed into developing a national guideline for integrating school eye health into the school health and nutrition program. Assessing project management and performance also helps VAO and CBP to have a base for potential follow-up on the Scale-Up School Eye Health (SSEH) project within Zambia. The evaluation will involve a wide range of stakeholders through a participatory approach. The evaluation results will be shared with these stakeholders to contribute to organisational learning in the different organisations.

2.2 EVALUATION QUESTIONS

1) Relevance/Quality

• How far has the project responded to the needs and priorities of the target groups as identified in the project plans?
• To what extent have the project’s planned activities and expected outputs been consistent with the overall goals and the attainment of their objectives?
• To what extent has the project approaches and activities been appropriate to the local context?
• How have the working relationships, communication and coordination between stakeholders been managed?
• Has the project design and implementation considered other sectoral interventions in the area?
• What are the views of the target group on the project, and what was the extent of their involvement in their implementation?

2) Effectiveness/Equity

• How effective has the project been as a disability-preventative and public health interventions in Petauke District of Zambia? What gaps/ opportunities exist?
• Are the objectives, approaches, and project design coherent and complementary?
• To what extent have planned targets, objectives and activities been delivered? In case of targets, objectives and activities were not successfully implemented, what is/are the reason (s)?
• How effectively did the project reach persons in need? How inclusive was the project benefiting persons in need in all service areas?
• How well did target populations participate in project decision-making processes?
• Has the awareness and understanding of eye health increased amongst partners (MoE, specifically teachers trained)? Has this occurred in the community? What can be learned about the awareness initiatives undertaken? Were some more successful than others?
• Have project personnel (teachers trained) received knowledge and training on the specific requirements of school learners living with permanent eye conditions? How are they using that knowledge? Did their behaviour change?
• Do women and girls, men and boys have equal access to the project activities, and do they benefit on an equal basis? What needs did each of those groups have in accessing the services offered through the project? Has the project considered those different needs and addressed them in its activities? Are different gender and age groups accessing eye health interventions equally? If not, what are the future recommendations for addressing the barriers?

3) Efficiency

• How efficiently has the project been implemented as compared with alternatives? Were the project structures adequate, and what adjustments are recommended if it was inefficient?
• How well have the project’s human, financial and technical resources been managed regarding transparency and accountability?
• What processes have been implemented to ensure transparency and accountability between CBP and VAO?
• How is monitoring data being collected and stored? Is the data used to inform relevant platforms (Health, Education, etc.)?
• Are existing learning processes such as reflection and internal review used sufficiently to keep the project plan updated?

4) Impact

• To what extent did the project contribute to preventing, treating, and managing eye health problems?
• What proportion of the target groups has benefited from the project interventions, and in what meaningful ways have their lives changed? Did the project improve the quality of life of the target group?
• What, if any, negative or unintended outcomes on the target groups arise from the project?
• How effective has the project been in creating changed behaviour regarding face washing, wearing prescribed spectacles, and seeking eye health services?
• What broader change in the health service provision and personnel can be seen as a result of the project?

5) Sustainability

• What is the sense of community ownership of the project? What changed for the persons with disabilities: lasting change in perception by the community and health service providers?
• To what extent has the project strengthened the capacities of local government and district-level stakeholders? Can they sustain the project’s achievements after the end of external support? If so, how?
• What major factors will influence the project’s achievement of, or lack of, sustainability?
• Is there any environmental impact of the project’s actions?
• What role will the government play in maintaining the project’s momentum?

6) Child Safeguarding

• To what degree do the teachers and health personnel understand the child safeguarding standards?
• To what degree did VAO and other implementers uphold child safeguarding, was the process efficient and effective in protecting children?
• How has a safe environment for children been established and maintained throughout the project’s planning and implementation stages?
• Are any activities related to the project that have created unforeseen negative impacts on children’s safety?

III.  METHODOLOGY

A participatory approach will be adopted for the process of this final evaluation. This evaluation will require using multiple data collection methods, combining qualitative and quantitative methodologies, being conducted in close collaboration with VAO, Queen’s University Belfast, and involving other stakeholders (e.g. Ministry of Health and Ministry of General Education).

Expected activities will include (though not limited to):

• Desk review
• Field observations and site visits in Petauke (Participating schools).

Key informant interviews and FGDs with VAO and MoH/MoGE staff (At National and District levels) involved in the project and selected project beneficiaries; health personnel, teachers and pupils and their parents/guardians.

IV. LIMITATIONS

• Some project sites may be cut off due to rain.
• Some beneficiaries may be unavailable due to school term breaks/holidays, transfers, and graduations.

V. EVALUATION TEAM AND MANAGEMENT RESPONSIBILITIES

5.1  Commissioning Responsibility

The Final Evaluation will be commissioned by VAO and CBP in collaboration with Queen’s University Belfast. Their responsibilities will include the following:

• Planning evaluation with all stakeholders
• Commissioning evaluation
• Approval of final evaluation report
• Dissemination of evaluation results
• Facilitation of logistics
• Provision of HR support in charge of the project
• Facilitation of communication with local stakeholders

5.2 Evaluation Team

The Evaluation Team is expected to have at least 2 experts.

Major Tasks:

• Lead the evaluation preparation and implementation work with the support of VAO Zambia and Queen’s University Belfast
• Prepare detailed technical and financial proposals, and submit them to VAO
• Develop assessment tools, methodologies and other works related to the assessment and revise them after comments from Queen’s University Belfast
• Conduct site visits/field observation
• Conduct interviews and focus group discussions
• Prepare a detailed evaluation report
• Prepare and participate in meetings and debriefings during the process.

The team needs to have the following qualifications:

• Advanced degree in Public Health, Medicine, Development, or other relevant fields.
• At least 5 years of work experience in this field.
• Knowledge of health sector policies and health systems in Zambia
• Substantial experience in conducting programme evaluations of similar nature.
• Experience conducting population-based surveys.
• Knowledge and experience in policy analysis, health sector planning and development.
• Knowledge and experience in strengthening partnerships and aid effectiveness, experience with donor coordination.
• Strong interpersonal, negotiation and communication skills.

5.3 Management of the evaluation and logistics

The VAO team will support access to all relevant documents, including planning documents, reports, results framework and assessment reports. Furthermore, VAO will provide administrative and office logistics and support scheduling the site visits in coordination with MoH and MoGE.

5.4 Expected Results

1)  Inception Phase: Desk Review and presentation of the work plan and evaluation methodology (2 days) until 5th of January 2023.

The consultants will prepare a work plan including their understanding of the issues addressed in the evaluation and the evaluation methodology in English. This will be presented to VAO and QUB.

2) Field visits (10 days from inception)

The consultants will conduct fieldwork at the project site and hold meetings with key resource persons to collect information per the requirements in the approved evaluation work plan. Stakeholders in the field are to be briefed on arrival and debriefed after the review.

3) Data analysis and presentation of preliminary findings (5 days after field visits)

The consultants will conduct an in-depth analysis of the data gathered during the inception and field phases. After the data analysis is finalized, an online presentation for VAO and Queen’s University Belfast should be organized to present and validate preliminary findings.

4) Final evaluation Report (5 days after data analysis)

The consultants will prepare a report describing the evaluation process, findings, learnings and recommendations. A draft report will be submitted 3 days after data analysis (for distribution to the implementing partner) to VAO and Queen’s University Belfast for feedback. Comments will be incorporated in the final report by the consultant team. The final report is to be submitted electronically, not later than 7 days after receipt of comments on the draft evaluation report.

5.5 Deliverables:

The consultants will prepare the following:

• A detailed methodology and work plan
• An Inception meeting and report
• A briefing workshop and presentation on preliminary findings for VAO and Queen’s University Belfast
• A final evaluation report in English (25 to 30 pages max)

The stated dates are indicative and will be discussed during the inception phase.

VI. APPLICATIONS

The consultant/s shall provide an offer (Expression of Interest) including the following:

• Description of the consultant, including CVs of the suggested evaluators.
• A detailed approach and methodology for achieving the goal of the assignment.
• A detailed schedule including steps to be taken and responsibilities of team members.
• Statement of availability for conducting the assignment.
• Financial offers, including professional fees, taxes, and incidentals.

The commissioning entity may require proof of similar evaluation experience in the form of previous reports and/or references during the selection process.

Selection of the candidate will consider the above criteria/requirements and the financial offer. VAO reserves the right to end the contract if the agreed-upon and contracted team members are unavailable for the evaluation and if no adequate replacement is provided.

The offer shall be sent in electronic format, by 23 December 2022, to chishimba.chibwe@visionaidoverseas.org and copy godfrey.mwelwa@visionaidoverseas.org