Knowing what difference projects are making motivates workers and their collaborators to renewed effort. CABMACC Monitoring and Evaluation initiative will seek to apply lessons from experience to decisions about current and future programmes. M&E presents alternatives for decision-makers to consider. M&E is an excellent learning tool as well as a means to improve programme/project performance and demonstrate accountability. Too often M&E is perceived as threatening; it should be seen constructive as M&E results can be used to obtain greater support from influential stakeholders.
Monitoring is the routine collection and analysis of information to track progress against set plans and check compliance to established standards. It helps identify trends and patterns, adapt strategies and inform decisions for project/programme management. In CABMACC:
On the other hand evaluation is a systematic and objective assessment an ongoing or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, developmental efficiency, effectiveness, impact and sustainability.
While monitoring and evaluation are different, it is important to remember that both monitoring and evaluation are integrally linked; monitoring typically provides data for evaluation, and elements of evaluation (assessment) occur when monitoring.
M&E are very different from audits. An audit is an assessment to verify compliance with established rules, regulations, procedures or mandates. Audits can be distinguished from an evaluation in that emphasis is on assurance and compliance with requirements, rather than a judgement of worth. Financial audits provide assurance on financial records and practices, whereas performance audits focus on the three E’s – efficiency, economy and effectiveness of project/programme activities.
This document provide monitoring and evaluation guidelines for CABMACC projects. These guidelines are a necessary base for the credibility of evaluation and transparency throughout the whole process, starting from the design of Terms of Reference up to the publication of reports. Monitoring and Evaluation (M&E) is important to assess if the programme is achieving set targets and working within the agreed work-plans.
The guidelines ensure quality and unbiased results by supplementing the M&E Plan in the CABMACC M&E framework. It aims to clarify scope and to provide guidance and tools for continued monitoring and evaluation for all CABMACC research projects. The guidelines are intended to be used by the Research and Capacity Building subcommittee of PAC and/or other experts assigned by the programme. Monitoring and Evaluation is a responsibility of PCO in collaboration with the implementing partners (Projects). However all partners will play an important role in ensuring the following:
The overall objective of the guidelines is to enable users to understand monitoring and evaluation requirements within the CABMACC programme context. Specifically, the guidelines are meant to:
It is expected that the monitoring activities will enhance programme implementation by ensuring effective delivery of both programme and project objectives. The feedback generated from the monitoring activities will enable the PAC members to provide relevant and timely guidance to the project implementers. The guidelines will ensure the adherence to logical framework and achievement of programme/project deliverables and milestones.
More importantly, the M&E exercises are expected to ensure availability and use of accurate, timely, and relevant data to monitor and evaluate the CABMACC initiatives and shape improved policy direction as research outcomes are being enhanced. They will also strengthen human and institutional M&E capacity at all levels. It is envisaged that M&E will promote creation of M&E partnership and linkages between various partners and stakeholders at the various levels to enable information sharing, dissemination and the use of data. M&E is also expected to promote management performance through evidence-based planning, policy making, budgeting and interventions implementation. Last but not least, it is expected that M&E will promote transparency and accountability in the use of resources by encouraging participation of all stakeholders in mots stages of project work.
The following are the agreed terms of reference for the research and capacity building sub-committee of CABMACC PAC.
a) Assess the administrative and financial systems of the project;
b) Evaluate the participation of key project stakeholders;
c) Assess the level of support from parent institutions;
d) Evaluate the collaboration framework between the Programmes Coordinating Office (PCO) / programme secretariat and Project teams;
e) Understand the extent to which the project teams are working between and among themselves;
f) Assess how beneficiaries and other stakeholders are participating in the project activities including students and other faculty members
g) Assess adherence to CABMACC and LUANAR financial and procurement regulations and guidelines;
h) Ensure careful and adequate utilization of Project facilities, equipment and materials;
i) Identify challenges and constraints hindering achieving progress towards meeting the agreed objectives, milestones and indicators;
j) Ensure adherence to the Monitoring and Evaluation Plans agreed in the project documents.
k) Evaluate the extent to which the project team is producing scientific products and services and other information, education and communication materials.
Step 1: Organising documents for the M & E team Before initiating the M&E of the projects, programme secretariat shall put in place all relevant documents and information related to the projects in a Drop Box three (3) weeks before the activity. These documents include proposals and contracts, past quarterly reports, annual reports, most particularly the millstones and indicators to be achieved in the monitoring period. Otherwise, such documents shall also be available in hard copies.
Step 2: Informing PIs on the M & E activity The programme secretariat in collaboration with PIs and those to monitor the projects shall develop a schedule and proper communication be provided to all stakeholders to be involved. This shall be done three (3) weeks before the evaluation exercise.
Step 3: Assembling M & E teams PAC delegated the Research and Capacity building sub-committee to undertake regular M&E on all CABMACC teams. However, depending on the nature of each research project, other experts will be co-opted to ensure the value for money and maximising the outcomes of the exercise. In this respect experts will be co-opted based on their experience on specific project based themes based on their commitments.
Step 4: M & E Inception Meeting All M and E teams shall convene (1) week before embarking on monitoring and evaluating activities to set modalities for the process and share relevant roles and responsibilities. During the meeting the following shall be covered:
Before departing for field M and E the following arrangements must be made three days prior to departure.
a. Booking of appointments
b. Transportation and fuel arrangements
c. Accommodation and upkeep allowance arrangements
The M and E unit for the PCO shall develop the annual M and E work plan for the programme and projects. The framework shall have quarterly M and E visits by the PAC sub-committee with monthly M and E visits by the M and E experts and/or PCO teams. This schedule shall further include information to monitor delivery of progress by other contractors under PCO. 4.0 Methodology
Step 1: Meeting with Project team
Before embarking on M&E field visits, the visiting teams shall first meet with the project teams lead by the PI where applicable to gather pertinent information on the project. This will assess on their understanding of the project and what they exactly doing. Will also assess their understanding of their roles and what is expected of them at project level. It will be important for the visiting team to inquisitively assess how they are being involved on specific issues within the project. The check list for collecting information from project teams is provided in Annex 1. The team may also meet with those responsible for finances and procurement. The M&E team will also be required to interact with students attached to various projects. The key issues to be discussed are student engagement, research progress, publications, participation in research and challenges being faced.
Step 2: Consultation with government officials at district level
This will assess their understanding of the project and what they are exactly doing. Will also assess their understanding of their roles and what is expected of them at project and district level. It will be important for the visiting team to inquisitively assess how they are being involved on specific issues within the project. The check list for collecting information from district government officials is provided in Annex 2. The assessment will dwell much on the interactions and implementation with the District Council and other relevant professional officers/offices including the description of engagement and responsibilities. Will also look into issues of awareness creation, use of information and sharing of information including partnerships and challenges. The visits will also assess the contribution of CABMACC projects to district development plans. The visit will also look at other similar initiatives at district level. (At both levels, district and area)
Step 3: Consultation with beneficiaries
Field visits are important for the M&E to get a correct visual of the stages at which the projects are. Project beneficiaries will be interviewed to assess the level of their involvement on the project issues. At the end of the trip, the visiting team might arrange a triangulation meeting with all relevant officials at district and national levels. The visit will concentrate on administrative arrangements being followed, identification of beneficiaries and staffing where relevant. Research and developments interventions being undertaken, community engagement, partnerships, awareness activities. Check list to facilitate community consultations is provided as Annex 3. This process is vital in cross checking information collected from both project teams and district officials.
Step 4: Field visits
When all consultations are done with all the key stakeholders, the M&E teams will be required to visit the project sites to verify the information obtained from the project teams and district officials. Project officers/students attached to projects where available will be asked to make appointments with relevant stakeholders and communities. The visits will ensure that there is minimum disturbances on beneficiaries’ other days to day activities. Special attention should be made to project sites where there are other competing activities. Visiting members are encouraged to take pictures of the beneficiaries with their consent. Taking pictures of children must be done with prior consent from parents or guardians. Special attention should be taken with respect to minority concerns and issues. Check list 1, 2 and 3 will be vital in triangulation with issues observed in the field.
Step 5: Financial and Procurement assessment
The M&E teams will critically look at the financial and procurement issues at project and local level. The guiding questions for this part are included as Annex 4.
Step 6: De-Briefing at District level
After gathering all the required information and verifying all the required activities, the M&E teams are supposed to debrief the district teams and the project teams to triangulate their findings. This would clear all areas requiring feed-back and possible inclusion in the final reports.
Step 7: Meeting with PCO
After team have finalised their project discussions and visits, they will also be required to collection further information from PCO. The check list for collection information from PCO is provided as Annex 5.
Step 8: Report writing
A Monitoring report with detailed findings from the M&E visits with clear recommendations is expected within five days after every M&E. All reports shall follow the reporting format as provided in Annex 8. PCO will give feedback on the M&E reports to M&E team members within five days.
Step 9: Providing feedback to PAC and Programme secretariat
The visits will be followed by presentation of the same to a plenary session at a later date. Where necessary the reports will be presented during the routine PAC meeting.
The indicators listed in specific project documents of the projects will provide evidence that results are being achieved. These Indicators will enable the measurement of actual achievement against planned or expected results.These indicators are an important tool in M and E as such, all M and E teams must familiarise themselves with these indicators before heading to the field. For all the projects the following generic indicators shall be tracked using the following Indicator Tracking Table as provided in Annex 6.
Based on the results generated from the M and E final scoring sheet (Annex 7), visiting M&E team members will make the appropriate recommendation for action by PAC and the Secretariat. The project teams may appeal this judgement to PAC with clear justification for improvement in the next 2 months while on suspended funding. Similarly hard working and good performers may be rewarded some incentives at the discretion of PAC and Secretariat on recommendation from the Research and Capacity Building sub-committee.
SCORE |
ACTIONS TO BE TAKEN |
|---|---|
75 – 100 |
This is excellent performance. There is need to encourage the team to transfer skills and techniques attributing to the success of the project to other projects. Such project will be role models for others and could organise some exchange visits to assist non performing projects. These projects can be getting quarterly financial disbursements as normal. |
60 – 74 |
This is very good performance. There is need to encourage these projects to work harder. These projects can be getting quarterly financial disbursements |
50 – 59 |
This is good performance with room for improvement. There is strong need for encouragement. These projects must be written some warning and get disbursements based on milestones. There is need for more technical and financial advice |
35 – 49 |
This is poor performance. These projects will be funded based on specific activities. There is need for a stern warning to such projects and provision of more technical guidance. There is need for project team capacity building |
less than 35 |
This is very bad performance that needs serious interventions from authorities. These projects must be written letters about the cessation of funding |
This section includes all the tools that will be used for data collection and reporting during and after the M&E visits. Different target groups have been assigned specific data collection tools and check lists.
This table shall be used for overall rating after gathering all the required information from all relevant target groups and beneficiaries. The decisions made in Section 6 are to be derived from the table on this link Overall rating Table