Terminal Evaluation of the SCI-SLM Project - GEF Medium-sized Project


01 December 2015

The medium-sized project 'Stimulating Initiatives in Sustainable Land Management' (SCI-SLM) was implemented through UNEP, and executed through the University of Kwa-Zulu Natal's Centre for Environment, Agriculture and Development, in coordination with four partner countries. Individual country programmes were implemented by South Africa, Uganda, Ghana and Morocco. Methodological support was provided by the Centre for International Cooperation at the Vrije Universiteit Amsterdam.

The overall objective of the project was to refine ways of stimulating the further improvement and spread of community-based sustainable land management initiatives while developing a methodology to upscale and institutionally embed SCI-SLM approaches at local and regional level in four pilot countries in Africa.

Dr Braby was commissioned to conducted the Terminal Evaluation of this project to assess project performance, determine its outcomes and impacts as well as their sustainability, and identify valuable lessons learnt and next steps of the SCI-SLM.

Within the methodology developed through SCI-SLM, stems the difference between social and technical innovation, and one of the key lessons coming out of the project was that social innovation is an untapped treasure in upscaling and replication. The way a community is organised and structured is the key to whether any initiatives will gain momentum or not. What makes some communities spread a novel idea better than others? The SCI-SLM went a long way to answering this question and ground-testing.

The SCI-SLM Terminal Evaluation had two main objectives:

(i)To provide evidence of results to meet accountability requirements, and

(ii)To promote learning, feedback, and knowledge sharing through results and lessons learned among UNEP and its main project partners, namely the GEF, CEAD in South Africa, TARGA-Aide in Morocco, MAAIF in Uganda, and UDS in Ghana, as well as the CIS in Netherlands.

This Terminal Evaluation focused on a set of key questions, based on the project's intended outcomes:

(iii)Has the project been successful in identifying community based SLM, creating interactive SLM databases, and analysing their technical, social and economic aspects?

(iv)To what extent has the project succeeded in stimulating and up-scaling community SLM initiatives (in terms of technical capacity, organisations structure, improved local governance, and improved communication) in each of the participating countries? To what extent has the novel "social innovation" concept been integrated into the four countries and even upscaled elsewhere?

(v)What evidence is there that demonstrates an increase in awareness on SLM initiatives amongst the policy makers, and to what extent can this be attributed to the project's activities and outputs?

(vi)To what extent has the project succeeded in developing guidelines and methodologies for the institutionalisations and upscaling of SLM initiatives in each of the participating countries?

(vii)To what extent has the project succeeded in contributing to the SIP Development and Global Environment Objectives, and South-to-South Exchange and learning in SLM approaches?

(viii)How effective and efficiently was the overall project planned, coordinated and monitored? What was the performance of the UNEP divisions and partners involved in the project?

The SCI-SLM was assessed with respect to a minimum set of evaluation criteria (on a six-pointscale) grouped into five categories:

(i)Strategic Relevance: focuses on whether the project objectives are consistent with the global, regional and national priorities.

(ii)Achievement out Outputs: assessing, for each component, the project success in producing the programme outputs and milestones as per the logical framework.

(iii)Effectiveness: Attainment of Objectives and planned Results: assessing the effectiveness of outputs achieved and the review of outcomes to impacts (using the Theory of Change approach).

(iv)Factors and Processes affecting Project Performance: covers project preparation and readiness, implementation approach and management, stakeholder participation, cooperation and partnerships, communication and public awareness, and country ownership and drivenness, financial planning and management, supervision and backstopping, and monitoring and evaluation.

Both quantitative and qualitative methods were used to evaluate the project achievements against the expected outputs, outcomes and impacts, and consisted of:

Desk review: A desk review of all the key project documentation supplied by UNEP and project staff, country partners and CIS, as well as the website.

Country Visits and Face to Face meetings: Of the four participating countries, three were visited, namely South Africa, Uganda and Ghana. Face to face meetings were conducted with National Coordinators and their core team in all three countries, project beneficiaries and communities whose initiatives were chosen (Msinga/Gudwini in South Africa; Bandera 2000, Mukono, RECPA, NACIA in Uganda; Moatani and Kandiga in Ghana), and a number of stakeholders (including NSC members) in all three countries. In South Africa, a meeting took place with the overall Project Coordinator as well as the Funds Management Officer of the project. Video and photographic documentation was taken where possible. In addition, the evaluator met with the Mid-term Reviewer of the project in Windhoek, Namibia.

Skype Interviews: Skype interviews took place with key project staff, including the UNEP Project Manager, three technical advisors from the CIS, the Project Coordinator (as well as meeting in person).

Questionnaire distribution: A questionnaire was distributed to all the country coordinators (those visited as well as Morocco) and key stakeholders.

Feedback mechanisms: Feedback was conducted during country visits to gauge evaluation results collected, as well as get feedback on the Reconstructed Theory of Change. These feedback meetings included brief presentations of initial results by the Evaluator, followed by a discussion. The Reconstructed Theory of Change was also presented (during country meetings as well as Skype meetings with the Technical Advisory Group) and feedback was given to the Evaluator. Feedback meetings were also held with CIS.

For a two-page information brief on the key evaluation findings, go here.

Clients: United Nations Environment Programme 09/15-03/16