Internal and external evaluations

Please check back later

Please check back later

Please check back later

The main conclusion from NCA’s scoping study is clear: it is indeed possible for NCA to maximise the use of its internal data in evaluations by pairing or matching it with openly available datasets. This presents a wide range of possibilities to address OECD DAC evaluation criteria, especially in relation to relevance, coherence and impact.

Programme overview

Civil society is at the core of NCA’s work. NCA works in partnership with multiple local civil society organisations (CSOs), in projects funded by Norad and other donors. From 2010–2015, civil society strengthening was a dedicated NCA programme. NCA’s 2016–2020 strategy treated it as a cross-cutting issue with its own mandatory outcome in all NCA global programmes. NCA’s partnerships with CSOs are based on principles outlined in its 2015 Partnership Policy, which include sustaining, and adding value to, partners’ work. 

Evaluation overview

In 2018–19, Javier Fabra-Mata and Quinn Coffey conducted an internal evaluation of NCA’s civil society strengthening work from 2016 to early 2019. The evaluation documented how NCA Head Office and country offices understand and operationalise strengthening civil society. It assessed the impact NCA’s impact in key areas, notably advocacy and networking. It analysed how NCA plans, measures and documents its efforts to strengthen civil society and identified lessons to improve this. The evaluation used a global e-survey, key informant interviews and focus group discussions. Research participants included NCA staff in Oslo and the field, and staff from local partner organisations, government departments and resource organisations. It included country case studies from Zambia and South Sudan. 

Key fininds and recommendations

No single concept or methodology defines NCA’s efforts to strengthen civil society.  

NCA’s primary contribution to strengthening civil society has been through building the capacity of partner organisations. Successful interventions involved cooperation between NCA partners as well as between NCA and partners. 

NCA’s partnership approach has prioritised financial, administrative and human resources (HR) capacity building, rather than leadership, governance or fundraising. NCA has not always prioritized competencies such as advocacy that could strengthen civil society. 

Civil society programming is not integrated into NCA’s country strategies and programming in a systematic way. Results often focus on partners’ participation in national and regional networks, overlooking thematic programme results such as water, sanitation and hygiene (WASH) committees, village community banks (VICOBA) or women’s groups.  

NCA should revise its Partnership Policy, including guidance on how to operationalise civil society strengthening.  

It should adopt a systematic and broad approach to assessing and developing partners’ capacities in line with their needs. 

NCA should review and update civil society planning and reporting tools. 

NCA should encourage strategic thinking about, and novel approaches to, its engagement with civil society. 

This report is the product of its authors, and responsibility for the accuracy of data included in this report rests with the authors alone.

Program overview

Somalia has struggled with the ruins and attendant consequences of conflict, violence, and human right violations for close to three decades. The situation has been compounded by cyclic drought, poor economic conditions, and massive displacement. The formulation of NCA Somalia strategy found 1.1 million displaced and up to 3 million people in need of lifesaving and livelihood support in the country. The situation called for continued NCA response within existing and new structures customized to the humanitarian needs and dynamics in the context. The sectoral focus of the strategy included Water, Sanitation and Hygiene, Gender Based Violence, Economic Empowerment and Education for sustainable development. The NCA 2016-2020 was formulated to respond to these needs from previous efforts customized to changes in the operating environment.

Evaluation overview

At the implementation midway, a review was envisioned to take stock of progress informing the remaining phase and the next strategy. The purpose of the MTR was to specifically assess the extent to which each thematic programme is progressing towards its stated outcomes and targets to provide actionable recommendations. The review also assessed the relevance and validity of each programme’s Theory of Change, examined if the changes anticipated at the outset of the country strategy are within reach, which factors are proving critical in making change happen or in hindering change.

Key findings and recommendations

The evaluation concludes that several of the programs (WASH, GBV, Economic Empowerment, Education for Sustainable Development and Emergence Response), were successful in some of the programmes target areas. Due to the difficult operational climate and security concerns, it is difficult to both track progress and to fulfill the intended interventions. Due to the wide coverage of the report, please see dedicated chapters for more in-depth information.

Program overview

The Norwegian Church Aid (NCA) in partnership with Women in Law in Sothern African- Research & Educational Trust (WLSA) with support from European Union (EU) implemented an Action contributing to increased access to justice for survivors and victims of Gender Based Violence (GBV) in Eastern and Western Provinces of Zambia.

Evaluation overview

The purpose of the evaluation was to assess the extent to which the project contributed to achieving the set outcomes for the project and assess effectiveness, relevance, efficiency, sustainability, and impact.                

Key findings and recommendations

Incidences of GBV in the project area have been reducing with time over the 3-year period. Community confidence of the local courts, police VSU, health department, Ministry of Community Development has increased over the project period, and this has increased the empathy to GBV survivors and victims. This is attributed to the training that increased awareness of the consequences of GBV and also increased interaction and coordination between the paralegals, traditional and religious leaders with the GBV justice delivery service providers. Cases are now documented and evidence better-preserved leading to improved prosecutions of GBV perpetrators.                              

The evaluation states that there is a need for more local sensitisation of communities to reach out to more people and further engage the community. There is a need to provide more training and enhance learning, especially for Local Court Justices and Officers who normally rely on Customary Laws to determine GBV cases.

A Trainer of Trainers approach should be adopted whereby the Officer trained by the Project in turn trains his work colleagues and share information on activities being undertaken. The evaluation recommends an increase in victim support, for example through the provision of safe spaces for victims during the trial. In addition, more thorough documentation and record keeping should be conducted.

Programme overview

This three-year economic empowerment pilot project in Mchinji district of Malawi in Southern Africa integrates smallholder farmers into oilseed value chains. The NCA project aims to improve profits from agriculture, especially among marginalised rural youth and women farmers. The project’s value chain development includes processing and marketing oilseed. Its methodologies involve training, policy advocacy, market linkages and market information.  

Evaluation overview

NCA undertook this mid-term evaluation to assess the project’s relevance and progress towards achieving its planned objectives. Lerato Lekhoaba conducted the evaluation in 2019, using both qualitative and quantitative data, and primary and secondary sources, notably a field survey among members of Mikonga Cooperative. 

Key findings and recommendations

This project directly contributes to Malawi government strategy and policy objectives, as outlined in the National Agricultural Investment Plan. The intervention has increased profits among participating farmers.  

The project team should explore other technologies to enhance farmers’ productivity, especially by supporting year-round oilseed production. It should conduct technology analysis before acquiring new machinery to ensure it is durable, and easy to use, maintain and replace.  

To increase the cooperative’s capital and commodity volume, the project should explore ways to increase cooperative membership.  

The project should make greater use of radio to raise community awareness of the cooperative’s products and services. 

This report is the product of its authors, and responsibility for the accuracy of data included in this report rests with the authors alone.

Programme overview

With funding from the Norwegian Ministry of Foreign Affairs (MFA), NCA implemented a humanitarian water, sanitation and hygiene (WASH) programme in the Mopti and Ménaka regions of Mali. This emergency programme targeted communities affected by the security crisis in Western Africa. 

Evaluation overview

In 2019, NCA commissioned external consultant Moussa Sacko to conduct an endline evaluation of the ‘Humanitarian WASH programme in the Mopti and Ménaka regions’. This evaluation assessed progress towards programme objectives using the OECD-DAC criteria of relevance, effectiveness, efficiency, impact and sustainability. It also aimed to gather opinions and lessons learned. Evaluation methodology included a document review, field visits to implementation sites, and individual interviews and focus group discussions with programme partners, beneficiaries and community representatives. All segments of the target population were represented in the sample. 

Key findings and recommendations

The programme was in line with Mali’s national WASH strategy and NCA’s country strategy.  

The programme improved access to WASH services in target communities. However, needs remain due to population movements and limited resources. 

Local implementation partners should employ staff from the intervention localities as community development agents. 

The programme design was gender-sensitive and inclusive of all community sectors, including people with disabilities. A needs-based approach resulted in actions that were generally accepted by all stakeholders. Gender equality was included in community-level awareness-raising. The presence of female partner staff facilitated discussions around certain themes with women.  

Most beneficiaries had no knowledge of feedback or complaint mechanisms. However, the programme had no negative impact on conflict in the target areas.  

More coordination between NCA’s local implementing partners would enable more synergy and sustainability in interventions and monitoring.  

Social cohesion is important in this conflict-affected context. Complementarity with a peacebuilding programme is recommended in future local interventions.  

The monitoring, learning and evaluation system must be strengthened, based on capacity assessments of partners. Digital data collection tools must be used to collect data for timely decision-making.  

This report is the product of its authors, and responsibility for the accuracy of data included in this report rests with the authors alone.

Programme overview

From 2015–2019, with funding from NRK Telethon, NCA implemented a water, sanitation and hygiene (WASH) programme for targeted communities in Sanghar and Thatta districts of Pakistan’s Sindh province. The programme aimed to strengthen community resilience to flood, improve access to safe drinking water, and provide adequate sanitation and hygiene. This included building the capacity of institutions that manage WASH facilities. The programme incorporated gender analysis, disaster risk reduction and grievance systems. 

Evaluation overview

NCA commissioned GLOW Consultants to conduct an endline evaluation of the programme in 2019. This evaluation used OECD-DAC criteria of relevance, effectiveness, efficiency, impact and sustainability to assess the programme’s design and implementation. It measured results against a 2015 baseline study, other local assessments and annual village profiling. Its methodology comprised a desk review, and field visits to conduct key informant interviews, focus group discussions and transect walks. 

Key findings and recommendations

This programme allowed better engagement with women and minority group beneficiaries than other interventions in Sindh province and elsewhere in Pakistan. However, women’s reticence and deferral to men reduced their involvement in programme activities, indicating a need for more comprehensive community mobilisation for behaviour change. 

Specific local challenges affected programme implementation, such as low rainfall and difficulty finding a constant groundwater source. Future programme design should consider these factors. 

Few people were willing to pay water tariffs introduced to ensure sustainable access to clean drinking water. Engaging local decision-makers may help with this. 

Future interventions should explore installing water meters to regulate solar powered water supplies and avoid over extraction. 

There is a need to further invest in the capacity building of water management committees.  

As women and girls also use latrines for washing, future interventions should consider separately draining wastewater from bathing. 

This report is the product of its authors, and responsibility for the accuracy of data included in this report rests with the authors alone.

Programme overview

NCA supports small agricultural producers in Tanzania to increase their economic empowerment, productivity and profits through various interventions in Hai, Moshi, Arumeru, Babati, Bagamoyo and Kilosa districts. Reflecting its 2016–2020 country strategy, NCA has implemented smallholder empowerment (SHE) projects concentrating on vegetable, fruit and poultry production. This model has included micro and next level investments to introduce drip irrigation and other innovations. NCA has also established inter-religious village community banks (IR VICOBA) to supply investment capital. 

Evaluation overview

In 2019, NCA commissioned The Open University of Tanzania Consultancy Bureau to assess the effectiveness, relevance and impact of NCA’s SHE projects in Tanzania. This included analysing the costs, payback period and rate of return for each investment type (vegetables, fruit or poultry). The assessment aimed to identify best practices and beneficial or problematic design criteria and delivery models. Research involved a desk review, project observations, and key informant interviews and focus group discussions among vegetable and poultry farmers.  

Key findings and recommendations: 
Vegetable production has variable costs, depending on the number of vegetable beds and their required maintenance time. 

Tomatoes generated high profit margins in their first season, except in one area. 

NCA should facilitate the availability of irrigation water in drought-affected areas.  

The project should help the poorest people to escape poverty by offering initial, short-term capital loans through IR VICOBA. NCA should encourage more men and youths to join VICOBA to promote their access to capital.  

NCA should educate farmers on the importance of health insurance as only 20% of the sample had cover.  

Establishing vegetable, fruit and poultry clusters could enhance the year-round supply of goods. 

NCA should promote effective collaboration between partners and government officials and provide specific tools that agripreneurs lack.

This report is the product of its authors, and responsibility for the accuracy of data included in this report rests with the authors alone.

Programme overview

Since 2016, NCA, Save the Children (SCI) and local partners have conducted a joint programme in Somalia, East Africa around specific forms of gender-based violence (GBV) against girls. The programme aims to help eliminate female genital mutilation/cutting FGM/C and child early and forced marriage (CEFM).

Phase 2 of this programme (2016–2018) sought to build on phase 1 (2014–2015).

Following a phase 1 evaluation, phase 2 expanded into rural areas where FGM/C prevalence is higher, and was implemented in six districts of Gedo region and Puntland state.

Evaluation overview

In 2019, NCA and SCI conducted a mid-term review to assess their joint programme against outcome indicators, and baseline (2016) figures where available. This evaluation combined quantitative, qualitative and participatory methods and used the reflective methodology to review the programme’s Theory of Change.

Unlike the mid-term review, the baseline study did not include qualitative data or distinguish between different types of FGM/C. This may have affected findings*.

Key findings*

Overall, FGM/C prevalence in the area increased from baseline to midline, especially among girls aged 5–10. There was a 13 percentage point reduction in respondents committing not to cut their girls in future.

FGM/C is increasingly medicalised. 63% of new cases were performed by health professionals, compared to 51% at baseline.

100% of FGM/C survivors indicated that the support services they received had been beneficial.

Although support for child marriage remains high and its incidence has increased slightly, actual incidence appears to be much lower than locally perceived incidence.

Despite increases in these practices, both female and male community members appear to be more aware of the health effects of FGM/C and CEFM.

This report is the product of its authors, and responsibility for the accuracy of data included in this report rests with the authors alone.

Programme overview

From 2015–2019, NCA and partners implemented Prevention, Response, and Outreach to Empower Conflict Related Sexual Violence (CRSV) Trauma Survivors (PROTECTS) in Syria, Lebanon and North Iraq. The programme was funded by the Norwegian Ministry of Foreign Affairs (MFA). It aimed to help conflict-affected, marginalised women, girls and boys affected by gender-based violence (GBV) and displacement to heal and increase their access to appropriate support services. By involving community leaders, men and boys, it aimed to promote social cohesion, prevent violence and help reintegrate GBV survivors in the Middle East into society.

Evaluation overview

Nahla Hassan, Anouchka Baldin, Gulnar Wikam and UIMS evaluated PROTECTS activities in 2016–2019. This evaluation took place in the field, using OECD-DAC criteria and the programme’s results-based framework. Results are published in External Evaluation of “Prevention, Response, and Outreach to Empower Conflict Related Sexual Violence (CRSV) Trauma Survivors (PROTECTS) in Syria, Lebanon and North Iraq”

Key findings

A participatory approach strengthened both project design and implementation. Surveys and needs assessments with beneficiaries increased community ownership and self-efficacy. Interactive community events engaged different groups, especially men and boys, to broaden understanding of GBV and reduce its acceptance.

The contextualisation of this project was based on the realities and cultural norms in each country. This ensured programme relevance and credibility.

The mobile unit model reached the most marginalised people in remote and hard-to-reach locations very effectively, raising awareness of – and linking people to – GBV services.

Establishing static centres to provide services created safe places for GBV survivors, helped engage the community and supported survivors’ recovery and reintegration.

This report is the product of its authors, and responsibility for the accuracy of data included in this report rests with the authors alone.