Evaluating programme implementation in a network organisation
In 2016, ODS conducted a mid-term programme and advocacy evaluation for an international non-governmental organisation (INGO). The programme was implemented in multiple sites (the Congo Basin, South America and China) by this network organisation and aimed to provide humanitarian aid to victims of violent conflict, compile best practices, and disseminate impacts to advocate for systemic change. ODS used a number of methods including interviews, focus groups and desk research and applied our custom Organisational Scan analysis tool to understand organisational competence, and to assess the programme against its objectives.
Our evaluation led to a number of interesting findings. Most notably, it became apparent that the aim to collect best practices for global comparison was not possible given the rigidity of both the programme’s data collection tools, and the network’s organisational structures. ODS recommended a change to the programme’s data collection, suggesting a move to a more intelligent, online data collection template which could capture more granular and contextual information. The result was a repository of good practices which end users could compare on a regional or international basis. Most importantly, the INGO can use the information to design and implement a multi level advocacy strategy by collecting the data, identifying priorities and feeding back the relevant outcomes to the various levels in which they operate, thus creating a constant information loop from which the whole network benefits. Changes in the organisation, its procedures and culture, were proposed to ensure sustainable conditions were in place to be able to achieve the above. These recommendations were also sequenced in a Roadmap format, which proposed responsible parties within the organisation who could take up these tasks.
This way, the evaluation not only looked backwards at the effectiveness and efficiency of the work done but also ahead to its potential for impact and sustainability. As strategic questions and organisational conditions were an integral part of the evaluation as well, this resulted in a holistic process which was useful in offering learning points as well as concrete solutions.