How do we Evaluate Programmes?
in-itinere evaluations of the programme of programmes (Country Learning Reviews)
If you are a programme manager, you can also utilise the learning from in-itinere or ex-post evaluations of projects in order to make changes to your programme or feed it into new programmes. Such an in-itinere evaluation can also be conducted for a programme of programmes. In development aid organisations, such an in-itinere evaluation of the programme of programmes is also called a Country Learning Review. The purpose of a country learning review is "To reflect and learn from what we do and how we do it". It is usually an annual reflection made by the programme and project staff and partners in which they step back and review their work at the level of the programme of programmes.
Besides analysing the information emerging from an in-itinere evaluation of programmes, you as a programme manager and the programme teams must analyse the changing socio-economic and political context and the current trends that directly affect programmes and projects. You and our team also analyse the sectorial trends, share the learning from programmes, identify the successes and failures, the institutional strengths and weaknesses, whether the programme is still making an impact or whether there is need to respond to changing contexts and trends.
In case the programme manager and the programme teams (validated by beneficiaries/clients) decide that the changes in the context require a change in the content and approach of programmes- both results and activities- they will have to make a change in the programme strategy or in the strategy of the programme of programmes.
Before changing the programme strategy, it would be useful to discuss the context and programme in details with internal stakeholders (programme team, concerned head office teams) and external stakeholders (other actors in the sector, donors, government) so that you can make informed decisions on the new programme directions.
The senior management of organisations (NGOs, private companies, government) translate the wider strategy into programmes and projects and presents it either to the public (as political agendas) or to donors. In order to identify donors, the programme manager can, along with the fund raising team, conduct a Donor Mapping exercise. The programme manager and fund raising manager meet different donors and present with the profile of the organisation. (See a sample Profile)
This learning event helps in making decisions about the organisational niche, the challenges we face, future direction of programmes. While Monitoring reviews are done at a project level and then at a programme level, Country Learning Reviews are conducted only at programme level.
Country Learning Reviews address the need to improve team learning processes and to link these to stronger context analysis, to reviewing change theories being used in programmes and to feed learning into decisions about our programme to make it more responsive and relevant to the country we are working in. Run in a collaborative way with partners and 'external challengers', this process includes asking questions about what we do and how we do it, what effects this has, examining the theories and assumptions that underpin our programme, as well as a review the external environment. This helps identify where the organisation can be adding value in the context, the “unique leverage point” as a an organisation.
Typically the Agenda of a Country Learning Review includes:
see a sample report of a Country Learning Review conducted by Oxfam GB in Afghanistan, 2008
The biggest costs are likely therefore to be for a 2-3 day meeting involving up to 20 people and an external facilitator to help the Country Director and other key staff design the overall process and the meeting itself.