How do we Monitor and Evaluate Projects and Programmes?

See the issue:   How to design and manage successful cooperation programmes?

_________________________________________________________


for a more complete treatment of the subject see the Wikibook Book.jpg Evaluating Development Cooperation
 


 

We said before that in case of development cooperation activities, where there are a number of actors co-operating in a non-hierarchical structure, "planning" and of "working along the plans" is an essential prerequisite for success. And that actions should be "beneficiary based", i.e. designed and managed to serve the need of the target population (and not the needs of the donors or of the employed organizations). Now the question is: how do we verify that plans are really designed throughparticipated approaches? How do we know if the organisations are really "walking the talk?". How do we find out is the project results are being used for sharing knowledge and improving the program cycle management?

The subject dealt with in this chapter of the handbook is more extensively debated in a separate Wikibook Book.jpg Evaluating Development Cooperation

Contents

  [hide] 
[edit]Monitoring

Project monitoring is done by project managers who need to know if activities are being undertaken along the plans, within the time and resource allocation that were decided and if they are delivering the expected outputs. In this sense monitoring is identical with performance measurement and belongs to the project execution and control phase of project management. Monitoring reports are done by comparing the project achievements against expected results, timelines and costs. It ansers to questions like:

[edit]Evaluation

Project evaluation is different: it is a judgment about the capacity of the project to contribute to the program objectives. In the project evaluation we see if the outputs delivered are consing to the expected outcomes. In the project evaluation we see if we are really inducing the changes that we would have liked so that the factors determinate poverty and suffering are diminished and development is facilitated. Evaluation is based on monitoring but adds a "judgment" on the correlation between activities performed, outputs delivered, changes induced objectives achieved and impacts obtained upon the factors generating the problems and opportunities that motivated the project. In evaluation we want to see if we are having the outcomes and the impacts that we wanted to have. While the delivery of project outputs is the responsibility of the project team, the achievements of project outcomes and objectives depends also by the way stakeholders utilize the project outputs to interact with the rest of the community and contribute to achieve the project objectives.
 

From the monitoring we learn if the activities are producing the expected outputs, that is a way of finding out if people employed are doing the job they were asked to do. When we evaluate we want to see if the project deliverables are leading to the expected project outputs.

Activities-Outcomes-Impact

These judgment are possible as far as there is the possibility of comparing what is happening to what was expected to happen: therefore in order to carry out an evaluation we need at least two commonly visualized scenarios: the one that was expected (the program/project plan) and the one that is factual (the facts resulting from monitoring). When comparison highlights difference between expectations and results, evaluation will attempt to find the causes of such discrepancies, attributing them either to lack of capacity of the implementing team, or lack of vision of planner, or to unexpected external factors, or other factors as it may be the case. If the project is still running evaluations report will possibly indicate how to redesign the remaining actions in order to conciliate expectations and results. If the project is already completed, evaluations will indicate the lessons learned from the project to be considered in the successive planning and implementation of similar projects. Evaluations are therefore an indispensable means for programme cycle management and constitute an essential element of its actions.

[edit]Authenticity

Since evaluation is a highly technical job, in development cooperation actions specialized professionals are employed as consultants by sponsors and donors. The problem however is that they are economically dependent to the same stakeholders who would like to be justified by the evaluators. They approved plans, the spent money for them ... and they want to say that projects worked! Traditionally from the evaluation reports they want data to pass over to communication experts that will have to convince the public of the results obtained by the projects and the positive changes that occurred in the lives of the beneficiaries. Hence big use of rhetorical language, which however often leaves the public quite cool, because the public do not trust those who justify themselves.

[edit]Communication

If project sponsors really want to learn about the project results and if they want to establish a healthy communication climate with all stakeholders then the rethorical approach should be removed and the dialogue modality should be chosen, which is really in line with the spirit and the principles of development cooperation. The persons who are trusted by the stakeholders should be enabled to build informed opinion and freely express independent judgment. In this manner they can learn about the projects and they can share their learning with their public.

As it is becoming ever more evident that communication has no authority if if its content is not developed in a process of reciprocal empowerment, then project donors are increasingly oping up to really independent evaluators and are asking independent media to come on board and express their unpiloted judgments. (a good exaple of this approach has been the sponsorship of EuropeAid to the project Eugad that has created the base upon which this handbook has been developed. (more in ⇒ The Vrinda project").

[edit]Learning
 

Responsibility and challenges of mass media

While working for Eugad we learned that in order to adequately inform the public about development cooperation programmes it was not sufficient to take along in the field independent journalists and leave them free to express their views because in any case journalist tend to follow the conventional logic of the main media channels that tend to repeat used stereotypes and are mainly at the service of advisement sponsors rather then the public itself. (see the issue ⇒Media and the international communication climate. In order to bypass the media gatekeepers we decided to avail of the potentialities of Internet and start the The Vrinda project where development actors share their views and their knowledge in order to inform each other and enable better informed decisions. The final aim of evaluation is to produce the learning that enables decisions based on informed judgments. We then learned that evaluations are really possible only if carried out in a dialogical communication climate. The rhetorical modality by seeking consent (from followers) does not really generate new learning, even if carried out by highly skilled professional evaluators; the participated approach to evaluation fosters a dialogical modality of relationship amongst all development stakeholders and enables the articulation of learning setting the stage for increased cooperation and dialogue.

[edit]See also

Issues icon.jpg ⇒ Does international aid really benefits the target populations?
 

In other sections of this handbook
 

Book.jpg Impact Evaluation


On other Wikibooks
 

Wikibooks Evaluating Development Cooperation
 

On Wikipedia
 

600X WIKIPEDIA LOGO.svg Evaluation
 


 


 

 

 

 

Defining evaluation  

Communication and Evaluation 

Guidelines 

 

 

Common Myths and Misconceptions About Evaluation

The ability to support and understand the many different contexts/cultures in the developing world

 

 

old files:

 

Templates

 

example of evaluation reports  :  http://www.asiandevbank.org/Evaluation/default.asp;  http://www.europeanevaluation.org

Other resources

A Web site on evaluation for the francophone : http://evaluation.francophonie.org/

Methodologies:  http://www.netricerche.net/;  http://www.delfo.net/methods/methods.htm

http://earth.prohosting.com/elecon/evaldevel/evaldevelopment.html

  http://www.developmentinpractice.org

 

 

 

Actors: http://www.assirm.it/associ.htm;  http://www.esomar.nl/directory.htm;  http://www.ipsos-explorer.it/cosa/frame.htm  (In basso a destra clicca su metodi e strumenti); http://www.gpf.it/cosafa.html;

 

 

 

 

 

-------------

 EU resources

 

this is an interesting site with good documents on evaluating development assistance,

 

V

 

The Network on Development Evaluation is a subsidiary body of the Development Assistance Committee (DAC). Its purpose is to increase the effectiveness of international development programmes by supporting robust, informed and independent evaluation. The Network is a unique body, bringing together evaluation managers and specialists from OECD development cooperation agencies and multilateral development institutions.

About the site:

This site explains the aims and objectives of the DAC Network on Development evaluation and our current work priorities and initiative as well as details on the work of our 30 member agencies.

The site is hosted and maintained by the DAC Secretariat on behalf of the members of the Evaluation Network as part of its knowledge management work.
 

Find information on:


- The Network's:

- Work of the Network members: 

Send instant messages to your online friends http://uk.messenger.yahoo.com

Communication and Evaluation

 

 

The Challenge of Designing, Executing and Implementing the Development Programmes and Actions 
TVP Manual - Section 2: Development Communication in Action: steps and tools -

see Manuals General Index


 

Project management demands a free flow of communication with and among project team members, and internal and external project stakeholders. The project team needs frequent information from each of its team members to complete and improve the project and to understand the needs and expectations of the project's beneficiaries. Project Communication management is the systematic planning, implementing, monitoring, and revision of the exchnage of information amongst the project team and the project stakeholders . Project communication management aims at timely and appropriate generation, collection, dissemination, storage, and ultimate disposition of project information and knowledge.
 

Types of Project Communication
- Internal communicating within the project team 
- External communication with upper management, beneficiaries, and external players 
- Close-out reporting 
 

Communication among all project stakeholders is one of the main factors for the project success. It is a prerequisite of getting the things done in the right way and in the right time. Knowledge is power: sharing knowledge is reciprocal empowering amongst project stakeholders.


A good project plan has, in the methodology section, a sub-section dealing with communication management. A good project implementation plan always contains a communication plan; in any case it is important to plan communication at an early phase of project execution.   The communication plan describes how the information a and communication needs of project stakeholders will be met: a communication manager will design, and implement such a plan; thereafter s/he will evaluate how efficient and efficacious communication has been as a support activity facilitating all other project tasks. 
 

Never underestimate communication in project management. Communicate well, and the project will succeed. Communicate poorly, and even the most efficient efforts may be misperceived, misunderstood and poorly valued.
 

In order to prepare and distribute properly the right information to the right stakeholders, the communication manager needs to:

  • Analyse the communication needs of each stakeholders
  • Identify the information for fulfilling the information needs of each stakeholder
  • Identify the Method and the Effort Required
  • Prioritise the Communication Options

Information distribution includes project performance reporting, but is not limited to that: it entails also all tasks required to satisfy the information needs of all project stakeholders.
 

Proper information delivery to external stakeholders is essential for the following features of project execution. 
 

  • Manage Acceptance of Deliverables
  • Manage Organizational and Behavioural Change
  • Gain Project Acceptance


 


In the structure of the project, communication management is considered one of the facilitating processes (along with quality planning, staff acquisition, risk response planning, procurement planning, solicitation planning). "Facilitating" does not mean unessential or optional: it only means that it is a process that varies in the sequence, is performed in parallel with other activities, have a two-way feedback loop with many core processes. (core processes of project management instead are performed sequentially and are divided into three main phases, i.e. project planning, project execution, project closure).
 


 


 

Communication, management and organizational development

With all focus on project management techniques, such as planning, scheduling, cost management, etc., it's easy to forget that communication occupies 90% of a project manager's job time. This includes conflict management, resolving ethical dilemmas, team building - all the soft skills that can make or break projects. In fact, most project failures are in some way tied to a lack of communication. Many projects are perceived as failures due to poor communication, even if all the technical components are done right. Likewise, there are projects where many mistakes are made, but the project is considered a shining success because it was well communicated.
Organisations are being increasingly asked to measure, i.e evaluate, their performance, in terms of results: i.e. relevance, impact, efficiency, effectiveness and sustainability of their programmes; humanitarian, development, campaign and public programmes alike. Stakeholders, including donors, are also demanding greater transparency and accountability from organisations. This requires that projects be developed within a logical framework (or a results based logic model) through the project cycle management process, be stakeholder participated at each stage and be based on the local context, needs and problems. Since stakeholders involved in each step are many and varied, the extent to which participation is fair, representative and constructive will depend on successful communication among them. Successful communication will directly determine the quality of the project and the relevance, impact and sustainability of project results. This, in turn, is directly dependent on the capacities of the stakeholders, including organisational staff. These capacities of the “what” and “how” of project development, implementation and evaluation, are developed among stakeholders through knowledge sharing, training and on-hands mentoring and support, all of which need communication skills in varying degrees.

So, here, we are looking at communication as “the tool” used by managers for leading teams and programmes towards a results based programme or project. It is also the tool by which lower level employees can follow their leaders in a spirit of authentic collaboration. 
From this perspective, communication is seen in a wider sense than the usual one. In this extended perspective, “communication” is not just an “exchange of information”: it includes all human interactions that make it possible to follow the steps of planning, implementing and evaluating projects. Communication is not the be-all and end-all of project management: technical aspects of planning, budgeting, procurement, etc. are equally important. But it is good human interaction (i.e. communication) that ensures objectives are pursued, stakeholders are motivated, teams work according to plans in a spirit of solidarity, the organization and its persons are empowered and a process of continuous learning is activated in the organization. (for further reference please see PMI web site www.pmi.org).

On the one hand, this new importance of communication implies that more training should focus on developing communication skills and learning communication tools. On the other, it is training itself that in learning organizations becomes a form of internal communication. Training and development within organizations con be seen as a strategic tool for the project/programme purpose and as a forward-thinking vehicle for change. This new function of training is much more than just the traditional role it once played in organizations. Knowledgeable about needs assessment and familiar with the latest training technologies, training professionals assist the organization in communicating its mission, goals, and objectives, as well as facilitating the achievement of the mission. More and more organizations are giving space to the training function to act as an independent project/programme purpose, receiving its funding from the other functions. Though this is not yet the norm, the move towards having the training department function like a project/programme purpose is growing rapidly.
 

Why social enterprises need to plan communication management.

NGOs face a formidable communication challenge, as they become primary actors of social change. Change can be seen as an opportunity, but it also generates resistance. Hence, the need for a communication strategy that generates consensus for desired changes. Communication planning also enables spokespersons of the organization to respond adequately to objections and hesitations of those who believe that change is not desirable, not feasible or badly managed.

 See Guideline: Why do organisations need to plan and manage their communication? 


Project management demands a free flow of communication with and among project team members, and internal and external project stakeholders. The project team needs frequent information from each of its team members to complete and improve the project and to understand the needs and expectations of the project's beneficiaries. Project Communication management is the systematic planning, implementing, monitoring, and revision of the exchnage of information amongst the project team and the project stakeholders . Project communication management aims at timely and appropriate generation, collection, dissemination, storage, and ultimate disposition of project information and knowledge.
In the structure of the project, communication management is considered one of the facilitating processes (along with quality planning, staff acquisition, risk response planning, procurement planning, solicitation planning). "Facilitating" does not mean unessential or optional: it only means that it is a process that varies in the sequence, is performed in parallel with other activities, have a two-way feedback loop with many core processes. (core processes of project management instead are performed sequentially and are divided into three main phases, i.e. project planning, project execution, project closure).


 

Attention: There is an important difference between Project communication and organizational communication . A project is a temporary and one-time endeavor undertaken to create a unique product or service, that brings about beneficial change or added value. This property of being a temporary and a one-time undertaking contrasts with organization processes, that are permanent ongoing perations aimed at creating the same product or service over and over again. The communication of these two systems is often very different and requires varying technical skills and philosophy, hence requiring the development of project management. Organizational processes instead are recurring and they regenerate themselves at the end of each completition.


A good Project plan has, in the methodology section, a sub-section dealing with communication management. A good Project implementation plan always contains a Communication plan; in any case it is important to plan communication at an early phase of project execution.

In the the project execution phase, communication management is the implementation of the plan and involves essentially two processes: preparing (producing) and sharing (distributing) information.
In order to prepare and distribute properly the right information to the right stakeholders, the communication manager needs to:
 

  • Analyze the communication needs of each stakeholders
  • Identify the information for fulfilling the information needs of each stakeholder
  • Identify the Method and the Effort Required
  • Prioritise the Communication Options.

Information distribution includes project performance reporting, but is not limited to that: it entails also all tasks required to satisfy the information needs of all project stakeholders.
 

Proper information delivery to external stakeholders is essential for the following features of project execution.

  • Manage Acceptance of Deliverables
  • Manage Organizational and Behavioural Change
  • Gain Project Acceptance


 

Communication among all project stakeholders is one of the main factors for the project success. It is a prerequisite of getting the things done in the right way and in the right time. Knowledge is power: sharing knowledge is reciprocal empowering amongst project stakeholders.
While executing the plan, the Project Manager must be aware of how the organization will use the information, and whether the plan is effective. He/she must be flexible and ready to modify the plan if portions of it are not working as expected or communications needs change within the Performing Organization. 

The Project Manager should periodically assemble the Project Team to review the status of the project, discuss their accomplishments, and communicate any issues or concerns in an open, honest, constructive forum. These meetings are ideal opportunities for the Project Manager to gain insight into the day-to-day activities of Project Team members, especially if the team is large and individual interaction between the Project Manager and each team member is infrequent. 
 

Throughout the project, team members assess the project communication plan. The original plan provides adequate information to manage project communication. However, it is important to verify whether the project communication plan is effective.
Periodically, the project manager asks the project stakeholders if the project communication is sufficient to suit their needs. In some cases, project stakeholders may need greater detail or more frequent delivery. In other cases, certain stakeholders may need summary information, or may request notification only if problems arise.


As a project progresses, events may occur to alter the way information is accessed or change communications requirements. During Project Execution, the Project Manager and Project Team must again review whether the Communications plan is still current and applicable to the present phase of the project.While executing the plan, the Project Manager must be aware of how the organization will use the information, and whether the plan is effective. HS/he must be flexible and ready to modify the plan if portions of it are not working as expected or communications needs change within the Performing Organization. Of the many mechanisms available to the Project Manager, Status reporting is particularly useful for communicating the performance of a project.


As a project progresses, events may occur to alter the way information is accessed or change communications requirements. During Project Execution, the Project Manager and Project Team must again review whether the Communications plan is still current and applicable to the present phase of the project.   In addition to having a solid Communications Plan in place, it is the responsibility of members of the Project Team to exercise good communication skill . Communication skill is critical to keeping your stakeholders informed, supportive, and enthusiastic. Smart planning and consistent information delivery keeps your project on track and helps avoid confusion.


When composing correspondence, progress reports, meeting minutes, etc., and when speaking with individuals face to face, the team members are responsible for clear, unambiguous, and complete communication of information. The receiver, in turn, must be sure information is not only received correctly and completely, but that it is understood. During Project Execution, the Project Manager, Project Team, and Stakeholders will share information using a variety of communication mechanisms. 


 

Ofthe many mechanisms available to the Project Manager, Status reporting is particularly useful for communicating the performance of a project. 

Project Team members must complete Progress Reports providing regular feedback to the Project Manager. These reports can serve a dual purpose – as a reporting mechanism to the Project Manager and also to the team member’s immediate supervisor. Progress Reports should document detailed descriptions of actual work accomplished and include Team members’ estimates of the effort they feel will be required to complete tasks. Progress Reports should also contain information regarding work to be done in upcoming weeks, and list any issues preventing completion of required tasks.   When correctly completed by the Project Team, the reports are very useful to the Project Manager for updating the Project Schedule, and for anticipating issues and proactively planning ways for their resolution. Using the Progress Report prepared by the Project Team, the Project Manager should complete a Status Report to be presented to the Project Sponsor. In this report, the Project Manager measures the “health and progress” of the project against the Project Plan. 
 

Project Team members must complete Progress Reports providing regular feedback to the Project Manager. These reports can serve a dual purpose – as a reporting mechanism to the Project Manager and also to the team member’s immediate supervisor. Progress Reports should document detailed descriptions of actual work accomplished and include Team members’ estimates of the effort they feel will be required to complete tasks. Progress Reports should also contain information regarding work to be done in upcoming weeks, and list any issues preventing completion of required tasks.

  See the collection of tools for Reporting project status
 



 


 


 


 


 
Case Study

Fabio Poggi, the current Councillor for International Cooperation in Modena (Italy), tells us that in Modena the International Cooperation issues are perceived as an important step of the approach of the Municipality towards citizens and Institutions, and a lot of initiatives have been going on related on the matter. However, in spite of that, what seems to be quite difficult is to take objective measurement of the impact that all these activities may have on Local institution and common people. As consequence, defining and monitoring objective Indicators to measure the impact that the activities of International have on Local Authorities and citizens has been requested as a useful way to take measure of the efficacy of the actions applied on a specific community.


Read more his opinion download and more in (italian language) ⇒  his interview

"In Modena there's the presence of: an Assessorship for International Cooperation, an Office for International Cooperation, an Office for European project, a Course for Volunteers of International Cooperation, a specific Fund for International Cooperation, a Centre of Universities dedicated to International Cooperation, many Schools and many Associations as well involved in International Cooperation. Unfortunately, I don’t have an objective measurement of the impact that all these activities may have had on the town, generally speaking. As regards the local Associations and NGOs, I can say however that – looking at their participation to Round Tables and Party and meetings for the magazine Modena Cooperazione and other activities, all arranged for International Cooperation - their own relationship and interactions have been continuously deepening and improving. As regards the impact of International Cooperation activities on common people, I can refer to the number of people asking to attend the Course for Volunteers of International Cooperation, or to the number of people attending the Master for International Cooperation or to number of people visiting the Party for International Cooperation. I think however it’s very important to define, and then to take measure of, the appropriate indicators to evaluate the impact of International Cooperation activities on the common culture, and for that maybe we can find the right ones just working together, inside TVP project”.
 
Image:Media.png

Link to the full written interview: http://www.TVP.eu/wiki/index.php?title=Interview_to_Fabio_Poggi

Link to the audio interview on Youtube (italian language) ⇒  http://www.youtube.com/watch?v=z209IxFsNfI  Part 1, description of activities and their impact on local common people and Associations; evaluation of indicators to measure the impact


 


 


 
Tools

As part of the Internet-based collection of materials for Evaluating Socio Economic Developmentec.europa.eu/regional_policy/sources/docgener/evaluation/evalsed/sourcebooks/method_techniques/structuring_evaluations/stakeholders/index_en.htm,it's available an interesting evaluation related to the principles of Participatory approach: description of methods, evaluation, monitoring, definition of indicators to take measure of effects and impacts, strengths and limitations of the method. Read more here 
 
Image:Media.png

www.TVP.eu/wiki/images/4/40/Tool_for_Participatory_approach_-_Evaluation_of_Socio_Economic_Development.pdf
 


 


 


 

See how these isses were analysed and debated in the Consultation process.
 


 


 


 


 


 
Tools

We are sharing here with you the guidelines and templates that we have used for managing the communication plan of TVP project and that you may find useful for other projects

Templates:

Guidelines
Image:Media.png

http://www.TVP.eu/wiki/index.php?title=How_to_report_project_performance
 

http://www.TVP.eu/wiki/images/a/ae/Project_Status_Report_Template_1.pdf
http://www.TVP.eu/wiki/images/6/6f/Project_Status_Report_Template_2.pdf
 




 

AddThis