Method for prioritising and ranking
How do we Evaluate Programmes?
used in PPA
Ranking or scoring means putting things in order and it reveals differences
within a population. It helps to identify the main problems or preferences of
36
people, and the criteria they use when deciding in what order to place things.
Further, it enables the priorities of different people to be compared.
Ranking exercises can be used in interviews or on their own and they can lead to
more direct and revealing questions (for example, Why is corruption a more
serious problem than illiteracy?). Tools such as ranking are very useful especially
if they are used to complement semi-structured interviewing. They may be used
either as part of an interview or separately. Pair-wise ranking, for example, helps
identify the main problems or preferences of individual community members, and
their ranking criteria, and enables the priorities of different individuals to be easily
compared.
Ranking can also be useful for collecting sensitive information, especially on
income and wealth. Informants tend to be more willing to provide relative values
regarding their wealth than absolute figures.
There are several types of ranking. The most common ones are:
(1) Preference ranking – Where people vote to select priorities.
(2) Pair-wise ranking – Where a matrix is drawn to compare which is the
preferred of two options.
(3) Direct matrix ranking or scoring – A way of identifying criteria for
choosing certain objects. It can be used as a means of understanding the
reasons for local preferences for such things as tree species or crop varieties.
The criteria are likely to change from group to group. Women and men may
also use different criteria.
(4) Wealth ranking – Where the communities classify themselves into various
categories of well-being, giving reasons for classification as well as how
people move into or out of particular categories.
5.9.1 Preference Ranking
The main objectives of preference ranking are:
To determine the main preferences of individuals and groups within a set of
items.
To compare the priorities of different groups.
Steps of Preference Ranking
(1) Choose a set of problems or preferences to be prioritised. This could be, for
example farming problems or preferences for domestic animals.
(2) Ask each respondent to give you her/his favoured items in this set, in order of
priority. Although there is no limit to the number of items each respondent can
handle experience shows that about 6-7 items would easier to manage. Beyond
this the exercise becomes cumbersome for most respondents.
(3) Repeat the exercise for several respondents
(4) Tabulate the responses.
37
5.9.2 Pair-wise Ranking
This is one form of preference ranking. Pair-wise ranking allows the PLA
team to determine the main problems or preferences of individual community
members, identify their ranking criteria, and compare the priorities of different
individuals.
Steps in pair-wise ranking:
(1) Choose a set of problems, or preferences, to be prioritised.
(2) Choose with the help of participants (or from previous discussion or from
a key informant), a reasonable number of the most important items in this
set5.
(3) Note down each of the selected items on a separate card.
(4) Place two of the cards in front of the respondent/informant and ask
him/her to choose the bigger problem (or more favoured preference)
giving reasons for the choice. Mark down the response in the appropriate
box in the priority ranking matrix.
(5) Ask whether the other of the two problems/preferences is in any respect
more important/more popular than the first. Note down the criteria in the
ranking criteria matrix.
(6) Present a different pair and repeat the comparison.
5 Although there is no specific limit to the number of items one may work with, a number of PLA
practitioners find large numbers of items rather cumbersome to use.
38
(7) Repeat steps 4-6 until all possible combinations have been considered
(and all boxes of the matrix have been filled).
(8) List the problems/preferences in the order in which the respondent has
ranked them by sorting the cards in order of priority.
(9) Check with the respondent whether any important problems/preferences
have been omitted from the list. If there are any, place them in the
appropriate position in the ranking table.
(10)As a useful cross-check to the responses, complete the ranking session by
asking the respondent about the biggest problem (or most favoured
preference) in the list (e.g., “If you could grow only one crop variety,
which one would you choose?”
(11)Repeat the pair-wise ranking exercise for a number of individuals, and
tabulate their responses.
Table 2: Example: Pair-wise ranking of selected crops
Millet G/Nuts Simsim Peas Beans Score Rank
Beans Beans Beans Dura Beans XXX 3 2
Peas Peas Peas Dura XXX XXX 2 3
Dura Dura Dura XXX XXX XXX 4 1
G/Nuts G/Nuts XXX XXX XXX XXX 1 4
Millet XXX XXX XXX XXX XXX 0 5
5.9.3 Direct Matrix Ranking
Direct Matrix Ranking allows the PLA team to identify lists of criteria for a
certain object. It allows the team to understand the reasons for local
preferences for such things as tree species, or crop varieties. The criteria are
likely to change from group to group, and women may have different criteria
for certain trees from that of men.
Process guidelines:
(1) Ask community members to select a set of items/objects that are important to
them and which they want to rank (Examples: animals, tree species, crops,
fruits, etc.);
(2) List the most important items (not too many);
(3) Find out criteria by asking:
“What is good about each item? What else? (Continue until
there are no more replies)”.
“What is bad about each item? What else? (Continue until there
are no more replies).
(4) List all the criteria. You may wish to turn “negative” criteria into “positive”
ones by using the “opposite” expression, for example “vulnerable to pests”
becomes “resistance to pests”.
(5) Draw up a matrix. For each criterion ask which object is best:
- “Which is best, then next best?”
- “Which is worst, then next worst?”
- Of the two remaining ask, “Which is better?”
39
5.9.4 Wealth (or well-being) ranking
This tool can be used to investigate perceptions of wealth differences and
inequalities in a community, to discover local indicators and criteria of wealth
and well-being, and to establish the relative wealth of households in the
community. It can be useful if, for example, one is interested in identifying
who the poor people are and how they may currently be targeted.
Wealth ranking is done by making a list of all households and asking different
people to sort them into categories according to their own criteria of ‘wealth’.
The term ‘well-being’ is often used, since perceptions of wealth usually
include non-economic criteria. Often only three categories are needed: the
poorest, middle and richest (or much better-off). However there is no “right”
number of categories, and communities should always be left to determine
their own categories using their own criteria.
Steps in Wealth Ranking
(1) Identify a group of people who are knowledgeable about the wellbeing
of the community and who represent different categories and
shades of the population (men, women, youth, educated, non-educated,
etc). These will serve as your community representatives.
(2) Explain the purpose of the exercise (for example to understand more
about the different categories of well-being or wealth in a community
with a view to recommending action or targeting).
(3) Ask the community representatives to list on cards all households in
their village. Only one name should be written per card. Remind the
40
team that no household should be missed in the listing (including those
for people who may be single, widows, or those without a home).
(4) Begin with a general discussion on poverty: how it is perceived in the
village, who is affected by it, how it has been changing over time, and
how it affects different people differently.
(5) Ask the team you selected to identify the main categories of well-being
in the village, and to spell out the specific characteristics for each of
the categories. A start could be three or four categories.
(6) Explain how the exercise will be carried out: one person reading cards
of household members and others assigning the card to a given
category.
(7) Follow up the assigning of cards with the question: “Why is s/he in this
category and not in another?” As the reasons unfold, add to your list of
characteristics. As the exercise progresses it may dawn on the
community that there are actually more categories than they had
previously defined. Add more categories as the need may arise.
(8) After all the cards have been assigned, ask the community members
whether they wish to make any changes to their allocations. If they do,
find out REASONS WHY. Following this ask them to count the cards
in each category.
(9) Find out whether in the last 5-10 years (period may vary) some people
have moved from one category to another – “upwards” or
“downwards”. Establish the nature of movement (sudden? Slow?
Other?) and the reasons for the movement.
(10) Establish with the community whether more people have been
moving “upwards” or “downwards” and the reasons why.
(11) Ask one of the community members to transfer the results of
the exercise to a large paper. A pie-chart representing different
categories of well-being may be one way of representing the outcome
of the exercise.
Things to note:
(1) Well-being analysis can be a sensitive exercise. Teams should way the pros
and cons before embarking on its use. Good rapport with the community is
always important for a successful well-being exercise.
(2) A lot of “side discussions” take place during the process of well-being
ranking. These are very important as they tend to bring out peculiar or
“uncharacteristic” issues and comments. Listen very carefully, especially, to
the silent voices.
(3) In undertaking well-being analysis you may find it necessary to separate men
from women, as there are sometimes likely to be different perceptions of
indicators of poverty.