Implementation – Monitoring and Evaluation
19.4 More on performance indicators
As it has been mentioned above, performance
indicators can be
defined as: “variables whose purpose is to measure change
in a process or function”. Indicators used by the promoter
(the contracting partner, usually local government) must be clearly
defined, accessible and transparent.
Quantitative and qualitative indicators
Indicators may be quantitative or qualitative in nature. The
average cost incurred and the time taken to complete a particular
item of work are both quantitative indicators. By contrast,
a community member’s perception of his or her satisfaction
with the results of a particular community procurement process
is essentially qualitative in nature. For comparison purposes,
it will usually be necessary to find some way of placing
some quantitative value on such qualitative indicators.
It is arguable that processes that cannot be measured cannot
be managed. This statement might be questioned by a group
that combines together to provide some basic item of shared
infrastructure, for example. However, it is true that in some way
such a group does measure costs against perceived benefits.
certainly widely accepted by governments that quantitative
information is needed if choices are to be made between a range
of possible development options. A common response to the need
to quantify indicators is the use of ranking scales in which,
for instance, a person’s
satisfaction with a process, activity or situation might
be ranked on a scale ranging from very happy, through happy,
indifferent and unhappy to very unhappy. Such ranking scales must
be set up at the time that indicators are identified and introduced
to the overall monitoring and evaluation system.
Process and output indicators
Indicators may relate to processes and the outcomes from those
Process indicators may relate to both the quality of the
process and its progress. Indicators relating to the quality
of a process might cover such questions as:
? Who is involved?
? What role are they playing? and
? What are the power relationships within the group?
Those relating to progress will be concerned primarily
with the question “Is the process occurring as planned?” Managers
can use information on the quality of a process as the
basis on which to amend and improve that process. Such
information can also be used to assess the outcomes of
that process and the possibility that this assessment may lead
to changes in the design of future initiatives. Indicators
of progress are immediately useful to the managers of a project
or process, but they may also help in the assessment of
the efficacy of that process.
Indicators relating to outcomes are required by managers
primarily to assess the success of projects and to measure
the achievements of those projects. They are intended to
help planners to determine the extent to which the project
or process achieved its objectives. At the project level,
the concern will be with whether it has achieved both its
specified outputs and its overall purpose. At a wider level,
there is a need to consider the overall impact of projects
and programmes on fundamental indicators – for instance,
those that measure the level of poverty in society.
Check what information the
performance indicator provides
When choosing performance indicators for service delivery,
local government managers need to take account of the following
factors for a performance indicator PI(x), which
is intended to measure the performance of service X:
- Is X an area which falls under the control of the user?
- Will the PI(x) measure what is needed?
- Will problems in area X be detected by the use of PI(x)?
- Does PI(x) give an idea of the magnitude of the problem?
- Is data available to compute PI(x)?
- Is PI(x) accepted by the people involved?
- Are there any other indicators that can help identify the cause
of the problem?
- Who will use PI(x)?
Performance indicators in monitoring and evaluation
Performance indicators are normally used in one of two ways.
First, they may be collected at regular intervals to track
the way in which a system is performing or an activity is
unfolding. Alternatively, they may be used to assess the change
resulting from a particular activity or project. In the first case,
they are being used to monitor the progress of a process,
while in the latter their purpose is to evaluate the
outcome of a project or process. Evaluation requires that the situation
be assessed both at the beginning and at the end of the project
or process. This suggests that there is a need to collect baseline
data relating to the proposed performance indicators before
a project or process starts [Tool 4]. Information collected before
a project or process begins can and should also be used to
inform decisions on what is to be done. This information should,
wherever possible, include data taken from previous initiatives
so that lessons from the past can inform the new venture.
<– Clearly defined performance criteria/targets
<– Selected and benchmarked performance
The above uses of indicators are linked strongly with the
project cycle of identification, implementation, operation
and assessment. The use of information to inform decisions
is an essential aspect of project identification. Monitoring systems
are required to ensure that implementation and operation
proceed as planned, and to provide the basis for corrective
action if there is a problem. As already indicated, process indicators
will be important in this respect. Evaluation systems are
required in order to assess the achievements and impact of
a project or programme. Evaluation requires the assessment of change
and hence is dependent on the existence of adequate baseline information;
this in turn means that performance indicators should be
in place at the beginning of any initiative. Timelines are
essential in the development and application of indicators.
Performance indicators and planning
A range of stakeholders can use indicators to assist them in
planning projects, programmes and policies. Planning starts
from the assumption that the evaluation of completed projects,
programmes and policies should provide an input into plans
for the future. In effect, evaluation of what has gone before
becomes part of the appraisal process for what is to come.
Most of the indicators presented in this study are concerned
with performance at the level of the individual contract.
However, it is important to recognise that, when brought together
and analysed, the information obtained through such indicators
can be important in shaping programmes and policies. For
instance, indicators of the local employment resulting from
specific schemes featuring community-partnered procurement might
provide a strong rationale for the development of changes in policy
to encourage such partnerships. A related point is that, while
individual schemes may result in improvements in the local
situation, assessment of the cumulative effect of a number of projects
is necessary if overall impacts of different approaches to the
provision of services are to be assessed.
Key points when using performance indicators
To summarise, the following points need to be kept
when using performance indicators.
1. They should be truly representative of
the quantities and characteristics that they are intended to represent.
2. They should be verifiable; in other words, it should be
possible to check that the values of the data or indicators
presented are reported accurately.
3. They should provide information that can
be used by decision-makers. As already indicated, this will often mean that they are
4. The information must be available in time to influence
5. They should be linked into systems that
allow feedback of information into the decision-making process.
Professionals involved in service provision have been traditionally
interested in the efficiency with which, for example, infrastructure
facilities have been provided. Their concern has been with
the cost of works, the time taken to complete them and their
quality. As already indicated, research is now taking a wider
perspective, exploring indicators that relate to the “softer” impacts
of service delivery. Some of these impacts are sensitive to the
process of service procurement and delivery; some are process independent.