Implementation – Monitoring and Evaluation
19.3 A framework for performance monitoring
Any improvement in the performance of PPP contracts depends
on the existence of a management system within which performance
improvements can take place; this leads to a “virtuous
circle” of improvement, as shown in Tool [19-1].
The key point underlying all of the above is that action
will be taken as a result of measuring performance in any particular
area of activity. It is important to understand that in some
cases the basic management structures at the institutional
level (both within the municipality and the private sector)
are underdeveloped; in such cases it is difficult for the municipality
both to gather relevant information on performance and from
that to develop action plans to improve that performance.
The more detailed framework shown in Tool [19-2] outlines
a number of stages in the process for measuring and evaluating
the effectiveness of the partnership.
Each of the stages from [Tool 19-2] is described in more
Stage 1: Performance evaluation
Action plans to improve service delivery need to be based on
a sound evaluation of the performance of different methods
of service delivery. Performance evaluation needs to answer
specific questions; these questions are relevant to those actors
who are in a position to take action relating to service delivery,
whether it is using conventional tender contract systems or
by involving local communities.
If they are to improve the process
delivery, action plans need
to be based on an evaluation of
Performance evaluation needs to take place against a number
of clearly defined criteria or targets, which have been set
for a particular reporting period and within the local context.
The approach adopted is to define performance indicators
having quantitative or qualitative values, which cover the field
of service delivery. Associated with each performance indicator
is a performance target; the status, or “performance”, of the PPP in meeting its
objectives is then assessed by comparing each performance indicator with its
respective target. This enables performance comparisons to be made:
- between different time periods for a given partnership
- between different methods of service delivery.
This Tool does not attempt to prescribe arbitrary performance
targets; these must be developed within the local context.
However, benchmark values are presented for a number
of selected key indicators.
Investigate the existing performance
systems to see whether they are
sufficiently well developed to
thorough evaluation of the service delivery process.
Stage 2: Performance reporting
Performance reporting provides the essential input to performance
evaluation. It not only reveals whether planned actions
have achieved their objectives, but also identifies common problems
and allows improvements to be built into the system for
the future. The development of a sound performance reporting system
along with the choice of appropriate performance indicators
are important elements in the evaluation of the service delivery
Stage 3: Characterising performance indicators
Performance indicators can be defined as “variables
whose purpose is to measure change in a process or function”.
They provide the information from which performance reports are
compiled, in order to assist in answering the questions posed
by performance evaluation of the service delivery process. Performance
indicators should be selected based on managerial requirements.
Characteristics of a good performance indicator are:
- there is a valid link between the indicator and the question
- problems will be detected by use of the indicator;
- the indicator gives an idea of the magnitude of the problem;
- the information required to define the indicator is readily
Information relevant to the partnership, which performance
indicators can measure, can be grouped usefully as
follows: general, time, cost, quality, inter-organisational
cooperation and partnership and socio-economic issues (guidelines
on each indicator group can be found in Sohail and Cotton,
It is useful to have an overview of some of the fundamental
aspects of a particular contract, for example who
initiated the works; the degree of community involvement;
and who has responsibility for operation and maintenance.
A number of indicators are presented covering issues of
ownership, roles and responsibilities and power relations.
Delays may occur at different stages of the procurement
process depending on the method of procurement, the management
systems in place and/or the personnel involved. Performance
indicators can measure the time performance of the procurement
process at different stages of the contract. Avoidable
delays may be identified and measures put in place to
reduce these in the future.
The cost of the work is generally estimated at several
stages of the procurement process:
- a preliminary estimate is prepared using “rule of thumb” methods;
- a cost estimate is prepared by the engineer, based on an existing
schedule of rates;
- a more detailed cost estimate is prepared by the engineer at
the tender stage; and
- the initial contract cost (the contractors’ estimate
for carrying out the work).
Performance indicators can be selected to
monitor the relationship between the engineers’ and contractors’ estimates
and the final cost of the contract.
If the work carried out by the contractor does
not meet the required standard, it will be rejected.
A single indicator based on the number of times work
has been rejected is proposed as a measure of quality performance. However,
in many situations truly independent valuations of quality
may be lacking, and it is valuable to investigate this as part
of an in-depth participatory assessment.
Make sure that performance indicators
the appropriate characteristic.
Use the groupings as a starting point
to focus attention on
the key areas.
Inter-organisational cooperation and
The ways in which officials and the community interact, and
the degree of co-operation between them, can affect the
efficiency and success of a project significantly.
In addition to the infrastructure itself, a community may
receive a number of other benefits as a result of infrastructure
procurement. These may relate to skills development through
training, increased employment and empowerment.
Stage 4: Selecting performance indicators for the service delivery
There is no fixed set of questions that can be applied to all
situations. The indicators selected by the municipality will
be different for conventional tender contract systems and
community-partnered procurement processes. These in turn will
then vary from place to place according to the local context. Once
the indicators have been selected, the local government must check
that they will assist in answering important questions about
the service delivery process.
It is advisable for the municipality to avoid collecting
large amounts of data (either through objective means or
using participatory techniques), which cannot subsequently
be put to the intended purpose.
Stage 5: Defining and selecting
The nature and form of the information systems is crucially
important for determining performance indicators and developing
performance reports. The local government must know:
- what information needs to be collected in relation to
each indicator; and
- where that information can be found.
Information about the service delivery
should be available through a
management information system.
For community-partnered procurement,
the most important knowledge
the community of users and may not be
recorded in a formal
This knowledge requires a careful review of the different
indicators in order to see whether or not information
will be readily available, and if necessary to plan for
the collection of the information required. Some information
may require processing before it can be used as an indicator.
It is also possible that information about the service delivery
process is available through a management information system;
however, often such information may be poorly developed or non-existent
within the municipality.
Distinguish clearly between
data collection methodologies.
(Likely sources of information for each indicator can be found
in Sohail and Cotton, 2000).
Stage 6: Collecting the information
Collecting information is more complex than appears at first
sight, as there are two distinct methods that can be employed.
- Some performance indicators can be assessed in an objective
manner by collection of performance data; collection
of this information could be done internally using the staff
of the institution or by using external consultants.
- Other performance indicators are based on qualitative information;
collection of this information often requires either
processing of available data or collection of new data using participatory assessments