I spoke to
Professor Mike Pidd this morning about Soft Systems Methodology and public services. Mike was surprised that so few public services have taken up SSM as an approach to improve their practice. I came across an interesting article which Mike had written about a soft systems view of
performance measurement in public services.
Mike referred to three reasons why we might measure performance in public services.
“1. To see what works: this is the use of quantitative indicators as a foundation for evidence-based policy by measuring and comparing the performance of different delivery and policy options. To be properly done, this requires the careful use of statistical methods and designed comparisons.Thus, if there are several options available, the performance of each can be measured and this information can be used for comparison to determine the best way to proceed. Alternatively, the impact of a single policy might be measured by comparing its costs and benefits to see if it is worthwhile.
2. To identify competences: this is the use of quantitative indicators to identify good performers (and, by implication, poor performers). Often the resulting performance data are published; for example in school league tables, the star ratings of NHS hospital trusts and, more recently, the performance of social service departments in UK local authorities. An ultimate aim of such measurement is to encourage the transfer of knowledge and expertise and to inform services managers about how well they are performing. The idea is that the identification of high performers will enable poorer performers to learn how to improve it also provides positive feedback to the high performers.
3. To support public accountability: this rests mainly on the publication of performance data to allow members of the public to see whether services are being delivered properly and offer value for money. Since New Public Management separates policy from operations, it is clear that the accountability loop must be closed in some way; otherwise there will be no link between policy and action. Though this measurement may, in theory, be for public use, politicians are intimately concerned with this aspect of measurement since part of their future may depend on it. Whether the general public is so interested is a moot point. The indications from local authorities (Miller, 2003) and health care (Marshall et al, 2000) suggest that public interest is rather limited.”
To summarise the three accepted reasons for performance measurement:
- To see what works
- To identify competences
- To support public accountability
I completed a draft of
The Child at the Centre last week which set out a draft format of our Department Standards and Quality Report using a wide range of performance indicators. The next day I explored the potential tension which can exist between what I described a
soft and scientific models of change but that they could be used in a complimentary manner.
The challenge when reflecting upon performance indicators is that they can dominate and overwhelm the person or group to whom they refer, with attention naturally focussing on the weakest areas – which can debilitate the organisation and their ability to make any effective response to the area requiring attention.
This took me back a problem I created at Dunbar Grammar school when we set up a departmental review process. It took the form of a peer review of a department by colleagues from the school. In one of the first departments to go through the process we identified many strengths and some areas which required development. However, when we presented our report to the department we attached numbers to each of the various performance indicators e.g. 1 = Unsatisfactory 4 = very good. Despite there being a very positive narrative the members of the department focussed on the numbers and wanted to engage in debate about the level allocated by the team – despite agreeing with most of the narrative. I learned a very important lesson that day – that in such circumstances we should negotiate the levels with the team by reflecting upon the narrative and the evidence which supports it – in other words people must take ownership of the self-evaluation process. However, there is a need to provide some form of comparative indicator to enable people to make some judgement about the overall quality of the work under consideration – and so I would stick with the proposed
The Child at the Centre model – but doing so within a culture where people take direct responsibility for self evaluation – with the role of the authority being primarily one of validation.
I've always operated on the basis that I would rather be aware of any weaknesses before any external form of evaluation took place – such as a peer review, authority review or HMIe inspection – I believe that such an approach provides an excellent foundation for progress – regardless of how negative the self-evaluation might appear at first – at least we were in control.
In my next post I'll explore some of the problems thrown up by the way we currently use performance indicators.