A couple of people have asked me this week what I think about the government's new public sector targets (see New Zealand Herald stories here and here, or the DPMC page (with associated fact sheers) here. It's fair to say that I have mixed views about these targets.
On the one hand, having a target gives something meaningful for the public sector to aim for. That's better than an aimless exercise in reproducing the status quo. That is essentially the government's argument, and it is somewhat persuasive.
On the other hand, I am mindful of Goodhart's Law, which essentially says that once you change the rules, players will change the way they play the game [*]. I prefer the version of the Law expressed by the anthropologist Marilyn Strathern (in this 1997 article (gated)), which is: "when a measure becomes a target, it ceases to be a good measure".
Looking at the public sector targets that have been proposed, there is a fair amount of scope for the players to change how the game is played, to ensure their success in meeting the targets (or, at least, looking like they are making meaningful progress towards the targets). Take the example of shorter stays in emergency departments. The target is: "95% of patients to be admitted, discharged, or transferred from an emergency department within six hours". Hospitals could make meaningful progress on this target by actually admitting, discharging, or transferring patients more quickly. However, that is hard work, and probably requires more resources, which they may not have. It is much easier to simply not record patients at the time they arrive at the emergency department (by having some sort of informal queue or waiting list, before their visit is recorded and the six-hour clock starts), or having the patients sit in ambulances (which is so common that there is even a term for it - 'ramping').
Or what about the aim of having fewer people on the Jobseeker Support Benefit. The target is: "50,000 fewer people on Jobseeker Support Benefit". That could be achieved by finding jobs for a net 50,000 people on the benefit. Again, that is hard work for the Ministry of Social Development, and probably requires more resources, which they may not have. It is much easier to simply reduce the number of beneficiaries in other ways, such as by making them more onerous to apply for in the first place, and imposing additional requirements on beneficiaries so that more of them are penalised by losing their benefits (and yes, that has already been announced).
You could tell similar stories about many of the other targets that have been announced. On the one hand, these are areas where society should genuinely seek improvement. Who would argue against having shorter waiting times at emergency departments, or having fewer people out of work? However, the unintended consequence of these targets may be that we get improved on what is actually measured (the number of patients being dealt with within six hours, or the number of people on the Jobseeker Support Benefit), but no meaningful change on what we actually want (hospital waiting times, or the number of people out of work).
None of this is to say that there shouldn't be targets at all, only that we should be cautious about interpreting success based on a single target that can be subject to manipulation. There is a case to be made that we need some secondary measures that allow us to ensure that the public sector is not gaming the primary measures. So, perhaps we measure the number of patients turned away from hospitals, ambulance utilisation, or patient satisfaction with hospitals, and benefit cancellations, 'discouraged workers', and the number of people outside the labour market (or the employment rate). There is no need to attach targets to those secondary measures, but monitoring them might help people to interpret how the public sector is achieving gains in the primary measures. [**]
Without the government keeping a closer eye on a range of secondary measures, in addition to the primary measures that are being targeted, you can see why I have mixed views about these targets.
*****
[*] What Charles Goodhart actually wrote, in this 1975 book chapter (gated) is: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes".
[**] As an aside, what we definitely don't need is multiple primary measures, such as the nine measures of child poverty that are used, thereby allowing the government to cherry-pick the measure that has the most positive interpretation on a given day.
No comments:
Post a Comment