A pollster warns: beware of the polls

Labour’s ministers have been led astray by the data they rely on. Time to change their ways

Trust me: I’m a pollster.

Doubtful? Bear with me. I have advised many campaigns and political parties over the years. I know what polls can do. As important, I know what they can’t. A big weakness of Keir Starmer’s government is that it has failed to tell the difference.

In the past few days, matters have come to a head. Maybe the lesson has been learned at last. If the policies on child benefit and winter fuel allowance are reversed, it will be good news not only for hard-pressed families but for anyone who wants Downing Street to do what’s right and not be bullied by flawed research.

The winter fuel allowance disaster is the most clear-cut example of what has gone wrong. Last July, polls and focus groups found widespread support for removing the annual tax-free payment from the better off. Keep it for the people who need it most was a popular idea, especially among Labour voters. Immediately after Rachel Reeves announced that most people would lose the allowance, YouGov asked:

Q. Do you support or oppose winter fuel payments to pensioners being means tested, so they only go to those on pension credit or other means‑tested benefits?

Among the general public, supporters outnumbered opponents by 47-38 per cent. Labour voters backed Reeves more decisively, by 61 to 25 per cent.

By mid-October, YouGov was telling a much gloomier story. When people were asked in an open-ended question to write down the worst thing Labour had done in their first hundred days, by far the biggest number said the decision on winter fuel payments. It was the choice of 34 per cent of the general public. Among Labour voters the figure was the same. This was an exceptionally high number for this kind of unprompted question. The next most widely cited complaint was the personal freebies that Starmer and others had received, cited by just 7 per cent of the public and 12 per cent of Labour voters.

The figures from Ipsos four months later were just as bad. They asked whether “the ending of winter fuel payments for pensioners who do not receive Pension Credit” was the right or wrong decision. Among all voters the verdict was: right 25 per cent, wrong 67 per cent. Labour voters divided 42-53 per cent.

That looks like a clear example of voters changing their minds. But wait a moment . Earlier this month YouGov repeated the question it asked last July. Its figures were remarkably similar to ten months ago:

So what has been going on? YouGov and Ipsos were testing the same policy choice: winter fuel allowance for benefit recipients, not for other pensioners. The gulf between the two sets of results is far too wide to be explained by random error or different sampling methods. The only significant difference was in the wording.

YouGov talked about “means-testing” and mentioned both pension credit and “other means-tested benefits”. By mentioning means-testing twice, and appearing to expand the range of recipients, it tapped into the widely held view that those who didn’t need the payment shouldn’t get it.

In contrast, Ipsos said nothing about means-testing. It asked about “ending” payments, and restricted the group of continuing recipients to those who specifically receive pension credit.

This is one of those cases where the precise wording matters. More than that, it demonstrates one of the great and permanent political truths. In any political controversy, the way choices are framed is vital to the way voters view them.

Hence the government’s initial mistake. Polls and focus groups appeared to show that it could both save money and win voters’ approval by framing the policy as withdrawing winter fuel payments from those who don’t need it. But within days of Reeves announcing the change, the public debate was framed differently: as punishing elderly people who struggle to make ends meet but were not quite poor enough, or too proud, to claim pension credit. It’s not that voters changed their minds after all the fuss that Reeves generated. They didn’t. The problem was that the initial research did not dig deep enough. It led ministers astray.

That’s the technical part of the failure: misinterpreting data about the public mood. But that flows from a more basic failing. Research was used to make policy, not to work out how to win the argument about its merits. What Reeves and Starmer should have done was decide both the principle and detail of the policy first. Who would be affected and by how much? If the right policy was going to be expensive, where would the money come from? If it was still necessary to hurt the hard-up, how would they justify what they were doing? Could they mitigate the hardship?

If that had all been done first, polls and focus groups could then have come into their own, to help ministers argue the merits of their decisions.

The same should have been the case with immigration and Europe as well as child benefit and the winter fuel allowance. Ministers have cowered in the face of shallow research when they should have decided what was right for Britain and then worked out how to win the public over. (Past Substack posts of mine have assessed the state of the debate on Europe and immigration.)

Reshaping the role of research would offer Starmer a further benefit. Voters aren’t fools. They may not follow the twists and turns of each policy, but they quickly build up a picture of our political leaders: whether they are honest, competent, fair-minded and so on. When Starmer sets out his views, the swing voters that decide elections sense whether he genuinely believes in what he is saying or pulling some cheap polyester over their eyes – which is the inevitable result of basing policies on flawed research rather than on clear principles and solid values.

One Guardian headline last Friday offered what we can only hope is a last hurrah for the old ways. Above an exclusive story by Pippa Crerar, the paper’s political editor, the headline told readers:

“No 10 delays child poverty strategy with tens of thousands more facing hardship; Flagship policy put back until at least autumn amid fears cost of removing two-child benefit cap will outweigh political benefit”

We all know that “political benefit” is in ministers’ minds at all times, whichever party is in power. But to see the trade-off stated so bluntly – putting reputation management ahead of rescuing families from poverty – is startling, especially from those at the heart of a Labour government.

Since that story events have moved fast. U-turns on both child benefit and winter fuel allowance are underway, not least in order to fend off a revolt by backbench Labour MPs. Some reports suggest that these changes are defeats for Morgan McSweeney, Starmer’s chief of staff and passionate poll-watcher. Far more important than the fate of individuals in Downing Street is the way Starmer and Reeves take big decisions from now on.

Good research can and should contribute to a healthy dialogue between the government and the electorate. It is never wrong to find out what voters think. But deciding policies is another matter. A mature democracy needs leaders not followers running the country. The best thing that could happen to Labour is for that to be the biggest lesson it learns from its first year back in office.