Peers beware

Flawed polling and flawed journalism are distorting the debate over assisted dying

It reads like the real thing: a report in a widely-respected newspaper adding new polling data about a serious controversy that is raging in Parliament.

Indeed, in a way, it is all those things. The paper is The Independent. The headline is genuine. and so is the poll.

The trouble is that the headline is misleading, the poll not what it seems, and the larger truth about public opinion sadly absent.

The issue is the bill, introduced by the Labour MP Kim Leadbeater, to legalise assisted dying for some terminally ill patients with less than six months to live. Having completed its passage through the House of Commons, it has moved to the House of Lords. Peers have set up a committee to examine the bill in detail and report back to the Lords by November 7.

Just now, peers are subject to extensive lobbying, as is entirely proper. Much of it consists of handwritten letters. But last week, peers received something rather different: a professionally produced A4 document on thick, glossy paper. It does not say who has sent it. At the top is the Independent’s masthead. Below that, it reprints (accurately and in full) a story the paper published on September 27.

The headline reads: New poll suggests public back peers in voting down assisted dying bill

The story says that peers opposed to the controversial legislation “have been emboldened by a new poll which suggests that 70 per cent think they have the right to turn it down”. The basis for this claim is this:

“[A] poll of 2,017 British adults by Whitestone Insight found that 70 per cent of those who take an opinion believe that peers have every right to vote against non-government legislation if they consider that it poses a significant risk to vulnerable lives”.

Whitestone Insight is a member of the British Polling Council. Its director, Andrew Hawskins, is a pollster with great experience. Its sample size is perfectly adequate. Despite all that, the sentence quoted above should ring three alarm bells.

The first is that the story named the pollster but not its client. Indeed, nowhere in the story – and therefore nowhere in the information sent to peers – are we told that the survey was commissioned by the campaigning organisation Right to Life. It wants to defeat Leadbeater’s bill.

The second alarm bell is the reference to the 70 per cent “of those who take an opinion”. Those words do a lot of heavy lifting. As Mark Pack points out in his latest Substack on polling, the actual figures are: yes 45 per cent, no 20 per cent, don’t know 35 per cent. To reach 70 per cent you need to exclude the “don’t knows”. Indeed, the first use of “70 per cent”, in the second paragraph of the story, is not even technically true: it omits the qualification about “those who take an opinion”.

The third alarm bell should make the loudest sound of all. It concerns what is left out from the reference to a “significant risk to vulnerable lives”. It does not refer to Leadbeater’s bill. But then why should it? The headline and the story’s first three paragraphs all refer to the bill. It would be tediously repetitive to do so yet again in paragraph five.

However, sympathy for the reader is not the reason why paragraph five, which gives details of the poll, swerves round the bill. It doesn’t refer to assisted dying because, despite the bold headline and stark intro, the poll itself makes no reference to it either.

Here is the question in full: “In your view, should members of the House of Lords be free to choose whether or not to vote against proposed laws that they believe to be harmful to vulnerable people, if those laws were put forward by an individual Member of Parliament rather than the Government”.

Of course, peers should “be free to choose” whether or not to support such laws. Freedom of choice is a cornerstone of our liberties. I should be free to swim in the North Sea in January wearing a dinner jacket. Whether that’s wise is a different matter. Had I been asked Whitestone’s question, I would have been one of the 45 or, if you prefer, 70 per cent. I want peers to pass the assisted dying bill AND keep their right to vote as they please, subject only to the general principle (not tested in this survey) that ultimately the will of elected MPs trumps that of unelected peers.

What we have, then, is a view about freedom of choice being interpreted as a warning to peers to vote down a bill that the question does not mention. To gauge public attitudes to the possible impact of the assisted dying bill, the proper course for pollsters is (a) to refer to the bill directly, and (b) to test propositions on both sides of the argument (for example on the promised safeguards). Whitestone’s question does neither of those things.

We should not be surprised that the story goes on to say: “The survey appears to go against previous polling which suggested a majority of the public are in favour of assisted dying.” Of course the polls differ. If there is one thing sillier than comparing apples with pears, it is pretending that both are apples, and then discovering that, my goodness, they are different.

What a pity that there was no discussion of what the other polls showed, or why they are different. Peers – and anyone else – who wish to know more can find a useful summary of the polls on the Parliament website. It says that support for Leadbeater’s bill has remained steady at around 75 per cent throughout the controversy. That is what we find when respondents are asked a direct, balanced question, rather than a rather odd question that makes no mention of the subject in hand.

This brings us to the core of the issue: conducting polls that campaigns want for publicity purposes. When I helped to run YouGov, I often dealt with clients that wanted to ask loaded questions. Here’s a notional example. “Given that the police stop and search innocent people for no good reason and can’t be bothered to go after shoplifters, how much confidence do you have in the police these days?” I would tell the (again notional) client that this is a loaded question and advise them to leave out the anti-police preamble. Among other reasons I would tell them that the results from biased questions would be worthless and not help their cause.

Often they would agree; sometimes they would not, and I would turn down the work. I told my team that YouGov’s reputation was worth far more than the few thousand pounds we would earn by doing flawed research. Occasionally I would see a news story reporting the results of a poll I had turned down. I was glad that YouGov’s name was not attached to it.

This is a matter as much for journalists as for pollsters. Campaigns commission polls in order to get publicity for their causes. They rely on the reputations of their pollster and media outlet to impress their target audience. A poll story in a trusted newspaper or news programme carries more weight than a self-serving advertisement. Hence the decision of opponents of the assisted dying bill to send peers the copy of the Independent’s report.

So how should journalists treat such polls? Here is a checklist. First, name the client as well as the pollster, so that readers/listeners/viewers know the full provenance of the information. Second, make sure that the poll really is about the issue set out in the headline and the first paragraph. Third, on polls about great controversies, report the don’t knows. Fourth, check other polls on the same subject. If there seems to be a discrepancy, find out why and report the explanation.

The Independent story did none of these things.

My concern with this research, then, is not just, as an ex-pollster with Whitestone’s rather odd question, but as an ex-journalist with the Independent’s report. Indeed, for the first five years of the Independent’s life, part of my job was to commission, analyse and report opinion polls.

The paper was launched with a brilliant poster campaign. Underneath an image of the Independent’s masthead was the simple question: “It is. Are you?” Those were heady, liberating days. We had no link or loyalty to any political or commercial interest. We sought to give the fullest, most objective account that was possible for fallible journalists constrained by space and deadlines. In those early, pioneering years the paper would surely have rejected the article it published last month.

The Independent. It was. Were you?