back to article Lies, damn lies and election polls: Why GE2015 pundits fluffed the numbers so badly

Whatever you may think about the outcome of last Thursday’s General Election, there is one issue on which public, politicians and pundits alike seem to be broadly united: how badly the opinion pollsters fared. They got it very wrong! Egregiously so, according to the editor of the Market Oracle, an online financial forecasting …

  1. Pete 2 Silver badge

    Measure what you value ...

    ... don't value what you measure.

    > This put Tories and Labour within a whisker of one another around the 34 per cent mark.

    But that's an irrelevant statistic. Who wins a First Past the Post election (as we have in the UK, designed primarily to produce clear, if somewhat less than truly representative, outcomes) is the party with the most seats, not the most votes.

    The system is supposed to be highly sensitive (there's only 1 winner, whether the majority is 1 or 10,000) to small differences in numbers of votes between the most voted for and the next candidate.

    Everyone: the pollsters, the media who commissioned them and the general public all know this. So to huff and puff and say "well, we were almost right with a measurement of something that's useless" is a ridiculous defence.

  2. jonathanb Silver badge

    Re: Measure what you value ...

    It may be an irrelevant statistic, but it is what the sample measures, then they try to extrapolate from that to number of seats. This works if there is a uniform national swing, but in this election, there wasn't a uniform national swing, or even a uniform regional swing, there were 650 different elections each with swings going all over the place. In one seat there might be a swing from Labour to SNP, and also swings from the other unionist parties to Labour. In another seat there might be a swing from Lib Dem to Labour and another swing from Tory to Lib Dem. Elsewhere you might have swings involving UKIP and the pro-EU parties.

  3. choleric

    Re: Measure what you value ...

    "Who wins a First Past the Post election (as we have in the UK, designed primarily to produce clear, if somewhat less than truly representative, outcomes) is the party with the most seats, not the most votes."

    The UK's FPTP system was not so much "designed" as evolved and developed over a long period of time (centuries).

    The clear outcomes you refer to are a byproduct of the system rather than the goal of it.

  4. jonathanb Silver badge

    Re: Measure what you value ...

    Constituencies always elected members to send to parliament. That has never changed. What changed was the gradual introduction of the party system. We had one group of MPs that supported William of Orange in the English Civil War. The other side called them "Whigs". Another group supported King James, and the other side called them "Tories". The Whigs eventually became the party now known as the Liberal Democrats, and the Tories eventually became the party now known as the Conservatives, but are still frequently referred to as Tories. The Labour Party arrived on the scene much later on with the extension of voting rights to the working class.

  5. Tom 13

    Re: there is a uniform national swing

    The outcome of the election DOES show a uniform national swing: away from Labour and toward Tory. There's no way around that fact.

    If there were a bunch of swings as you describe, they would have been offsetting with no noticeable change in the national election. At the very least you would have had a situation where the nominal majority needed to form a coalition with another party. That didn't happen.

  6. disgruntled yank Silver badge

    "shy tory"

    In the US we speak of "voting-booth candidates". Years ago, Jesse Helms was a couple of times polled as losing his senate seat from North Carolina, but then won handily. Then Marion Barry, who had been videotaped consuming crack cocaine, was elected mayor of Washington, DC, despite polls showing him doing badly. In both cases the voters polled presumably answered according to the tone of the question rather than according to their true intentions.

  7. Anonymous Coward
    Anonymous Coward

    Re: "shy tory"

    I don't know in the US but I think here people are less and less likely to respond to pollsters. I think people used to feel special because someone asked what they thought but now-a-days, probably thanks to the ad men, people are just fed up with being pestered.

  8. auburnman

    Re: "shy tory"

    City life steels you against interacting with strangers on the street. After a certain amount of practice in dodging nutters, drunks, chuggers, chavs and possible muggers, pollsters become just another obstacle to dodge.

  9. Marshalltown

    Re: "shy tory"

    In the US there was at one time a very strong tendency to tell pollsters to get lost. My dad used to remark when asked for a political view that "there's a reason for secret ballots." His rule was that one should never discuss politics or income in polite company.

  10. Anonymous Coward
    Anonymous Coward

    Re: "shy tory"

    The media were pretty anti-Tory this time around. Look at the SNP "GET THE TORIES OUT" campaign and various Socialist Worker placards. You'd be risking a kicking to admit to be voting Tory - I don't think that Labour voters faced that level of intimidation, well since Blair/Brown left at least.

  11. RegGuy1

    Re: "shy tory"

    Well, take this morning. 10:00. The phone rings. The same as the bloody thing does every morning around this time. I occasionally answer it -- i did this morning. 'Sir, we are undertaking a survey...'

    At this point I politely inform them I'm not interested and hang up.

    In the past I have gone along with these, sometimes offering offering incorrect answers ('did you shop in JJB sports...' -- why yes I did, cleverly not telling them I'm really a fat bastard and wouldn't be seen dead within 500m of such a store).

    Strangely over then next few days I then received additional phone calls trying to sell me sports gear. Co-incidence?

    So no, I don't do surveys, because in this post Thatcher world, every twat wants to sell me something.

  12. John H Woods Silver badge

    Re: "shy tory"

    "The media were pretty anti-Tory this time around" --- AC.

    They most certainly were not. Most of the mainstream press came right out and said who they were supporting: e.g. read

  13. Ray Gratis

    Re: "shy tory"

    "The phone rings"

    Well, get a call-blocker like the rest of us!

    Tight b*stard.

  14. Long John Brass Silver badge

    Re: "shy tory"

    But wait! There's more!

    For $9.99 you can get your very own call blocker

    Call now on 0800 fck you for details

  15. John Robson Silver badge

    Re: "shy tory"

    Tight means not plugging the phone in....

  16. mir

    Re: "shy tory"

    Interesting bit of analysis over at (the admittedly right-wing) Order-Order, suggesting that the Tory vote share was pretty much what was expected, but the labour vote share was low - not so much "shy tory" as "lazy labour"*, according to them.

    *your offence at this term may vary depending on your particular favourite flavour of blood sucker.

  17. Primus Secundus Tertius Silver badge

    Re: "shy tory"

    @John H Woods

    The newspapers are allowed to be biased, and they are generally honest about their bias. The BBC is supposed to be not biased, and therefore hides its bias.

    BBC election night began at 9.55pm, and for a few minutes they were relishing the prospect of a Labour coalition. Then they were stunned by the exit poll, and more or less behaved themselves.

    However, in 1992 they did not admit until the following morning that the Conservatives had actually achieved a majority of parliamentary seats. Same again this time.

  18. Why not OTP?

    Re: "shy tory"

    Don't know about 1992, but in 2015 they had *not* in fact won a majority until the morning results were in. Nevertheless as the exit polls were revised the BBC coverage made it clear that a majority was likely.

  19. Alistair Silver badge

    Re: "shy tory"

    "....depending on your particular favourite flavour of blood sucker."

    At some point in our ragged political past over 'ere north of the 52nd, we had a municipal candidate with the unenviable name of Leech. First initial was B.

    Was never quite sure if the loss was due to any of the combinations you can mangle that name into.

  20. Wiltshire

    Re: "shy tory"

    You're correct, Order Order did mention that, but they were reporting Ipsos MORI, who have a different theory: that the numbers were skewed by "Lazy Labour" supporters not bothering to vote.

    The final MORI poll suggested 12.5 million people would vote Tory, just ahead of 12.2 million voting Labour.

    On the day, 11.3 million voted Tory – but some 3 million Labour supporters went missing.

    Where did they go?

    MORI chief Ben Page says: “we think not voted. Predicted turnout 82% day before. Was only 66%”

    Their analysis suggests it was young left-wing voters not turning out on the day that explains the difference.

    As most of the twitterati that follow Russell Brand are young (and, it seems, gullable), it seems they followed Russell Brand's earlier (and much tweeted) advice to not vote. Or perhaps they did not bother to register to vote, and by the time Brand had changed his mind, it would have been too late for these youngsters to change their mind as well, and register. They would have disenfranchised themselves.

    Which might be a useful lesson for them in believing what a media clown had said.

  21. Dazed and Confused Silver badge

    Re: "pollsters become just another obstacle to dodge."

    It's a good idea to give the drunks a wide berth but you can happily take the piss out of pollsters and the million and one marketing zeebs that life throws in your path.

  22. captain veg

    Re: "shy tory"

    I don't much like this term. Shyness tends to stop people doing things rather than keeping their activities secret.

    So I suggest Conserfurtive.


  23. Pav

    Re: "shy tory"

    You're pretty much always risking a kicking admitting to being a Tory in Scotland

  24. Esme

    Re: "shy tory"

    I dunno about that. I thought Labour had lost the fight before the election, the minute I heard Milliband rule out any possibility of an alliance with the SNP. I didn;t expect the SNP would do as well as they did, but the referendum on Scottish devolution showed that the SNP was clearly making ground. So, with Labour having rejected any alliance with an obvious natural ally, and the polls all predicting no overall majority, that hands the election lock, stock and barrel to the Tories. Or so it seemed to me. I wouldn't be surprised if a lot of those 'shy' Tories mightn't have been undecideds that simply decided here was no longer any point voting Labour after Milliband shot himself in the vote re the SNP.

    For myself - a pox on party politics. And the jerrymandering that's taken place over the last 40 years. Next election I'm voting for the individual I feel is nearest my ideal of (a) being an honourable person and (b) that I think will do best for the area that I live in. And if there aren't any I think sufficiently trustable, then for the first time in my adult life, I simply won't vote.

  25. Chris Miller

    3% margin of error

    Is correct for each individual poll, but if you take a 'poll of polls' average of 9 (allegedly) independent polls, you should be able to get a result with 95% confidence of being within 1%. The fact that the poll of polls was still badly wrong strongly suggests that some kind of systematic error is taking place. Indeed, one pollster (Survation) have already apologised, because their poll on the day before the election gave almost the correct result, but they suppressed it because they thought it must be wrong, as it didn't agree with everyone else!

    It's not the case that 'First Past The Post' renders opinion polls irrelevant, they just require to be applied with care (under the assumption that any swing will be roughly consistent at a national level).

  26. Jane Fae

    Re: 3% margin of error

    Think you'll find the poll of polls was giving far better than 3% margin, which is why the percentage issue was/is systemic...

  27. Pete 2 Silver badge

    Re: 3% margin of error

    Although the pollsters claim a 3% margin, that is clearly shown to be false. You just have to look at the results of the various polls, where the spread from one poll to another was much greater than 3%. When Sky News was broadcasting before the election they had a rolling graphic on the screen that summarised the most recent polls. Looking at the numbers as they went by, one could see that few of the polls were within 3% of each other.

    It would be easy to dismiss the spread by saying that the polls were taken on different days. However, if there really was so much variation: one day to the next, then the pollsters were measuring a quickly changing variable to too high a degree of accuracy. The poll might have been accurate to the stated degree, but its shelf-life at that accuracy was so short that it was probably obsolete before it was even published.

  28. Tony W

    Re: 3% margin of error

    The Survation apology is very revealing - it shows that polling is far from being the scientific process that they like to pretend. Anyone who trusts Survation after this is clearly not interested in the truth, unless they can regain some credibility by promising to publish whatever they find without fear or favour.

  29. RegGuy1

    Re: 3% margin of error

    Err, not so fast ther Mr Miller. How do Survation know they got the correct result? It could have been an outlier. The fact that it mached the real outcome is not the same. Opinion could have changed since the poll was taken and the actual vote was taken.

    So it could just be luck that the two match -- the Survation poll may have been measuring a different distribution than what happened on the day, because all those don't knows may have made their minds up in a different way to how Survation thought. We have no idea, and never will know how near the Survation poll was to the real distribution of voting intention *at the time the poll was done*. That's NOT the same as the distribution of votes a day or so later in the voting booth. The two are NOT the same.

    Polls are only snapshots.

  30. Cari

    Re: 3% margin of error

    It could be an outlier, or maybe they took into consideration the same factors the bookies did when working out the odds.

    From what I could see leading up to the election, the only places that were so wrong about predictions, were the left-wing media, its readers, and the polls.

  31. Tom 13

    Re: The poll might have been accurate to the stated degree

    Not a Brit so I can't confirm your observation about the poll differentials, but assuming what you said is true, that means they should have been aware of a systemic problem. The truth is, most people's opinions won't vary that wildly that quickly. While there are multiple possible reasons, the bottom line is that you should already know you have at least one systemic problem.

  32. Tom 13

    Re: Err, not so fast ther Mr Miller.

    You can dance around it all you want, but an outlier that more closely matches the actual results is not an outlier. It may have been an outlier from their targeted distribution amongst the voters, but that only corrects a systemic problem.

    The claim that people haven't made up their minds/are changing their minds quickly is blowing smoke up your arse. They don't. It may be polite to accept that fiction in most social situations, but polling isn't one of them.

  33. Neil Barnes Silver badge

    We don't vote for parties...

    which renders the whole thing of opinion polls somewhat pointless.

    We vote for a local representative, whom we feel best suited to represent our local issues. It is only because these representatives claim a party affiliation that the party can claim any kind of majority.

    So even if we are a rabid party member, and vote for the party of our convictions come what may, we're *still* only voting for a local member. To be honest, I'd like to lose the party thing completely; it skews any semblance of sanity from the system and requires our carefully selected candidate to follow the party line, come what may. Let's see more loose coalitions!

  34. Charles 9 Silver badge

    Re: We don't vote for parties...

    The trouble is, as BOTH sides of the water proved hundreds of years ago, is that people NATURALLY form cliques or blocs. George Washington himself expressed it AND was right about the whole thing (he was against parties, too). BUT the behaviour is basically human nature and practically inevitable because parties represent strength in numbers: gangs for lack of a better term. George ended up being labeled a Federalist against his wishes.

  35. Grikath Silver badge

    Re: We don't vote for parties...

    or from the article: "In one sense, no. We get the government we vote for."

    Because you don't. Locally maybe, but on a national level the local-representative election system the UK uses will heavily skew the seat distribution on even minor differences in voting totals. Because the local winner takes all, and only needs a marginal victory to take that victory, the system ensures that a "majority" national government representation will always be an actual minority by votes.

    Because the difference needs only be so small locally, it's also very hard to accurately poll for or model outcome, unless you have limited (bi-partisan) options and/or a very conservative voter base that's not likely to switch, because you cannot use national totals to predict the outcome for a single constituency. Which as far as I can tell happened here more than pollsters getting Wrong Answers. They simply applied the data wrong.

    The system in place in the UK works fine for a bi- or tripartisan setup, but this election was typical for having five major players, each with their own major issues and programs. And that's where things went wrong, because anyone who's done any system analysis can see the the "dilution" of votes would mean that massive amounts of voters would basically get shafted, as their representations in votes would never stand a chance of being materialised in the house of commons.

    The worst hit, and potentially the most dangerous artefact of this, is UKIP, who did get a fair share of the vote, but lost out in the dance-of-chairs. Like them or not, it is terribly dangerous to ignore a set of sentiments that, by votes and by distribution, is a serious nationwide affair that now has no representation in government, and as such no release valve. Even if UKIP does not survive this, the sentiments will not disappear and it will bite peeps in the back the next few years.

  36. John Brown (no body) Silver badge

    Re: We don't vote for parties...

    "The worst hit, and potentially the most dangerous artefact of this, is UKIP, who did get a fair share of the vote, but lost out in the dance-of-chairs."

    Although I do understand your point and where you are coming from, is it fair to foist a second placed UKIP candidate as MP onto a constituency which voted for eg Labour or Tory?

    I'm not sure what the answer is...

  37. Primus Secundus Tertius Silver badge

    Re: We don't vote for parties...


    Yes, yes, but then you look at the alternatives, of which the most important is the European Parliament.

    There, one votes for a party list. There is no way of voting against a particular candidate unless you also vote against his or her colleagues. Voting against is a very important facility.

  38. Vincent Ballard

    Closed lists

    Being able to vote against a particular candidate may be nice, but the real downside of closed party lists is that it makes the candidates much more worried about pleasing the party hierarchy than the electorate. If each of the main parties can pretty much guarantee 3 seats out of the 10 going in a region, you want to suck up to the person who decides which names go in the first 3 slots. Open lists all the way.

  39. keithpeter

    Re: We don't vote for parties...

    "Although I do understand your point and where you are coming from, is it fair to foist a second placed UKIP candidate as MP onto a constituency which voted for eg Labour or Tory?"

    There is another way.

    Scottish assembly has roughly 75% seats awarded by FPTP in constituencies and the other 25% based on a regional list transferrable vote system. Your vote basically gets used twice. I can't get the data broken by constituency so I can map it into the 9 English government regions (each with their own list) yet, but rough estimate on that through on England results as a whole would suggest something like 14% of 150 seats allocated to the regional lists or 20 seats in total. Better than 1 but not the full proportionality.

    Do you think 20 seats would be enough of a 'pressure release valve'? Certainly give the kippers a voice.

    Irony: the party now in power is the one least likely to consider radical change in the voting system (and the Labour Party never really explored that either when they had the chance).

    Disclaimer: unshy labour voter

  40. nijam Silver badge

    Re: We don't vote for parties...

    > Because you don't. Locally maybe, but on a national level the local-representative election system

    > the UK uses will heavily skew the seat distribution on even minor differences in voting totals.

    > Because the local winner takes all, and only needs a marginal victory to take that victory, the

    > system ensures that a "majority" national government representation will always be an actual

    > minority by votes.

    That is true of PR to exactly the same extent as it is true for FPTP.

    How so? Because, at the end of it all, there is only a single government, and only a minority of voters voted for it* (as opposed to any other potential goverment). PR just gives a different kind of minority goverment and - no surprise - is supported mainly by parties who believe they'd do better than they do under FPTP.

    * Unless you live in a single-party state, then all the votes cast will be for the governmment.

  41. Tom 13

    Re: Let's see more loose coalitions!

    So you want to turn Britain into Italy or worse Greece? Sounds like a very, very bad plan to me, and as noted above, I'm not even a Brit.

  42. Titus Technophobe


    as people went to vote they remembered Tony Blair, and Gordon Brown thought ah that David Miliband ... would be much the same.

  43. Tom 38 Silver badge

    Re: Labour

    as people went to vote they remembered Tony Blair, and Gordon Brown thought ah that David Miliband

    Well done for paying attention to who was in the red corner...

  44. Titus Technophobe

    Re: Labour

    I didn't take much notice of the Blue corner either .. just thought that Ed Cameron was less like Gordon Brown and Tony Blair.

  45. Pen-y-gors Silver badge

    "Shy Tories"

    There is no such thing as a 'shy tory'.

    What there are, are a large number of "deeply, deeply ashamed Tories". They know that what they do (i.e. vote Tory) is incredibly selfish, shameful, anti-social and just downright WRONG. But they are too frightened and ashamed to admit it. A bit like strangling kittens or enjoying a loving relationship with a goat, which many of them do as well. But it's not the sort of thing one admits to the neighbours. They're all right jack.

    Time to end the secret ballot and make people admit to their disgusting and vile practices so that they can be rightfully punished.

    Oh yes, and bring in proper PR.

  46. disgustedoftunbridgewells Silver badge

    Re: "Shy Tories"

    You may find it easier to have the Tories as nasty and Labour or whoever as a paragon of virtue, but it doesn't in any way reflect reality.

    I'm a Conservative voter ( of the socially progressive, libertarian, small state variety ). I believe the state should leave us alone while not spending too much. I think bribing the electorate with more spending is reckless and crass. Low business rates, for example, is responsible because it encourages employment. Unpopular with lefties because it helps big business, but it's a practical, sensible, thing to do.

    I acknowledge that Labour generally aim to do the best they can for the country so I don't vilify them when I believe they're wrong, only for things such as trying to wage class warfare and outright lying ( take the "bye bye NHS" type comments over the last few days from their supporters ).

  47. Pen-y-gors Silver badge

    Re: "Shy Tories"

    What makes you think I count the incompetent Stalinists in the Labour party as 'paragons of virtue'?

    Bring in proper PR now - but still ban evil devil-worshipping Tories from voting - you know it's the right thing to do!

  48. Cari

    Re: "Shy Tories"

    Do you come in HD? I've been looking to replace my TV with something a bit better.

  49. auburnman

    Re: "Shy Tories"

    I don't know if you're joking or not - I sincerely hope you are - but the secret ballot is a keystone of proper Democracy. Just imagine the gaming of the system that could occur if your political affiliation was forced into the public domain.

    Fancy working in the capital? You'd better be blue through and through because the money markets love the Tories, and now they can quietly check on you.

    As for PR that was unfortunately voted down in the referendum a few years back (The death knell of Lib Dems in my 20-20 hindsight opinion.)

  50. Velv Silver badge

    Re: "Shy Tories"

    Ashamed to admit it, or actually frightened for their lives? And your post just proves my point.

    ALL parties now have an element of fundamentalist following that are not afraid to abuse anyone they think disagrees with their viewpoint. Being in Scotland I've noticed it most with SNP followers, from the purely shouty, through the vile to the utterly aggressive. Following canvassers for other parties around the street and shouting abuse when people answer their door. There's free speech and then there's intimidation. (And I repeat, it's across all parties)

    I doubt anybody who voted Tory is ashamed of their belief in that being the right vote for country. But they are frightened of the potential backlash from everyone who doesn't think it's the right thing. It's the principal reason voting is anonymous, and the reason it must remain anonymous.


POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2018