Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

  1. What is all this bullshit about bemused ‘targeted women’? The only time bemused made a comment critical of women was when they made a stupid statement. The critical comment had nothing to do with gender and everything to do with stupidity. Build a bridge snow flakes.

  2. The Tripartite Pact of Palmer, Morrison and Hanson will deliver government, Kooyong, Goldstein and Higgins to Lord Shorten of Maribyrnong.
    The South will triumph over the northern slack jaws and their Anning cousins.
    All Hail Victoria!

  3. I’m of the view having seen what’s happened since he burst into the election campaign, that Clive is going to have many candidate issues. Already he has one candidate threatening to walk away from the UAP and the election hasn’t even been held, much less that person being elected himself.

    One thing Clive’s emergence has done is drown out PHON. Which I suppose can be counted as a good thing.

  4. clem attlee @ #550 Friday, April 26th, 2019 – 6:55 pm

    What is all this bullshit about bemused ‘targeted women’? The only time bemused made a comment critical of women was when they made a stupid statement. The critical comment had nothing to do with gender and everything to do with stupidity. Build a bridge snow flakes.

    PB is replete with stupid comments.

  5. Matt31 says:
    Friday, April 26, 2019 at 6:11 pm
    Does anyone here think for one second that if things were looking up for the Coalition that Morrison and the Coalition’s media backers would’ve been running around demanding more debates? Since when do incumbent Prime Ministers want multiple debates with an opposition leader? This all tells me they’re even more toast than I thought.

    _____________________________________________

    Says to me that they will have Scrott primed up to do full CAPSLOCK talking over the top of anything that moves and maybe with a bit of crap someone has dug up to discombobulate Bill and give them ‘an edge’ for a day or two.
    Bill’s townhall background will be a natural advantage, so knowing that, the LNP can only be going for a “powershot” to crow about.
    Not that there is much in it for the punter, but news organisations will love it.

  6. beth @ #659 Friday, April 26th, 2019 – 4:55 pm

    The Tripartite Pact of Palmer, Morrison and Hanson will deliver government, Kooyong, Goldstein and Higgins to Lord Shorten of Maribyrnong.
    The South will triumph over the northern slack jaws and their Anning cousins.
    All Hail Victoria!

    It’s like America. They’ve got City-Country divide, we’ve got North-South divide.

  7. Our great LNP will win the May 18 election and these are the seats that they will win to retain government

    Dunkley
    Longman
    Cowan
    Corangamite
    Herbert

    That’s why I say that our great LNP is leading 51.5 to 48.5

    Well done our great LNP running to the finish line

  8. Is it just me, or having been through quite a lot of Federal elections campaigns, this one seem very odd? Odd, in the sense that nobody, as yet, seems to be paying any attention at all. Compared with some campaigns of yore, when one could get into a punch-up, over this or that, today…………….not much at all. Not that TV news is much watched these day, but the election rates a brief mention about 15 minutes in…………On-line news seems much the same. Is this the sign of the disengaged electorate? For the first time in my voting existence I am postal voting………..I am over the ritual of the sausage sizzle and sponge on Saturday………….

  9. Zoomster

    I remember you saying every school had a school nurse, and Bemused contradicted that. So did I. What you wrote was wrong and misleading.

    Therefore I am Bemused too.

  10. Tricot @ #558 Friday, April 26th, 2019 – 7:03 pm

    Is it just me, or having been through quite a lot of Federal elections campaigns, this one seem very odd? Odd, in the sense that nobody, as yet, seems to be paying any attention at all. Compared with some campaigns of yore, when one could get into a punch-up, over this or that, today…………….not much at all. Not that TV news is much watched these day, but the election rates a brief mention about 15 minutes in…………On-line news seems much the same. Is this the sign of the disengaged electorate? For the first time in my voting existence I am postal voting………..I am over the ritual of the sausage sizzle and sponge on Saturday………….

    I will be very interested to see how people behave at the pre-polls when I start working there.

  11. Gee Wayne, Cowan is a big call. Sure it is marginal but what on earth have the Liberal got to offer the good voters of Cowan? The WA Libs are broke, the local State members fit in a minivan, they are on the back foot and some of their ‘stars’ are struggling to survive at all. What a shame the Liberals insulted and peed off their biggest ‘star’ in Julie Bishop……..How many votes did she get in the leadership from her WA colleagues – or anyone else for that matter? What a great, great party the Liberals are when it comes to women – or anybody else, for that matter. Why anybody with an IQ of over 100 would vote Liberal has always been a puzzle to me.

  12. EGW @ #565 Friday, April 26th, 2019 – 7:06 pm

    Rex Douglas @ #547 Friday, April 26th, 2019 – 6:51 pm

    https://www.afr.com/news/economy/trade/rudd-warns-of-world-of-pain-for-global-economy-20190426-p51hdy

    Rudd still thinks he’s relevant when the rest of the world probably sees him as a joke.

    He was interviewed by Bloomberg TV in his capacity as the president of the Asia Society Policy Institute so they must think he has something relevant to say.

    Strange seeking wisdom from someone who was sacked by his colleagues for his incompetence.

  13. There was also a comment about Rudd in glowing terms.

    Unfortunately obsessives simply cannot hide the object of their obsession, no matter how hard that they try.

  14. Strange seeking wisdom from someone who was sacked by his colleagues for his incompetence.
    _________________
    And then reinstated by many of the same people.

  15. For anyone interested

    Anthony Klan
    @Anthony_Klan
    ·
    4h
    I’ll be on Sky News at 7.40pm tonight to discuss all things water. Tune in!
    #buybacks #taxpayers #transparency #auspol2019 #watergate

  16. Hola from near the Dirrandbandi Show!
    Many Bludgers would be pleased to know that this year ‘s cotton crop is the lowest on record. 45000 bales. Biggest crop is 4500000 bales. Environment up here is not quite totally fucked yet…. but even the goats look crook. The drought is a cracker.

  17. I also wish Possum would visit us with his wisdom more often! When I first started following this blog around 2005, we also had Possum’s blog to follow. I recall he shut it down when he realised he had to go off and get a job!! His statistical analytical work was just fantastic.

  18. Boerwar @ #587 Friday, April 26th, 2019 – 7:19 pm

    Hola from near the Dirrandbandi Show!
    Many Bludgers would be pleased to know that this year ‘s cotton crop is the lowest on record. 45000 bales. Biggest crop is 4500000 bales. Environment up here is not quite totally fucked yet…. but even the goats look crook. The drought is a cracker.

    Stay away from the goats, Boerwar…

  19. Well I’m bemused when I consider that anyone would consider voting for that spiv and con-artist Clive Palmer. Clive believes in Clive, nothing else.

  20. Outsider:

    And Grog used to comment here too, before he was vilely identified personally by that News Corp hack James Massola. Whatever happened to him by the way? Haven’t heard from Massola in forever.

  21. “The weevils are winning, the Weevils are winning!”

    I don’t know what that means but I am sure that it will all become clear in due course…

  22. Rex Douglas @ #574 Friday, April 26th, 2019 – 7:12 pm

    EGW @ #565 Friday, April 26th, 2019 – 7:06 pm

    Rex Douglas @ #547 Friday, April 26th, 2019 – 6:51 pm

    https://www.afr.com/news/economy/trade/rudd-warns-of-world-of-pain-for-global-economy-20190426-p51hdy

    Rudd still thinks he’s relevant when the rest of the world probably sees him as a joke.

    He was interviewed by Bloomberg TV in his capacity as the president of the Asia Society Policy Institute so they must think he has something relevant to say.

    Strange seeking wisdom from someone who was sacked by his colleagues for his incompetence.

    Oh you mean those clowns who later realised their mistake and reinstated him?
    My last response to your trolling Rex.

Leave a Reply

Your email address will not be published. Required fields are marked *