Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

Comments Page 13 of 17
1 … 12 13 14 … 17

  1. EGW @ #599 Friday, April 26th, 2019 – 7:24 pm

    Rex Douglas @ #574 Friday, April 26th, 2019 – 7:12 pm

    EGW @ #565 Friday, April 26th, 2019 – 7:06 pm

    Rex Douglas @ #547 Friday, April 26th, 2019 – 6:51 pm

    https://www.afr.com/news/economy/trade/rudd-warns-of-world-of-pain-for-global-economy-20190426-p51hdy

    Rudd still thinks he’s relevant when the rest of the world probably sees him as a joke.

    He was interviewed by Bloomberg TV in his capacity as the president of the Asia Society Policy Institute so they must think he has something relevant to say.

    Strange seeking wisdom from someone who was sacked by his colleagues for his incompetence.

    Oh you mean those clowns who later realised their mistake and reinstated him?
    My last response to your trolling Rex.

    ‘those clowns’ actually installed him twice 😆

  2. Grog was right. Give the media a chance to pick its own belly fluff, and pick they will.

    Today Bill Shorten apparently made several serious, important policy announcements.

    However, these were trumped by an all-day, front page shitstorm about who’d debate whom, where and when, with a bonus story about how a nobody journo and photographer got hit on by a youthful idiot supporting a no-hoper Senate candidate. Reason? Both stories were all about the media… aka the media’s favourite subject.

    And ABC TV News dutifully followed ONLY those two stories.

  3. Confessions, James Massola is the Fairfax correspondent in Jakarta. Following in the fine footsteps of Michael Bachelard and Jewel Topsfield.

  4. Speaking as an occasional poster I couldn’t give a shit if EGW is actually Bemused.

    Provided EGW/Bemused treats others, and their opinions, with respect, he should be welcomed, regardless of any past history.

    Indicentally, the very fact that EGW has (as far as I can see) been doing exactly that, is reason enough to accept – if he really is Bemused – that he has turned over a new leaf.

    More importantly, let’s remember that Bemused and EGW are both on the right side of the political divide: with the good guys, not the crazies

  6. I remember that during Abbott’s election campaign, Joe was the first to call out abbott’s ‘truck driving’ stunt where Abbott pretended to drive a big truck. Joe correctly explained all the licences required and how long it takes to get a semi-trailer truck licence. Abbott did not and could not have got in the time, a licence to put that truck on the road, and if he did, he would be arrested. So Abbott the Truckie was a complete sham who never drove that vehicle out of the company’s car-park. Well done, Joe6pack,

  9. Puffy:

    I’m confusing J6P with someone else then. This other commenter wasn’t a truckie, but an ex army person like Rummel.

    Who knows. So many years ago now.

  10. EGW

    Homicides in Australia are at a rate of some 220 pa. Mentally ill perpetrators, around 7%, so about 15 total. With such small numbers, the occasional multiple in one incident will skew numbers a lot. Family members are much more likely victims than strangers. So I was wrong, but only by 50%! You are about as likely to be killed by lightning as by a schizophrenic, and about 15 times more likely to drown.

    Oh, the worry! I shouldn’t get out of bed. But over 90% of Australians die in bed. AAArggghhhh!
    Did you know that over 95% of Australians who died of heart attacks had watched television in the previous week?

  11. Also Grog – Greg Jericho – continues his work with the Guardian. I doubt that he’s ever bought James Massola a beer…

  12. Provided EGW/Bemused treats others, and their opinions, with respect, he should be welcomed, regardless of any past history.

    Well, not easy to have future history.
    #redundancyisshit

  13. Speaking as an occasional poster I couldn’t give a shit if EGW is actually Bemused.

    Provided EGW/Bemused treats others, and their opinions, with respect, he should be welcomed, regardless of any past history.

    There was a time when sockpuppetry was frowned on and shunned. Especially when a commenter had been banned from commenting because of consistently poor behaviour.

  15. More proof that Liberal party of Victoria is captured by ultra-conservatives. This article is by ‘The Age’. My query is why is SMH not investigating about Lib candidates of NSW?

    https://amp-smh-com-au.cdn.ampproject.org/v/s/amp.smh.com.au/politics/federal/liberal-candidate-linked-same-sex-marriage-to-paedophilia-20190426-p51hij.html?amp_js_v=a2&amp_gsa=1#referrer=https%3A%2F%2Fwww.google.com&amp_tf=From%20%251%24s&ampshare=https%3A%2F%2Fwww.smh.com.au%2Fpolitics%2Ffederal%2Fliberal-candidate-linked-same-sex-marriage-to-paedophilia-20190426-p51hij.html

  17. Chinda63 @ #604 Friday, April 26th, 2019 – 7:30 pm

    Speaking as an occasional poster I couldn’t give a shit if EGW is actually Bemused.

    Provided EGW/Bemused treats others, and their opinions, with respect, he should be welcomed, regardless of any past history.

    Indicentally, the very fact that EGW has (as far as I can see) been doing exactly that, is reason enough to accept – if he really is Bemused – that he has turned over a new leaf.

    More importantly, let’s remember that Bemused and EGW are both on the right side of the political divide: with the good guys, not the crazies

    An occasional but very good poster. Quality beats quantity any day.

    Some would do well to reflect on this:

    “Great minds discuss ideas; Average minds discuss events; Small minds discuss people.”
    Eleanor Roosevelt – US diplomat & reformer (1884 – 1962)

  21. C@tmomma
    says:
    Friday, April 26, 2019 at 7:43 pm
    And even tinier minds indulge themselves in discussions about themselves.
    _____________________________________
    Very few on here have discussed themselves as much as you. 🙂

  23. Wayne, if I thought you were old enough to make a legally enforceable contract I’d put a grand on Labor in Longman with you.

  24. yabba @ #610 Friday, April 26th, 2019 – 7:36 pm

    EGW

    Homicides in Australia are at a rate of some 220 pa. Mentally ill perpetrators, around 7%, so about 15 total. With such small numbers, the occasional multiple in one incident will skew numbers a lot. Family members are much more likely victims than strangers. So I was wrong, but only by 50%! You are about as likely to be killed by lightning as by a schizophrenic, and about 15 times more likely to drown.

    Oh, the worry! I shouldn’t get out of bed. But over 90% of Australians die in bed. AAArggghhhh!
    Did you know that over 95% of Australians who died of heart attacks had watched television in the previous week?

    All you say is probably true and I take your word for it.
    A couple of observations:
    1. Society can do little about the risk of being struck by lightning, but it should treat the seriously mentally ill to protect their lives and the lives of others.
    2. Mentally ill people represent a greater risk to themselves than anyone else and the death rate from suicide now exceeds the road toll.

    Cheers

  27. yabba:

    [‘Did you know that over 95% of Australians who died of heart attacks had watched television in the previous week?’]

    That’s a disturbing stat – demonstrably safer blogging?

  30. Outsider:

    Thanks for that. Obviously I don’t consume Fairfax media when it comes to Indonesia. 🙂

    Meanwhile Grog jags a gig where he gets paid to offer his expertise and commentary publicly.

    Grog/Greg Jericho: A gazillion
    James Massola: who?

  31. Actual Sportsbet market for Longman:

    Longman (QLD)
    All in betting, Others may be added.
    Labor $1.11

    Coalition $5.50

    One Nation $31

    Australian Progressives $41

    Greens $61

    United Australia Party $71

    Australian People’s Party $81

    Fraser Anning’s Conservative National Party $101

  34. Confessions @ #630 Friday, April 26th, 2019 – 7:51 pm

    Mavis Davis @ #626 Friday, April 26th, 2019 – 5:49 pm

    yabba:

    [‘Did you know that over 95% of Australians who died of heart attacks had watched television in the previous week?’]

    That’s a disturbing stat – demonstrably safer blogging?

    Or stick to youtube 😀

    I’d hazard a guess the stats are roughly similar pursuing the sedentary pursuit of watching You Tube. A brisk walk might be more productive wrt reducing your risk of heart attack. 🙂

  35. BB,

    I understand your frustration but the policy announcement today, Friday, will be spread over social media heavily by labor. Of course it would have been good to get more coverage on the TV news but I thought, for once, Mark Riley gave a good coverage on it for Shorten.

    Now, from a political optics angle both nine and Riley did give a not so flattering coverage of the Palmer deal with the coalition. Morrison having his mug anywhere near Palmer will be a big negative for him.

    Sadly, the MSM are only interested in the fluff. However that should not be a surprise. Play the cards you are dealt.

    So, I think, overall labor would be happy with the coverage today.

    Shorten not the story. Morrison in the frame and not in a positive way with Palmer. Even the debate story does not come across well, optics wise ,for Morrison. He is the PM but today his whinging and complaining about a bloody debate leaves him looking more like a opposition leader than PM.

    Labor would be well aware that ,policy wise, it is up to them to pass on the detail and get the message out. That would be a given in their campaign strategy.

    Anyway, just the opinion of a enthusiastic amateur.

    Cheers to you.

  38. Ven, given Stephen Conroy’s assertion to the effect that anyone opposed to his bloody stupid internet filter must be a paedophile, for which I’m unlikely to forgive him anytime soon, I don’t think RWNJ’s have the monopoly on crapulent puritanism.

  40. There’s a guy I like reading at The Pub, calls himself Aguirre & he points out that the media is using their long obsession with the PPM figures as their main focus in this campaign. The idea being to keep the whole thing to a 2 person contest, lambast one & give the other a free pass.
    I hope I haven’t misrepresented him too much, don’t think so.

  41. HughB
    says:
    Friday, April 26, 2019 at 7:58 pm
    Ven, given Stephen Conroy’s assertion to the effect that anyone opposed to his bloody stupid internet filter must be a paedophile, for which I’m unlikely to forgive him anytime soon, I don’t think RWNJ’s have the monopoly on crapulent puritanism.
    ________________________________
    Ah the internet filter. 3 years spent annoying the shit out of us for no political or practical gain. Conroy was a disaster.

  42. C@t:

    Yeah it was tongue-in-cheek. Obviously it’s sedentary lifestyles that increase the risk of heart disease.

    I saw on the Washington Post website yesterday or day before that the WHO has released new guidelines for children recommending that zero to very limited screen time for 1 to 5 year olds. I blinked at the thought that there are families out there putting 1yos in front of TVs.

  43. Confessions says:
    Friday, April 26, 2019 at 7:37 pm
    Speaking as an occasional poster I couldn’t give a shit if EGW is actually Bemused.

    Provided EGW/Bemused treats others, and their opinions, with respect, he should be welcomed, regardless of any past history.

    There was a time when sockpuppetry was frowned on and shunned. Especially when a commenter had been banned from commenting because of consistently poor behaviour.

    Fess

    My recollection is that Bemused was banned because he abused William. If he has been allowed back, presumably William has decided that a lengthy stay in coventry followed by a suitable apology is sufficient to redress the insult.

  45. HughB @ #638 Friday, April 26th, 2019 – 7:58 pm

    Ven, given Stephen Conroy’s assertion to the effect that anyone opposed to his bloody stupid internet filter must be a paedophile, for which I’m unlikely to forgive him anytime soon, I don’t think RWNJ’s have the monopoly on crapulent puritanism.

    Conroy was not the most cerebral of ALP politicians. I don’t think he was seriously into policy.

  47. Darn @ #643 Friday, April 26th, 2019 – 8:01 pm

    Confessions says:
    Friday, April 26, 2019 at 7:37 pm
    Speaking as an occasional poster I couldn’t give a shit if EGW is actually Bemused.

    Provided EGW/Bemused treats others, and their opinions, with respect, he should be welcomed, regardless of any past history.

    There was a time when sockpuppetry was frowned on and shunned. Especially when a commenter had been banned from commenting because of consistently poor behaviour.

    Fess

    My recollection is that Bemused was banned because he abused William. If he has been allowed back, presumably William has decided that a lengthy stay in coventry followed by a suitable apology is sufficient to redress the insult.

    Your recollection is not correct.

  49. As a parent whose 1 yo used to wake up at about 5.30am, I was rather relieved that I could stick her in front of the (reasonably good quality) kids programs on the ABC at that time of morning and go back to bed for another hour or so. Sometimes sanity, or having a life other than 24 hr child carer, demands that some low effort entertainment be utilised. I don’t claim it as ideal practice, but, realistically, it’s a very handy tool in the box.

  50. BSA Bob:

    I’ve had a debate with a journo on Twitter about this.

    I invited this person to look at PPM polling vs actual election results since 2007, and to agree it’s a shit metric upon which to predict an election outcome (I said it politer than that).

    The upshot was I got blocked.

    #AusVotes2019 in a nutshell.

Comments Page 13 of 17
1 … 12 13 14 … 17

Leave a Reply

Your email address will not be published. Required fields are marked *