Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

  1. Oh, I see, the local Liberal troll here is called “Wayne”…
    It’s “Tim of Altona” back in The Guardian…
    It was “poet in a paperbag” at the Independent Australia website…

    Liberal trolling is becoming hard these days… and extremely ineffective.

    How many of them will be still employed after the Federal election?…. Okay guys, remember: One in the morning, one in the afternoon, keep going, unemployment will open up fantastic new opportunities in life for you….. 🙂

  2. I love your precision.

    When one combines, maths, law and information technology in the one limited brain almost everything goes wrong.

  3. Tony Abbott has a plan…

    I vaguely remember that name, is he the idiot that ate an onion and promised us a faster, cheaper broadband while delivering a much more expensive slightly slower than ASDL not quite broadband?

  4. Here is my recollection over what happened when bemused was banned:
    1. A bunch of people were having an argument and behaving badly, Cat, GG, bemused, etc.
    2. William asked everyone to behave better.
    3. The only person who didn’t stop was bemused. Part of the reason bemused didn’t stop was because he was slowly working his way through responding (with insults) to a bunch of posts (many of them also insults) in chronological order and hadn’t reached William’s comment yet.
    4. Bemused was banned temporarily, or his comments deleted or somesuch.
    5. William posted that a person that had been involved in the argument (can’t recall if he named bemused directly) had been banned permanently (or long term) specifically because that person subsequently (having taken exception to William’s actions) decided to have a go at William via email.

  5. I have, mainly through laziness, an active subscription to kayosports, yes i know far to close to the evil that is Murdoch. My bad.

    Anyway they just ran a Labor add, 100% negative i liked it a lot.

  6. Jeepers, apart from anything else, another 3 years with Morrison (or any chancer who will do away with him) and his crew would really like being in the nut bag with the nutters running the joint.
    What would this mean?……..A Liberal party tottering further to the religious right; a Liberal party tottering the to economic right with “every man and woman for themselves” as their matra; a Liberal party kowtowing even more to the ratbags that make up the Nationals and giving Clive Palmer what he wants as a capitalist fellow traveller. That is just the start………….
    It was once thought that John Howard would be PM as long as Menzies and some of us thought that winter would never end. It did, but that does not mean the Young Lib types, now making their way up are any more attractive than the reactionary lot before of them.
    Which ever way it goes, the Senate will be a hoot……………….

  7. C@tmomma @ #633 Friday, April 26th, 2019 – 7:53 pm

    Confessions @ #630 Friday, April 26th, 2019 – 7:51 pm

    Mavis Davis @ #626 Friday, April 26th, 2019 – 5:49 pm

    yabba:

    [‘Did you know that over 95% of Australians who died of heart attacks had watched television in the previous week?’]

    That’s a disturbing stat – demonstrably safer blogging?

    Or stick to youtube 😀

    I’d hazard a guess the stats are roughly similar pursuing the sedentary pursuit of watching You Tube. A brisk walk might be more productive wrt reducing your risk of heart attack. 🙂

    100% of them had farted.

    I’m pretty sure that didn’t kill them, though.

  8. Thanks DisplayName, that info will be handy to whoever is writing The History of Poll Bludger, Vol. VII, The Malcontents: Bemused and his antagonists, Black Inc, 2039.

  9. yabba @ #712 Friday, April 26th, 2019 – 8:58 pm

    C@tmomma @ #633 Friday, April 26th, 2019 – 7:53 pm

    Confessions @ #630 Friday, April 26th, 2019 – 7:51 pm

    Mavis Davis @ #626 Friday, April 26th, 2019 – 5:49 pm

    yabba:

    [‘Did you know that over 95% of Australians who died of heart attacks had watched television in the previous week?’]

    That’s a disturbing stat – demonstrably safer blogging?

    Or stick to youtube 😀

    I’d hazard a guess the stats are roughly similar pursuing the sedentary pursuit of watching You Tube. A brisk walk might be more productive wrt reducing your risk of heart attack. 🙂

    100% of them had farted.

    I’m pretty sure that didn’t kill them, though.

    You are absolutely 100% correct there, yabba. I don’t know of anyone who has succumbed to Death by Fart. 😆

  10. DisplayName @ #710 Friday, April 26th, 2019 – 8:54 pm

    Here is my recollection over what happened when bemused was banned:
    1. A bunch of people were having an argument, Cat, GG, bemused, etc.
    2. William asked everyone to behave better.
    3. The only person who didn’t stop was bemused. Part of the reason bemused didn’t stop was because he was slowly working his way through responding (with insults) to a bunch of posts (many of them also insults) in chronological order and hadn’t reached William’s comment yet.
    4. Bemused was banned temporarily, or his comments deleted or somesuch.
    5. William posted that a person that had been involved in the argument (can’t recall if he named bemused directly) had been banned permanently (or long term) specifically because that person subsequently (having taken exception to William’s actions) decided to have a go at William via email.

    Not a bad chronology, but errs in a few details.
    It all started when GG inferred that bemused was a paedophile and the master of the blog apparently thought this vile behaviour was OK.
    3. is correct.
    4. close to the mark.
    5. William had previously taken exception to having his actions queried/challenged on the blog so there was a resort to email. But the rules were arbitrarily changed so this became unpardonable.
    6. A false account of those emails was posted.

  11. C@tmomma says:
    Friday, April 26, 2019 at 8:19 pm
    Indulgent commentary about his granddaughters. Tick!

    Has chaperoning duties to the local swimming pool been mentioned yet ?

  12. Bill Shorten is Australia’s Saviour!
    Provincia de Nuestro Señor Bill Shorten, el Salvador del Australia!
    The LibNats are under the shoe trodden on dog waste.

  13. Tricot:

    A re-elected Morrison govt would give us more of the same as what we’ve already gotten. Zero policy reform, more jobs for mates, more corruption and dodgy deals, and absolutely nothing done to protect our environment or abate our GHGEs.

  14. Nath, Wayne (& EGW)
    You should all take up yoga, 9 years in the wilderness is a long time, that’s how long the Liberal & Country parties will take to regroup after May 18 rejection slips are delivered.

  15. David Marler
    ‏ @Qldaah
    1m1 minute ago

    David Marler Retweeted David Marler

    Debates are debatable. Apart from gotchas and gaffes, I don’t think we get much more out of them than we do from the regular interviews & press conferences. It turns into a contest of who can challenge the other to the most number of debates. #ausvotes #qldpol

  16. Get Up is truly vile. That bilious Ad against Tony Abbott is simply bottom-dwelling muck.

    The man has zero say on ANY policies on mitigating climate change at present and this blatant lie that the ‘coalition’ remains inactive on climate change is just so stupid. The only difference is a target as far as I can see. One is costed, one is as reliable as a Labor surplus (or ANY ALP estimate of expenditure…remember Kevin Rudd bringing to the election a 4.3 Billion NBN? I do… how many magnitudes more is it now).

    What beggars belief is that Get UP, this same vile organisation has the gall to impugn Peter Dutton for (correctly) suggesting that there are a great many residences in Dickson that can accommodate any number of people with disabilities… Something Ali France would know if she set foot in the electorate for anything more than a photo op. The ‘man on the street test’ would clearly show that Dutton’s comment innocuous and the Get Up ad defamatory in the extreme.

    Only the most blind, morally bankrupt, partisan bottom feeders would justify this behaviour (as I am sure they will here, given the quality of comments I have read on this blog so far (honestly… WHERE do you people COME from???).

    The irony is that videos like these can backfire badly and possibly have… every man and his dog have distanced themselves from this organisation as a result.

  17. By my recollection the only Republicans to speak publicly in shame/horror/outrage/dismay is Mitt Romney. Where is the supposed party of national security in the wake of Mueller’s report? And again the silence from their quarter when it comes to the gross misdeeds of their party’s president in office is deafening.

    President Trump was furious.

    He had just learned that special counsel Robert S. Mueller III’s investigation went beyond Russia’s interference in the 2016 campaign and into the White House — and that Trump himself was now under scrutiny for his actions in office. The next day, he attempted to oust Mueller, only to be thwarted by his White House counsel, according to the special counsel’s report.

    So Trump turned to the one person he could long count on to do his bidding: Corey Lewandowski, his former campaign manager, described by senior White House advisers to investigators as a Trump “devotee.” In a private Oval Office meeting, the president dictated a message he wanted delivered to then-Attorney General Jeff Sessions: that he needed to give a speech announcing he was limiting the scope of the investigation.

    Trump’s efforts to enlist Lewandowski as a back channel to try to curtail the probe, detailed in 10 pages of Mueller’s 448-page report, provides a new window into how far the president went in trying to hold back the special counsel.

    https://www.washingtonpost.com/politics/stymied-by-aides-trump-sought-out-loyalist-to-curtail-special-counsel–and-drew-muellers-glare/2019/04/25/d58c79de-66ad-11e9-8985-4cf30147bdca_story.html?utm_term=.ab5222cd0dc5

  18. [Bemused’s favourite topic was always bemused.]

    Spot on.

    Thankfully Bemused has been outed. How dare he come here under an assumed name to dish out his abuse again.

    Good riddance to bad rubbish!

  19. There was young man called Wayne
    Who had a very small brain
    Bree is his daughter
    Who is also a Liberal supporter
    But their party is going down the drain.

  20. Sceptic
    says:
    Friday, April 26, 2019 at 9:07 pm
    Nath, Wayne (& EGW)
    You should all take up yoga, 9 years in the wilderness is a long time, that’s how long the Liberal & Country parties will take to regroup after May 18 rejection slips are delivered.
    ___________________________________
    I don’t mind a bit of Yoga but Tai chi is the SHIT.

  21. C@t @8,11pm

    Was not it you who was criticising someone earlier about talking about the self.

    How come everyone here knows so much about you. Now you add more info.

    So it is only other’s who shouldn’t talk about themself ……… but it is ok for the self appointed gatekeeper.

    You should stick to the one position.

    Cheers

  22. It was obvious to anybody except the likes of nath and wayne etc., that bemused would out himself sooner or later. Nothing surer.

  23. This is going to be a comments closed blog from tomorrow no?

    Can someone find the clown who used the run the all in brawl that was Ozpolitics and have in start it up again.

  24. this blatant lie that the ‘coalition’ remains inactive on climate change is just so stupid

    For sure. I mean it takes a lot of continuous effort on the part of the ‘coalition’ to ensure that we do absolutely nothing or go backwards.

  25. I must say though, hats off to Pegasus. He didn’t fool her from the get go! Fooled me but I have always been gullible. 🙂

  26. C@tmomma
    says:
    Friday, April 26, 2019 at 9:20 pm
    I must say though, hats off to Pegasus. He didn’t fool her from the get go! Fooled me but I have always been gullible.
    _________________
    Well you are a Shortenite!

  27. Sky News and the Manly Daily are hosting a live debate between Abbott and Steggall on May 2 at 4.30pm eastern.

    I assume it will be broadcast live on Sky, and I assume Abbott has only agreed to a debate in his own electorate he’s held for 2 decades because he’s under threat. This is delicious, I love how Steggall has given him a run for his money and I hope she can prevail as the Member for Warringah.

    https://www.instagram.com/p/BwtwbDBpX6h/

  28. Palmer was no clown, though some of his posters may have been.

    I tend to agree, but you’d have to be a clown to run that place, and yeah he wised up.

Leave a Reply

Your email address will not be published. Required fields are marked *