Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

  1. Simon² Katich® says: Friday, April 26, 2019 at 10:31 am

    A very recent (tho pre campaign launch) Politico/Morning Consult poll (a large sample) had Biden ahead of Trump by 8 pts (excluding undecideds).

    A Biden-Klobuchar ticket would be Trumps worst nightmare. But I doubt that would work.

    Bidens running mate choice will be more important than usual.


    Extremely unlikely but this article poses an interesting question :

    Could Joe Biden Pick Barack Obama as his Running Mate in 2020?


  2. The other point is to remember that if you think you’re going to convince/evangelise to people HERE and it lead to conversion … you’re wasting your time.

  3. Could Joe Biden Pick Barack Obama as his Running Mate in 2020?

    Sorry, dont have time to read it now. But I thought the VP has to meet the requirements to become President. So ruling Obama out?

  4. Morrison giving away a little too much here:

    Mr Morrison said Mr Shorten had the “presumption” that he would win the election.

    “Bill needs to get off the coronation tour and he needs to get on the campaign,” he said.

    The statements escalate a political dispute over the scrutiny of the two leaders as early voting stations open on Monday ahead of the May 18 election.


    I haven’t seen the slightest indication that Bill Shorten thinks he’s involved in a “coronation”. All the commentary (including from Crowe, who wrote the article above) denies it too. In fact the meme is about how *badly* things are going for Labor.

    So where does ScoMo get the “coronation” idea from, except out of his own head, or from his own party sources?

    Morrison has his own problem: his “team”. There isn’t one, not that we can see, anyway.

    People are starting to notice. It’s ScoMo at the footy, drinking the beer, wearing the funny hats, clapping the happy, kissing the babies, doing every presser, being the one-man band. Where’s the rest of the gang?

    You can’t just be Captain all the time, scoring all the tries, kicking all the goals, keeping the score, bringing out the oranges and blowing the whistle. You need a team to follow you. The absence of a team means it’s either not up to scratch, or ScoMo doesn’t trust it. Another way of putting both propositions is: ScoMo only trusts himself.

    He is renowned for not being a team player. This tendency has gotten him into all his scrapes in the past, and gotten him fired from every job he’s been fired from. And tnere’s more than one of those. As for the team, they can’t be happy either. ScoMo’s behaviour is a slap in the face for them.

    Now he wants more debates, with only a few weeks to go. Tell him to get stuffed, Bill. If ScoMo wants endless debates, deny them to him. He’ll only bung on that Wall Of Sound thing again. And while you’re at it, demand to know just where his team is, and who they will be.

  5. If your taxable income from all sources was $209,343 in 2017-2018 you are in the top 2 percent of all tax filers.

    If your taxable income from all sources was $270,795 2017-2018 you are in the top 1 percent of all tax filers.

    These figures are from a spreadsheet prepared by Danielle Wood of the Grattan Institute. The spreadsheet presents the Grattan Institute’s estimates of taxable income for tax-filers by percentile (for financial year 2017-2018). They mostly use the 2 percent ATO sample file, which provides a lot of detail on distributional questions.

    So giving a tax cut to someone whose income is $250,000 should be about 4,742 on a government’s To Do list.

  6. Turnbull reckons it’s “frankly absurd” that “Western” nations have underperformed on developing 5G technology, leaving the field open to a couple of Asian and a couple of Scandanavian companies (Scandanavia is apparently not “Western” enough for Malcolm).

    I would have though it’s “frankly absurd” for Malcolm Turnbull to be lecturing ANYONE on how to build better networks, 5G or otherwise.


  7. Cheryl Kernot
    21m21 minutes ago

    Morrison asked how he can justify Palmer preferences here in Townsville of all places. His rehearsed deflection is: “We’re not supporting him. We want your number 1 vote. There have been no policy discussions.” Completely ignores Senate possibility. Ended press conference #auspol

  8. Michael West @MichaelWestBiz
    1h1 hour ago

    Push for “small government” has gutted the public service, sent govt costs through the roof and put billions in the bank accounts of the Big4 @Triskeltic


    AFTER three days of public hearings and dozens of submissions into the Government’s unexplained splurge on consultants – particularly the biggest winners the Big Four accounting firms – the federal Joint Committee of Public Accounts and Audit has quietly taken the Inquiry into Australian Government Contract Reporting out into the back paddock and dispensed it with a quick bullet to the head.

  9. Well, I think we can call Herbert a Labor retain.

    Bill Shorten running with Jonathan Thurston; Scott Morrison running with Clive Palmer.

    Optics even the least politically engaged can’t miss.

  10. Simon² Katich® says:
    Friday, April 26, 2019 at 10:48 am

    Could Joe Biden Pick Barack Obama as his Running Mate in 2020?

    Sorry, dont have time to read it now. But I thought the VP has to meet the requirements to become President. So ruling Obama out?

    Did they get this idea from Russia?

  11. Bushfire Bill
    Friday, April 26th, 2019 – 10:51 am
    Comment #204

    In the same way that a good book will trigger research into the subject matter – your post brought old memories to life – herewith a couple of quotes:-


    Elevenses ☕

  12. Andrew says:
    Friday, April 26, 2019 at 10:32 am

    “As a bystander in the eternal arguments over who stuffed up emissions negotiations in the Rudd I era, I’d say both sides neglect the degree to which both the Greens and the ALP underestimated the ferocity and sheer petty bloody mindedness that Abbott would bring to destroying whatever was eventually agreed upon.

    By which I mean that I don’t think it mattered a jot whether the original CPRS got up or not. What we know from the perspective of 2018 is that Abbott’s only political skill is tearing stuff down. He was making trouble for Turnbull as opposition leader before he rolled him over emission reductions, and I reckon it’s pure fantasy to think that if the CPRS had been passed, he would have quietly accepted it; he was going to roll Turnbull at some point anyway, or at least try, and if he’d succeeded, he would have cheerfully ditched Liberal support for the CPRS, campaigned against it as a carbon tax, and quite possibly we’d have ended up at a similar place to where we are now. I don’t think that either the ALP or the Greens appreciated that at the time – Abbott was known to be a bit crazy, but thought to be ineffective. We all know how that turned out.”

    Interesting perspective but IMO there is a fundamental flaw to it.

    Political crazies like Abbott are only empowered by political vacuums.

    When the Greens felt left out over the centrist CPRS deal agreed to by Rudd and Turnbull they blocked that legislation. This created the vacuum where no carbon policy could be regarded as minimally acceptable to a majority. Abbott exploited that vacuum.

    On the counterfactual where the Greens had said some action on climate change was better than none (and then worked to get even better legislation) there would have been no political room for Abbott to work in. IMO most voters would have regarded the CPRS as being a solid start to effective emissions reduction and given Abbott’s climate denying claims a wide berth. Medicare and the ABC have survived, NOT because political crazies like Abbott are not potentially effective, but because most electors simply accept the current health/media arrangements are worth keeping in the main.

  13. I find it amazing as to the level of constant vitriol here on PB between mainly greens and Labor supporters!
    There seems to be a total lack of understanding of even the most basic principles of how our parliament was set up to function. For a start it was never to be about “party” politics , blue, red, green, yellow, purple with pink spots etc… but the house was set up to represent local constituents, with the Senate to be an oversight to scrutinise policy. What we have now are all these people stating that “their party” have the right to do what ever they want uncontested if they form government… that leads to dictatorship!
    I have seen it mentioned here on many occasions (even today) that the Greens have no right to dictate policy as they only get 10% of the vote? Well, true and untrue, are we saying that (according to our awesome bludger track) Labor even if they win the election, they also have no right to dictate policy because they have only attracted 37% of the vote?
    So in another perspective, roughly
    1 person in 10 will vote Green and 9 in 10 won’t
    3.7 people in 10 will vote Labor and 6.3 in 10 won’t
    3.8 people in 10 will vote LNP and 6.2 in 10 won’t
    Under any measure every party (ON, Palmer, etc included) have way more people not voting for them than supporting them…. Hence to constantly be preaching the my way or the highway approach does a total disservice to the way in which our democracy was set up to run, and in my humble opinion, the exact reason why the Coalition has torn itself to shreds internally since coming to power in 2013.

  14. I received an sms poll on Wednesday with the following questions:-

    OzPanel>Did you decide who you would vote for in the Federal Election before the campaign started on 11 April? Y=Yes N=No

    OzPanel>And why did you decide to vote that way? A=Always vote the same way C=It’s clear which party or independent is addressing my major concerns X=Other reason

  15. Shorten’s campaign is focussed on the ground in the seats he needs to win.

    Seeking approval of the MSM is irrelevant.

    Labor is winning.

    Why would he be interested in a forum structured to equalise exposure to both PM aspirants.

    A lot of voters still don’t know who Morrison is. However, they know Shorten thanks to the persistent naming of Bill throughout the last couple of years by various Libs and Nats. (Joyce named him 72 times in that Karvelis interview).

    Thanks for the free publicity,guys!

    Morrison is obviously desperate for a chance to speak at the undecideds.

    Seems to me that Morrison needs the deabates far more than Shorten.

  16. ‘The debates. Mr Shorten does well in those.’

    Sorry, but no.
    The town hall debate things with Morrison will be a disaster for Bill.
    Morrison’s easy preachy keen style of hosanna in the highest and motor mouth delivery will leave Bill gasping for air and the punters and media wetting themselves over how Morrison wiped the floor with Bill…..just preparing you all.

  17. National Press Club chief executive Maurice Reilly said: “The National Press Club has made a submission to hold a leaders’ debate. The club was the traditional venue for the leaders’ debate in previous elections, and we did the first one back in 1984.

    “In the absence of a debate commission, which is well-established in other countries like the US, the National Press Club is the most trusted neutral venue to hold the debate because it would provide it to every network in prime time.

    “We’re disappointed that we’re not doing it.”


    Notice the choice of the word ‘dodging’ rather than ‘refusing’, and the quote of Morrison’s words in the link.

    Dr James Farrow @jmfarrow
    16m16 minutes ago

    It’s the MSM that keeps cutting Shorten off and preferring Morrison’s vacuous beer drinking over Shorten’s policy speeches. When the MSM is complicit the Liberal’s can serve up lies and spin as they like with impunity. #auspol

  18. ‘”I’m happy to have one with Nine. We’re having one with Seven on Monday. We’re having one with Sky at the end of [next] week. I’d be happy to have one with the ABC.

    “I’ve personally been in touch with [ABC television host] Leigh Sales. I said I’d be quite happy for Leigh Sales, Bill Shorten and I in the same room. Fine by me.’

    No shit.

  19. mundo – Bill has done how many Town Halls?

    We’ve all been through enough elections to draw our own conclusions. The only Fed election I’ve gotten wrong since paying attention was 2004.

    You’ve given up, that’s fine, I don’t think you actually ever had hope or expectations to give up, but that’s by the by.

    And yes @Goll… probably a bit harsh, but I’m getting exhausted coming in here and just scrolling past tomes of information from people who think place is full of the open-minded…

  20. If the Coalition are wanting more debates it suggests they are not too confident about their position. You don’t go for more debates if you think you’re ahead (or even close really).

  21. mundo says:
    Friday, April 26, 2019 at 11:13 am
    ‘The debates. Mr Shorten does well in those.’

    Sorry, but no.
    The town hall debate things with Morrison will be a disaster for Bill.
    Morrison’s easy preachy keen style of hosanna in the highest and motor mouth delivery will leave Bill gasping for air and the punters and media wetting themselves over how Morrison wiped the floor with Bill…..just preparing you all.


    Morrison does not do debate, as you seem to concede. Morrison motor mouths talking points. This approach does NOT fare well in political debates where a bit of concentration on the actual content of the answer to the question is seen as being pretty relevant.

    Shorten should definitely agree to the debates, the more the better. Apart from anything else it will give voters the opportunity of comparing him as PM with the other candidate – and that can’t ever be bad for a hopeful challenger.

  22. ScoMo thinks he’s a master communicator… he’s running a one-man campaign.

    The strategy is clear, the messaging coming out of the campaign is that Morrison is front and centre and anything less than a disaster for the Coalition on 18 May will be presented as occurring because of him, therefore giving him licence to stay on as leader.

  23. Cud Chewer says:
    Friday, April 26, 2019 at 11:08 am

    Barney we have absolutely no need for Uranium

    And we don’t live in an isolationist world.

    If a mine can meet proper environmental guidelines I don’t have much of a problem with it being allowed to proceed.

    The problem I see is that those guidelines don’t seem to be the primary obstacle that needs to be overcome.

    Too often it seems to be an exercise in box ticking and empty promises.

  24. The big problem for journalists on twitter is that the comments are a million times more informative, funny and incisive than anything they offer.

    ltep @ #224 Friday, April 26th, 2019 – 11:24 am

    If the Coalition are wanting more debates it suggests they are not too confident about their position. You don’t go for more debates if you think you’re ahead (or even close really).

    Tick. Yep. A PM who proactively wants three debates is a country mile behind.

  25. WRT the ‘Uncle Tom Cobbley and All’ debate between Greens, Labor & other progressives as to who should do and own the policy development i suggest this is a better way to go:
    “I believe our task is to identify the best proposals from many different thinkers and shape them into a coherent alternative. Because no economic system is only an economic system but intrudes into every aspect of our lives, we need many minds from various disciplines – economic, environmental, political, cultural, social and logistical – working collaboratively to create a better way of organising ourselves that meets our needs without destroying our home. ”

  26. Watching from afar, it does seem to me that Morrison is running a ‘presidential’ campaign. If he wins (which heaven forfend) he will have an impossibly hard job as he’s well known not to be a team player. I’d say the cat and dog fights within his ‘cabinet’, even in opposition, are going to be very entertaining.

  27. I think, in the absence of polling, Morrison’s comments on the the debates are the clearest indication of how things stand at the moment.

    Little change or the Government thinks it’s going backwards. 🙂

  28. I find it amazing as to the level of constant vitriol here on PB between mainly greens and Labor supporters!

    You get used to it. Like shingles, the pain slowly subsides. It may take weeks, months or years.

  29. Bill Shorten should counter with a proposal for a series of policy debates with the relevant minister/shadow minister to participate.

    Bowen has already offered and Frydenberg has run a mile. Butler v Taylor, Burke v Price, Plibersek v Tehan.

    Morrison’s one-man show is necessary because his team is awful. Such an offer highlights that and puts policies at the centre of the campaign.

  30. Kay Jay, thinking he’s the only person who knows anything or who can get the job done properly is ScoMo’s achilles’ heel. It’s a Führer Complex.

    You can see it in the smirk and the swagger, the arrogance, and the inability to ever concede fault. You can see it in the way he is conducting the campaign.

    I’m surprised no-one’s complaining.

  31. Guardian

    The Australian Council of Trade Unions is launching its largest ever election advertising blitz on Friday.

    The campaign is bigger than its 2007 Your Rights at Work campaign, which toppled the Howard government and WorkChoices laws. That campaign had a $10m+ ad spend.

    The peak union body’s new television and radio ads feature workers aged 20 to 60 talking about their working life and their difficulties with job insecurity, cost of living pressure and low pay rates.

    Some are tradies, transport workers, miners, bar managers and swim teachers.

    In one ad a young woman called Laura talks about having five jobs to make ends meet. Another young woman says she’s worked seven days but only pulled $300.

  32. lizzie, re Morrison’s presidential style campaign: reminds me of Rudd’s campaign in 2013. If you look at both those elections as just furniture saving exercises, it makes sense.

  33. Good line.

    Shorten says the only thing the Coalition has done to equalise the income gap is to suppress all wages and make sure everyone is paid less.

  34. I think, in the absence of polling, Morrison’s comments on the the debates are the clearest indication of how things stand at the moment.

    Or Morrison just has oodles of confidence in his abilities (he probs thought he smashed it with Aly). Doesnt he have a desk ornament that says ‘you da man’ or ‘FIGJAM’ or wtte.

  35. Morrison pressing Shorten for more debates is a “ strange “ look to say the least.

    Usually the opposition leader is the one doing the chasing. Interesting optic.


    Bowen and Catherine King are locked in to debate their rivals. At least one other debate has been organised but I cannot recall which portfolio.

  36. Big A Adrian

    Morrison is unconscious of his own need to hold the fort. He reprimands a reporter for interrupting the local candidate, but then quickly takes over when he thinks the local man is offline. Then it gets awkward, so he closes the presser. That’s one of his skills, closing questions down. 😉

  37. People with nothing to lose push for debates.

    Looks like the press are impressed with a PM that puts beer mugs on his head; but the rest of us arn’t.

  38. Morrison is on his own in this campaign.

    It is partly hubris and mostly because he has no one else.

    Dutton is neutered. Hunt is fighting his own electorate battles as is Porter and a number of other senior liberals such as Pyne, Bishop and O’ Dwyer are retiring.

    Even if Morrison wanted back up the cupboard is bare.

    Compare that to labor. Plibersek is campaigning as are Albanese, Keneally on the Bill Bus and Wong across the country.

  39. “lizzie says:
    Friday, April 26, 2019 at 11:40 am
    5h5 hours ago

    Threat of Greens winning in Brisbane now has Liberal MP pretending he might not support Adani mine.”

    Hmm. He might say that mightn’t he. I have seen nothing suggesting that Bartlett will do better than 3rd. He gave up a Senate seat to focus on Brisbane, so this might just be a way to try to grab attention. Sounds silly and desperate doesn’t it.

  40. “10 years later”

    That article I linked to was written on the 14th of January, 2010. It’s been there that whole time. It is a contemporary account of what the Greens did at the time and why.

    And nah, I’m really not interested in going round in circles on the CPRS anymore. I’d just be repeating myself.

Leave a Reply

Your email address will not be published. Required fields are marked *