Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

  1. Peter Stanton

    That’s called negotiating.

    After all if the Greens position and what you call demands was so bad Gillard and the CONSERVATIVE Independents would have walked away.

    This same crap was brought about the Medivac Bill. Bottom line. The Bill passed. Labor wasn’t even in government. It wasn’t Labor’s Bill. It was the Independent’s.

    Labor people on this blog have to get over this my way or the highway mentality

  2. “Bah, Shorten ruling out working with the Greens on climate policy is so underwhelming. Exactly the same mistake Rudd did and it wrecked his government. Gillard did a deal with Bob Brown and it was a solid result.”

    That’s crap. If Labor goes to the election with a set of policies the public who votes for them expect Labor to implement those policies not Greens policies but Labor policies. Greens think 10% of the vote and 1 seat in the House of Reps gives them a mandate to have their policies implemented. Oh please!

  3. guytaur @ #50 Friday, April 26th, 2019 – 8:10 am

    Peter Stanton

    That’s called negotiating.

    After all if the Greens position and what you call demands was so bad Gillard and the CONSERVATIVE Independents would have walked away.

    This same crap was brought about the Medivac Bill. Bottom line. The Bill passed. Labor wasn’t even in government. It wasn’t Labor’s Bill. It was the Independent’s.

    Labor people on this blog have to get over this my way or the highway mentality

    You better tell Di Natale

    Di Natale also wants it known that his party is prepared to vote against a climate policy it regards as insufficiently ambitious

  4. When Labor win the Election they will be implementing the policies they campaigned on.

    The Greens making radical demands and holding the process hostage is unlikely to succeed.

    Labor’s environmental policy is pretty much theNEG revisited. As this is Liberal policy that passed in their Party room three times and the fact they didn’t implement it, is at the core of their electoral unpopularity. So, they are likely to support the policy when introduced simply to get the issue off the agenda.

    So, if the Greens want to play “Our way or nothing “ games, they should prepare themselves for nothing!

  5. The Greens can no longer secure a loan from the bank of credibility.
    The Greens need to make alternative funding arrangements or the impasse will unfortunately remain.

  6. For fans of KK.

    Peter van Onselen
    ‏@vanOnselenP
    7m7 minutes ago

    I’m hosting for Waleed on The Project tonight….and joining us in the fourth chair for the first time is @KKeneally #auspol 6:30pm

  7. Bill MaherVerified account @billmaher
    2h2 hours ago

    So tired of hearing about Mueller’s “trail of breadcrumbs”. And of everyone still being so deferential to him – will get into this “bigly” tomorrow night when Real Time returns. Leaving it up to congress – that’s what I call a #FecklessPunt

  8. Vogon Poet.

    Exactly the position Bob Brown and the Greens had with Rudd and Gillard.

    We know which one of those two passed legislation.

    So we know having both Greens and conservative Independents negotiating directly works. Gillard proved it. That’s the path to successfully passing legislation. The Rudd path fails.

    Labor won’t even try. They know the LNP are climate deniers and can’t be appeased.

  9. C@tmomma says:
    Friday, April 26, 2019 at 6:57 am
    Labor just keeps rolling out worthy initiatives
    —————————————

    These initiatives are the dividends we all enjoy when a government is bold enough to close off “sacrosanct” tax perks enjoyed by the already very comfortably off. That is where the money is coming from for Labor’s initiatives, unlike the magic pudding of speculative modelling assumptions the Coalition is relying upon.

  10. I’m wondering what else Scott’s mob have hidden during the ‘celebrations’.

    Scott Ludlam
    ‏@Scottludlam
    3m3 minutes ago

    when government goes rancid: a toxic uranium mine that that Tjiwarl mob are fighting on country and in the courts, approved in secret and announced under cover of anzac commemorations

  11. “Is this the longest we’ve ever gone in a campaign in the modern era without polls?”
    There were no polls for a long time in the Victorian election until the last day or two showing Labor on 53 and 54. It was 57+ on election day. The media has a narrative, and they don’t want to ruin it with unsupportive polls.

  12. I would have thought that the Greens being satisfied with anything the ALP did would be a violation of their business model, which is all about virtue signalling rather than real world outcomes.

    In my area (inner northern Melbourne), the Gs are firmly in support of affordable housing in principle, but not in practice when proposals come up that offend their upper middle class constituency’s desire to preserve their property values. Somehow there’s always a minor design fault identified that allows the Gs to weasel their way out of supporting anything practical and worthwhile. Not that the Victorian Socialist councillors are any better.

    The same thing plays out across a wide range of policy areas.

    One other thing – as generally a lurker rather than a regular poster here, it riles me that people don’t have the guts to use their real names when posting.

  13. So, seat polling is more unreliable and it is biased in favour of the Coalition…. who would have thought?… 🙂

    In any event, still no new polls…. Yeah, we know that all the holidays, commemorative days, etc. are expected to interfere with the pollsters’ activity, but then, was choosing this breaks-packed period of the year for the election campaign just a chance event or was it by design?…. Surely ScuMo hopes that the more distracted the voters are, the easier it will be for Murdoch et al. to do their job…… Will it work?…. Nah!

  14. Another 11th hour approval slipped through prior to caretaker mode.

    The Morrison Government signed off on a controversial uranium mine one day before calling the federal election, and did not publicly announce the move until the environment department uploaded the approval document the day before Anzac Day.

    The Yeelirrie Uranium mine, located 500 kilometres north of Kalgoorlie in Western Australia, requires both federal and state approval.

    The state approval of the proposed mine is still being fought in the state’s Supreme Court by members of the Tjiwarl traditional owners.

    In 2016, the West Australian Environment Protection Agency advised the mine not be approved, concluding it posed too great a risk of extinction to some native animals.

    The former Liberal Barnett government controversially approved the mine in 2017, just weeks before it lost the West Australian election.

    https://www.abc.net.au/news/2019-04-26/government-approved-uranium-mine-day-before-election/11047252

  15. Sohar.

    Yes. This is precisely why Morrison chose maximum holidays.
    For very practical reasons it limits polling.

    In fact that’s why the media is right in saying the campaign really starts this coming Monday.

    Voters will be over their holidays.
    My view has not changed. I still see the odds against the LNP and that it’s Labor’s election to lose. So yes it’s not won. However the betting markets are saying Labor to win for good reason.

  16. Notwithstanding the absence of new polling data, as it stands there are four distinct, nontrivial, possible outcomes of the upcoming election, considering that there may be a reasonably sized cross bench.
    (a) Coalition majority
    (b) Coalition minority
    (c) Labor minority
    (d) Labor majority

    The most intriguing situation is (c). Would this wind up like Steve Bracks in 1999, or, rather, end in disaster much like Julia Gillard where there is ruthless ball-wrecking from whoever the Opposition leader is? Also, with the Murdoch mainstream media in perpetual campaign mode for the Coalition, would this bring down the new government sooner rather than later?
    I believe situation (d) with a small enough majority could wind up the same way, with resignations and defections leading to a minority within a few months, or at most, a year or two.

    Also, what are your thoughts about the stability of another Coalition minority?

  17. There were no polls for a long time in the Victorian election until the last day or two showing Labor on 53 and 54. It was 57+ on election day.

    I can’t imagine there being no polling until the end of the campaign! 😮

  18. “Greensborough Growler says:
    Friday, April 26, 2019 at 8:14 am
    When Labor win the Election they will be implementing the policies they campaigned on.

    The Greens making radical demands and holding the process hostage is unlikely to succeed.”

    Perhaps. Although whoever holds the balance of power in the senate will also claim a mandate – especially if Labor get less than 50% of the primaries (which they won’t).

    To think everyone else will simply roll over seems fanciful to say the least.

    Of course, if Labor and the LNP agree, nobody else can stop them

  19. guytaur says:

    Yes. This is precisely why Morrison chose maximum holidays.
    For very practical reasons it limits polling.

    More important for them is the saving in campaign $s. They were pretty skint last time and no Lucien Aye to tip in a lazy $million or so this time around.

  20. Oh, and I notice some interest about Di Natale’s statement on Labor and common climate change policies, etc.

    A message to Richard (and Greens supporters here): Focus on the current campaign first, get enough votes to become relevant in the Senate, and then approach PM Bill Shorten. Because if you Greens go backwards and your vote is not needed in the Senate, as the ALP Government can pass legislation with the help of other progressives (e.g. independents), the Greens will become numerically irrelevant. In which case they can still sit at the negotiating table, where PM Shorten will welcome them, but they will have to take on a much, much humbler attitude…. Will they be politically mature enough to do so?….. Let’s wait and see…..

  21. With the ho0-ha over Palmer’s preferences I was under the impression that in 2013 most of the PUP preferences went to the Coalition anyway. Does anyone have any stats laying around on where the PUP preference went in 2013?

    Steve Watson – many of us choose not use real names for many reasons. Mine was because of employment – its not wise to broadcast your political views if a state/ federal public servant etc

  22. Gecko

    I am saying Labor and the Greens will cooperate. My reasons for that are very simple. Despite the campaign grandstanding. It just comes down to self interest.

    Self Interest says Labor and Greens cooperating makes Climate policy stronger. If you are for action on climate change according to science it’s that simple.

    The fact is the deniers won’t accept anything as they have proved comprehensively.

    Labor is not stupid. They know this

  23. Wow, some real confusion over why the voting public reacted so badly after the Gillard Government passed (actually pretty good) climate change legislation in 2010 in concert with the Greens and Independents. It was because Gillard was hung over her blatant backflip (“No carbon tax under any government I lead”). The public may or may not have disliked the actual policy. What they vehemently rejected was what they saw as a breach of faith on a clear election promise.

    The Keating Government suffered a similar fate after raising every wholesale sales tax rate in John Dawkins’ 1993 budget, less than six months after winning an election largely on clear, firm opposition to increasing regressive indirect taxes (aka the GST). At the time, that was also widely seen as an outrageous breach of faith. Except for when the hapless Downer briefly led the Libs, the Keating Government never recovered from that.

    And remember, Rudd’s popularity remained sky-high right up until April 2010. Then he announced he was putting climate change action on ice for the time being, after campaigning on it as the “great moral challenge of our time”, and his popularity sank like a stone thereafter.

    Bill Shorten and the rest of the party know that what kills Labor every time is being seen to have breached faith with the voters. That is why they are upfront about their tax reforms. That is why they are upfront about not unrealistically raising expectations on the pace of climate change action. Those pushing for “further and faster” at this stage, with the election still to win, need to remember that.

  24. Morning all

    From what I have gleaned from reporting on news programs etc, is that Morrison is running a great campaign and the coalition are closing in on Labor. Yet to hear that Labor have this election in the bag.

  25. Michael A

    Exactly. That’s why I used the word cooperate between Greens an Labor.

    The unrealistic part Di Natale states is being objected to doesn’t get in the way of those expectations.

    Voters expect Labor to take realistic action on climate change.
    So unless commentators think Labor is going with unrealistic policy thus breaching voters expectations I don’t see the problem


  26. guytaur says:
    Friday, April 26, 2019 at 8:10 am


    This same crap was brought about the Medivac Bill. Bottom line. The Bill passed. Labor wasn’t even in government. It wasn’t Labor’s Bill. It was the Independent’s.


    The bill Di Natale tried to skittle in the morning? If only got through because Di Natale was not in the room but on TV trying to scuttle it, leaving Adam Bandt to negotiate. The Greens lack of influence has come about because of their complete lack of honor. Don’t cry about it, do something about it.

  27. Well my hopes faded early.

    Here I was hoping that we would see a thread without the Greens – Labor continued war or at least a f**king day without out reading the same endless comments from the same posters (you must all have your comments saved as a text file they are so same / same).

    But no, first skirmish began at just after 0700h. So the day is set for another day, same as yesterday and the day before and the day before that.

    Thank you to those that inject some comments not related to the endless war to end all wars but it is not enough to overcome the continuing bore of post after post by the same coterie of posters.

  28. Fess

    For one, not surprised. There is now a very crowded field of democrats running.
    Personally, he has been in politics a long time and was VP. He would be the best candidate if you consider these things.
    He has experienced a lot. Losing first wife and daughter in accident, and recently his son to brain cancer. He is resilient to say the least.
    My friend actually met him when he came to Australia a few years back. She said he was so easy to talk to and very personable, with excellent communication skills.
    He would need a female running mate if he became the nominee.
    As mentioned, I really like Adam Schiff, but he hasn’t even put his hand up.
    Eric Swalwell is good too. But Americans being a little cray cray about their guns, not sure if gun control platform he has, will garner enough support.

Leave a Reply

Your email address will not be published. Required fields are marked *