Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

  1. FredNK

    Ah. Yes. The narrative. It was Labor trying to weaken Phelps deal not Phelps.
    That’s why the whining about Di Natale.

    How dare he point this out.

    It was NOT Labor’s legislation.

  2. Biden is the obvious establishment candidate, and will get significant support from it.
    They will do whatever it takes to keep all those… radicals (progressives), off their lawn.

  3. Guytaur, yes. Commenters here seem to be falling into the trap of demanding Labor or the Greens rule in or rule out various “red lines” before the election is even over. This frames the discussion in terms of promises to exacting constituencies that will either be kept or broken, and that approach is both unhelpful in crafting excellent policy and risks exposing Labor (in particular) to what has been historically the most electorally fatal accusation that can be made against them: breach of faith.

    Much better for the broad left to frame the climate change debate in much more general, “big picture” terms: “who do you trust more to meet this dire challenge: Labor or Coalition?”, than in argy-bargy over who will hold the whip hand in Labor-Greens-crossbench negotiations, or in pre-empting the substantive content of those negotiations.

    In fact I think it is unhelpful to draw any attention at all to the prospect post-election negotiations exactly because it frames the debate in a way targeted at Labor’s greatest vulnerability. (RDN is either a political numbskull, or he intended just such a wedge against Labor when he prematurely raised the subject of post-election Labor-Greens negotiations recently.)

  4. “Your example proves my point. To get action on climate change Gillard had to concede to the Greens demands. That ended well didn’t it.”

    That’s what happens in a minority government. The Greens succeeded in getting Labor to agree to implement the carbon price/ETS and in return they supported the Gillard government all the way until the end, even turning a blind eye while Labor was engaged in the Rudd vs Gillard civil war. Along with the independents, the Greens ensured that the government lasted it’s full term.

    As for it ending well, yes it did actually. The Bandt/Gillard/Ind government was very productive, despite all the negative media attention it received. The Greens are not to blame for Labor destroying two decent PMs in one go and handing power to Abbott.

  5. Vic:

    Yes Biden is obviously well qualified. I don’t see him winning the nomination, but it is very early days. I’m all for any Democrat candidate who can beat out Orange Crush, but I agree that Swalwell would be a long shot.

  6. Fess

    I look at who would appeal to a majority. So far, it is a Biden, Swalwell and Schiff if he were to run.
    I can’t see the other candidates being able to do so.

  7. Michael A

    Labor was at the time attacking the Greens using the same extreme rhetoric the Murdoch media and some Labor partisans. Publicly.

    In the media unlike PB. It was a 24 hour wonder not noticed by most voters.

    What has happened since is the public slanging match has mostly ceased.
    I hope it stays that way as it keeps the focus on who is going to win the election

    Edit: as a result the preference talk today is about Clive Palmer not the Greens.

  8. The long term problem for the greens, is that to deliver for their ‘base’, they have to drag Labor to the left so far that its rejected by mainstream voters.

    Labor and Green can haggle and negotiate all they want, but if they cant sell it to mainstream voters they both get hurt by it, Labor moreso than the Greens.

    If greens focused on socially progressive, Libertarian issues then there wouldnt be a problem, issues like SSM (in the past), Death with Dignity, Legalising Marijuana, all those issue wedge Labor and proper (Non-conservative) Liberals.

    But i expect Greens will try in vain to break the world with their outright ban on coal exports by 2030, which i expect is pretty much impossible to implement in the current international legal framework.

  9. Guytaur and others, the endless repetitive flame wars achieve what exactly?
    Why don’t you all wait and see what happens on 18 May?
    Then, if you wish, have at it. Until one knows the political landscape after 18 May all of this to and fro is useless.
    That being said, I bet not one Party negotiator from any side takes any notice of what is said on PB in their deliberations.
    Give it a rest, there are bigger fish to fry along the road to May 18.
    Think about and discuss that more often. I’d like to hear more on what is happening on the ground in your various seats. Discuss what is happening and your take on it.
    That would be far more interesting to digest as we await further polling.

  10. I should think that today many electors are asking who Clive Palmer is and what he represents, and whether this new development affects their estimation of the Liberal party.

  11. Fozzie Logic says:
    Friday, April 26, 2019 at 7:25 am
    I’ve renewed my Newspoll predictions between now and May 17th. They will all
    be 52/48 to ALP as of all the polls they do over the weekends they will always
    pick the one most favourable to the LNP so they will skew in the LNP’s favour, for how
    bad they are… Essential will bounce around the true, fixed in aspic, result of 53/47

    I don’t think they are trying to show bias to LNP, I just think they’re wary of the 53% mark.
    Have a look at elections back to 1972. A TPP of 53%+ has resulted in a landslide.

  12. “And remember, Rudd’s popularity remained sky-high right up until April 2010. Then he announced he was putting climate change action on ice for the time being, after campaigning on it as the “great moral challenge of our time”, and his popularity sank like a stone thereafter.”

    This is one of the main reasons I switched from voting for Labor in 07 to the Greens in 10. I liked Rudd as PM (I liked Gillard too) but I was very angry with him when he first decided to try and do a deal with Turnbull and then when that failed pretty much abandoned climate change altogether.

  13. Great analysis Prof Bludger.

    So, if I understand it correctly, the upshot is that seats are more likely to bolt unexpectedly from Coalition to Labor, than the other way around?

    The only Liberal gain I can recall that no-one saw coming was Banks in 2013 (?).

  14. Girlfriend says to boyfriend “have you been having sex behind my back?”
    Boyfriend (slightly bewildered) “yeah… who’d you think it was?”

  15. Vic:

    Biden’s name recognition is a huge advantage for him.

    Meanwhile I’m looking forward to Barr’s testimony before Congress next week. He certainly has some questions to answer.

  16. Michael A @ #103 Friday, April 26th, 2019 – 9:11 am

    Guytaur, yes. Commenters here seem to be falling into the trap of demanding Labor or the Greens rule in or rule out various “red lines” before the election is even over.

    They should, to an extent. Shorten needs to stop trying to be all things to all people.

    When someone on $250k/year asks him “where’s my tax cut?”, he needs to say “on that salary, you won’t be getting one” instead of “we’ll look into it”. When journalists hound him on Adani, he needs to rule out both 1) allocating even a single dollar of government funding going towards the project, and 2) overturning agreements already made by the current government.

    This frames the discussion in terms of promises to exacting constituencies that will either be kept or broken

    …you mean, like in every democratic election between now and whenever the first recorded elections were held?

    Labor shouldn’t box itself in with undeliverable promises. Neither should they run a muddled, promise-lite campaign on vague ‘trust us’ vibes. They should loudly and clearly promise some basic things:

    1. No Federal funding (or subsidies) for Adani (or any other new coal mines)
    2. No tax cuts for the rich or big business
    3. Bigger, better, and fairer tax cuts for everyone else
    4. A Federal ICAC (or whatever they’re calling it)
    5. Investment in renewables/action on climate-change

  17. The most important point on climate change action at this point in time, is that any legislation is enduring legislation.

    So far we’ve had, blocked, passed, rescinded, so we’re where we were 12 years ago.

    Labor recognises this, and as a starting point want something that won’t be ripped up and discarded with a subsequent change in Government, but also can be further developed and expanded.

    The gun went off a long time ago and we’re still standing on the start line!

  18. FredNK

    As I said the narrative.

    I don’t buy it. I saw the live comments myself.
    They were not wrecking comments. Labor was not negotiating with the government. The Greens had already agreed with Phelps.
    It was Labor trying to weaken the legislation with amendments. So if it was going to fail it would have been due to Labor not coming on board.

    All Di Natale did was say this in public.
    I am not a fan. I did say at the time it did not need to be said publicly. However that doesn’t make Di Natale. a wrecker.

    To see wrecking look at Abbott.

  19. The Guardian

    The Liberal party has just released a statement on the election debates:

    The prime minister looks forward to Monday’s debate in Perth and next Friday’s People’s Forum in Brisbane.

    The Liberal party has also been in discussions with Nine and the ABC to participate in two further leaders’ debates in the final weeks of the campaign.

    Mr Shorten is so far refusing to participate in either of these nationally televised debates.

  20. Very good points being made. If Labor is looking particularly strong in the polling, Newspoll will still have an incentive to sit around 52 – 48 both to keep its paymasters happy and a knowledge that few parties get more than 53 in an election. For the libs, 48 may be the new 47, 46 …

  21. This is one of the main reasons I switched from voting for Labor in 07 to the Greens in 10. I liked Rudd as PM (I liked Gillard too) but I was very angry with him when he first decided to try and do a deal with Turnbull and then when that failed pretty much abandoned climate change altogether.

    This really really poor understanding of our Parliament and how it functions and what PM’s can do and can’t do, is surprisingly broad across the left. Astoundingly ignorant. Either that or this hilarious little fiction is what people tell themselves to feel better about the greens assumption that they could say ‘no action is better than the CPRS’ for nearly 3 years and then go to an election where Labor would win easily and they could control the ‘nothing’, and then getting it hopelessly wrong. Then astoundingly the stuffed up the next one too and even more astoundingly 10 years later are promising to destroy our third attempt as a country if it doesn’t meet up to their high standards (where we have 10 years of evidence their high standards have a 100% fail rate).

    Who knows perhaps doing the same dumb thing will work third time.

  22. Labor’s support for ‘carbon disaster’ in Beetaloo basin condemned:


    Labor’s support for unlocking the gas supply from the Northern Territory’s Beetaloo basin has drawn the anger of environmental groups, who say its emissions would dwarf those from Adani’s proposed Carmichael coalmine.
    But environment groups say that would undermine Labor’s target of reducing carbon emissions by 45% of 2005 levels by 2030.

  23. “If greens focused on socially progressive, Libertarian issues then there wouldnt be a problem, issues like SSM (in the past), Death with Dignity, Legalising Marijuana, all those issue wedge Labor and proper (Non-conservative) Liberals.”

    The Greens campaigned for SSM for decades. Big tick on achieving that one, even if it didn’t happen the way it should have.

    Euthanasia/dying with dignity is another issue the Greens have long supported and which now has very wide ranging support. It’s a very sensitive issue and it’s best if politics and ideology are both kept out of it as much as possible.

    After campainging for medical marijuana for decades and achieving widespread support for it with the help of the medical community, the Greens are now championing the cause for full legalisation of cannabis. A decision was made long ago to focus on getting medical marijuana legalised first as the priority.

    Richard Di Natale is also the perfect leader to champion causes relating to drugs, considering he’s a former GP who has worked extensively in the drug and alcohol health sector. It’s rare that a politician is so highly qualified on a specific policy like that. He’s also the perfect leader to champion the Greens’ pill testing policy for the same reasons.

  24. “Di Natale also wants it known that his party is prepared to vote against a climate policy it regards as insufficiently ambitious ”


    From the article Shorten speaking well to that issue. To me those kind of comments by Di Natale signal that the Greens have not learned a thing from the last few years of Climate Inaction.

    Even at a raw political level the Greens HAVE to learn that the “ALP/Green Coalition” theme is one that does not help the ALP electorally. Simple fact of life. And here they are actively promoting during an election campaign a theme that would give the likes of the Murdochracy something to latch onto, go hysterically feral about, and divert attention from the positives the ALP will have to annonce over the next couple of weeks.

    Fine, i dont expect the Greens to be wanting to “help” the ALP, they are not the ALP and are their own party. But, the main game this time is to get rid of the Coalition. So, i expect them to refrain from going out there and actually trying to damage the ALP’s chances.

    I am really not happy with the Greens over this.

    They are acting like they want no action but lots of argument about Climate and Energy policy. Maybe that suits them in terms of maintaining political relevance, in the same way that the Libs want Borders / AS to be an issue that never goes away. Some nice people in the Greens…but they are run by opportunistic arse-holes.

    If that’s where they are on this they absolutely miss out on my primary vote in both houses. Yup, they get preferenced above the Lib/Nats but that hardly a compliment is it?

  25. Hughes hasn’t been targeted because it’s on an almost 10% margin. Labor are campaigning in seats they already hold instead which should give you an insight into what’s happening.

  26. Barney

    The only way to have enduring climate legislation is to keep calling out the deniers.

    Remember Rudd thought he was dealing with people who did not have as good as a flat earth mentality

  27. guytaur says:
    Friday, April 26, 2019 at 9:34 am

    It was Labor trying to weaken the legislation with amendments. So if it was going to fail it would have been due to Labor not coming on board.

    Do you mean the changes that stopped the Bill being unconstitutional and thereby neatly sidestepping the Government’s objections to it? 😆

  28. Di Natalie is unlikely to be the Greens Leader after the Election.

    Losing a couple of Senators will destroy his credibility.

    He’s uninspiring and has made a total hash of the job.

  29. guytaur says:
    Friday, April 26, 2019 at 9:48 am


    So it was claimed.

    Yet we know how fraught The High Court Shall So Rule is.

    Well the changes were certainly effective in killing those claims.

  30. https://www.theage.com.au/federal-election-2019/democracy-at-stake-parties-warned-australia-at-risk-of-us-style-cyber-manipulation-20190424-p51gu3.html

    Former privacy tsars and technology experts have warned the major political parties they must dramatically strengthen their cybersecurity to protect the growing mountains of private data gathered on voters that could be used by foreign adversaries to manipulate elections.

    More than two months since the Morrison government revealed that the three major parties had all been victims of a cyber attack early in the year, the Liberal, Labor and National parties have provided scant detail on how they were affected, nor what they have done to improve their defences.
    The cybersecurity standards of the major parties – who are exempt from the Privacy Act – have come under fire from experts in the wake of the cyber attack, which also hit the parliamentary computer network.
    He and other experts said parties used the electoral roll as a foundation to build elaborate databases on voters, enhanced by information fed by MPs’ offices and likely from social media.

    “There is no doubt that political parties have extensive databases on a very large proportion of the Australian population. They are voracious in trying to suck up anything they can from anywhere. Anybody who goes into an electorate office will be sucked dry,” he said.

  31. Hannity pondering bailing on Fox News because he no longer feels the network is loyal to Trump: report

    A new Vanity Fair analysis behind the scenes of Fox News reveals an interesting new piece of information: longtime Fox megastar conservative commentator Sean Hannity has told friends that he plans to leave the network after his current contract expires in 2021.

    According to sources, Hannity is angry at the Murdochs, the powerful conservative family that owns the network, believing they are insufficiently loyal to President Donald Trump and secretly out to get him.

    Another source claimed that Hannity even “told Trump last year that the Murdochs hate Trump, and Hannity is the only one holding Fox together.”



  32. Someone was rabbiting on about this Kate Ashmor person yesterday. I’d never heard of her until then, but she seems to have perennially foot in mouth disease. Long may she continue to be the face of the Liberal campaign in NSW.

    Angry school teachers have confronted a state Liberal MP demanding to know whether he also believes private schools are “far superior” to public schools, as the federal Liberal candidate for Macnamara has said.

    The exchanges at Thursday’s ANZAC Day ceremony in Elwood were sparked by revelations the federal candidate for the seat of Macnamara in Melbourne, Kate Ashmor, described the quality and teaching at private schools as “far superior” to public schools in a letter to a local newspaper in 2001.

    While Ms Ashmor has apologised for referring to the wife of Labor leader Bill Shorten as a pig, she has not returned calls about her comments criticising state schools or about her opinion that childless leaders like Julia Gillard “lack empathy”.


  33. GG:

    Di Natale also has very poor leadership skills. The hash he made over the sexual assault and bullying complaints inside the party being a total case in point.

  34. a r says:
    Friday, April 26, 2019 at 9:29 am

    I once thought your way about what elections were fundamentally about. But so often governments are deceptive about the true state of things in the economy and wider society, that it is unrealistic to expect an opposition to possess sufficiently reliable information upon which to satisfactorily base precise promises.

    I now prefer to see elections as opportunities for the voting public to choose the general direction they want the country to head in from that time onward for the time being: conservative or progressive. Any expectations upon elections that are more precise than that are just not feasible imho. Nobody has sufficient information to be able to predict either the successful passage of exact policies through Parliament, or the effect those policies will have in the real world even if fully and exactly passed as promised.

    So, I tend to dismiss the usual to-and-fro over precise costings, and even the somewhat more serious debates over exactly how a given party in a given election is striking the balance between competing objectives in a given policy space, as not really important in deciding who to vote for. No, I basically always view the country as needing to be “more progressive”, so I vote Labor. Others see the country as needing to be “much much more progressive” and so vote Green then Labor. All this is good as far as I am concerned. And none of it really hinges on how people predict/promise things like inter-party discussions go after the election.

Leave a Reply

Your email address will not be published. Required fields are marked *