Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

  1. Puff,

    The Labor proposal is for the panel to consist of one ABC, one Nine,(?) because they both wanted to host one, and a third selected by the NPC.

  2. WeWantPaul

    can only guess they have some idea what fracking is, and you well, don’t.

    Excuse me, but from what I know fracking isn’t good. Perhaps you can enlighten me as to it’s benefits. I’d be really interested as having been involved with work relating to it for some months I’d really like to build some positives into that research.

  3. “The Labor proposal is for the panel to consist of one ABC, one Nine,(?) because they both wanted to host one, and a third selected by the NPC.”

    Now, lets look at the likely ScoMo pong to Shortens latest ping?

    ScoMo: No Bruthas and Sisters….daH NPC are just not Fair Dinkum coz they are part of that danged Canberra Bubble….believe me…i wear a baseball cap!!!!

  4. “I’d really like to build some positives into that research.”

    Then dont look at the long term hydrology aspects then, your ambitions will be thwarted. 🙁

  5. Polls out this week have shown the Liberal Party still retaining Corangamite and Deakin, if Labor can’t win those seats, they won’t gain other seats like La Trobe or Chisholm. Shorten is starting to be rejected even in his own home state of Victoria, Labor won’t make any gains in Queensland or NSW.

  6. This LNP government is a rolling shitshow.
    It doesn’t deserve to be re-elected.
    We all know this – even very small circus of LNP diehards still posting here know it, deep down.

  7. imacca

    Do you mean lowering the ground water by 1/2 a metre for 200 years?

    There are no positives beyond basic job creation.

  8. That’s true Steve – in fact, I’ll lose all respect for the Australian voter if they return these third-rate losers. But I dont expect that to happen.

    That never happened in previous years: deep down I knew Beazer had been a windbag in 98 and 01, and that Latho was a nutjob in 04. As much as I wanted them to win – I knew it.

    Not this time. We the Australian people wont be able to take ourselves seriously if these clowns continue as our government.

    The punters get this, and they’ll get it right on May 18. It’s time to change.

    Even Turnbull will be voting Phelps rather than LNP. If they cant even get their own recent PMs to support them, why would anyone else?

  9. A 9 Network hack was put down exceptionally well by Bowen at the NPC (he had written an article in the Financial Review which Bowen seized on when the Costello hack repeated his assumptions as a question)

    Have any of the usual suspects on here addressed the specifics they were invited to address instead of the nonsense carry on they repeat – and repeat and repeat and repeat. (Infinity)?

  10. Lefty-e
    Yes exactly.Also Murdoch wants the voters to be disengaged so he can manipulate them with his bullshit for the small amount of time that they are with Liberal propoganda.Brainwashed morons like Bree and Wayne are just the type of empty heads they are looking for and there are many more out there just like them.

  11. Sky News Australia
    @SkyNewsAust
    ·
    1h
    .
    @J_C_Campbell
    : It’s tightened here. If you talk to both sides, there is a view Labor’s expectations of what they’re going to win are being quietly dialled down.

    MORE: (link: https://bit.ly/2BuFqi1) bit.ly/2BuFqi1 #Sharri

  12. Meanwhile I’m hearing some journos have been trying to get a few stories off the ground. One being the Australian services Union and members monies being misused.

  13. Lols

    Sky News Australia
    ‏Verified account @SkyNewsAust

    .@billshortenmp: The Liberals have a sudden interest in debating us. If Mr Morrison wants to spend more time listening to me, he can always vote for me.

  14. Yep news ltd have also done one in Bass here in Tas stating that the coalition is in front there. In isolation theses seat polls can get people fired up and sre usually small samples, hence the need for more thorough polls

  15. Good morning Dawn Patrollers.

    Peter Hartcher writes that around the world, market economies are under stress. The Opposition Leader’s plans to lift fairness – and wages – might be the tonic our economy needs.
    https://www.smh.com.au/federal-election-2019/could-bill-shorten-be-the-saviour-of-capitalism-20190426-p51hny.html
    Shane Wright on Morrison’s deflections from questions on the deteriorating state of the economy.
    https://www.smh.com.au/federal-election-2019/scott-morrison-defends-his-record-as-signs-warn-of-slowing-economy-20190426-p51hka.html
    A very good examination here from Paul Bongiorno of the Coalition’s campaign.
    https://www.outline.com/34ebzr
    Karen Middleton reveals how former AFP chief Mick Keelty, in his capacity as the Northern Basin commissioner for the Murray–Darling Basin, is examining links between political donations and water licences, and calling for proceeds-of-crime laws to be expanded.
    https://www.outline.com/3vvtUS
    Katharine Murphy has a detailed look at Day 16 of the campaign.
    https://www.theguardian.com/australia-news/2019/apr/26/campaign-catchup-2019-morrison-falls-into-a-coal-black-hole
    Martin McKenzie-Murray writes that while high-profile candidate Julian Burnside has animated the Greens in Kooyong, party strategists fear there is a lot to lose.
    https://www.outline.com/3gnkeD
    Bill Shorten says he is “not convinced” on the union movement’s demand that he extend industry-wide bargaining rights beyond low-paid industries, as the ACTU unveils a massive ad blitz calling for a dramatic rewriting of workplace laws.
    https://www.smh.com.au/federal-election-2019/shorten-not-convinced-on-union-demand-to-extend-industry-bargaining-20190426-p51hj5.html
    Labor has angrily rejected suggestions it dragged its feet on national security after Scott Morrison used the example of just one unlegislated bill to criticise the opposition in the aftermath of the Sri Lanka attacks.
    https://www.theguardian.com/australia-news/2019/apr/26/shorten-angrily-rejects-pms-accusation-labor-dragged-feet-on-national-security-bills
    Mike Seccombe tells us that data shows the Coalition’s 2016 election victory hinged on just three ethnically diverse seats – Banks, Reid and Chisholm. This time around Labor is shaping its campaign around winning back these communities.
    https://www.outline.com/7DMtkL
    Ross Gittins reveals that the sad reality is that every year huge amounts of taxpayers’ money is wasted on infrastructure – and much of the damage is begun in election campaigns.
    https://www.smh.com.au/business/the-economy/here-s-an-election-promise-we-ll-waste-a-shedload-of-money-on-infrastructure-20190426-p51hhh.html
    Laura Tingle writes that if the government wasn’t so determined to paint itself as a small government party that is mostly interested in giving taxpayers a tax cut, it might have addressed the funding crisis in aged care.
    https://www.outline.com/kuy5ST
    Stephen Koukoulas explains how the RBA has an inflation problem.
    https://thekouk.com/item/675-another-hit-and-miss-the-rba-has-an-inflation-problem.html
    The AFR posits that the Coalition could win the election by limiting losses in Victoria and Queensland and picking up seats in NSW, Tasmania and the Northern Territory.
    https://www.outline.com/2XdA8J
    It wasn’t just the umpires that gave patrons the shits at the Anzac Day AFL match!
    https://www.smh.com.au/national/victoria/one-by-one-people-went-down-footy-fans-collapse-at-official-anzac-day-lunch-20190426-p51heo.html
    Shane Wright explains why death taxes could actually be a good idea.
    https://www.smh.com.au/federal-election-2019/enriching-equalising-efficient-why-heinously-unpopular-death-taxes-could-actually-be-a-good-idea-20190422-p51g6z.html
    Why voters under 30 will be crucial to the election result.
    https://www.outline.com/ss4m9L
    Sam Maiden describes saviour Clive Palmer as a false prophet for the Libs.
    https://thenewdaily.com.au/news/election-2019/2019/04/24/clive-palmer-pauline-hanson/
    Michelle Grattan writes about Palmer’s “flypaper sticky” deal with Morrison.
    https://theconversation.com/view-from-the-hill-palmer-flypaper-sticky-for-both-sides-116096
    Adele Ferguson tells us how Wacka Williams has said it’s time for regulators to lose the feather and use the sledgehammer. Fair Work Australia in particular.
    https://www.smh.com.au/business/workplace/lose-the-feather-and-use-sledgehammer-senator-puts-regulator-on-notice-20190426-p51hlm.html
    And the SMH says it’s time for law makers to stand up for victims of franchise scandals. It says the industry’s governing code is a joke. Its provisions are farcical and when breached the penalties are pathetic.
    https://www.smh.com.au/business/companies/time-for-law-makers-to-stand-up-for-victims-of-franchise-scandals-20190425-p51h5j.html
    Paula Matthewson cuts through the election lies to find the truth.
    https://thenewdaily.com.au/news/election-2019/2019/04/26/election-2019-paula-matthewson/
    The benefit of hindsight is a wonderful thing and in the wake of the Murray-Darling ‘disaster’, water allocations from the Fitzroy River in north-western Australia region are being approached with extreme caution.
    https://www.canberratimes.com.au/story/6091619/murray-darling-mistakes-drive-water-future-across-australia/?cs=14329
    The inquiry into the federal government’s spendathon on government contracts, especially on that global elite of corporate welfare recipients, the Big Four audit firms, has been axed — because of the federal election, we are told. The paradox is that the blow-out in government costs comes despite claims by the Government that the outsourcing binge is all about “small government”.
    https://www.michaelwest.com.au/big-four-inquiry-into-government-consultancy-binge-gets-buried/
    Coalition MP Kevin Hogan has left the door open to sitting on the crossbench if Bill Shorten wins the May election, turning him into a key figure in the event of a hung Parliament.
    https://www.canberratimes.com.au/story/6092505/nationals-mp-offers-no-promise-of-party-loyalty-post-election/?cs=14225
    A federal Liberal candidate for the seat of Scullin has apologised for linking same-sex marriage with paedophilia during the marriage equality debate, after being questioned by The Age.
    https://www.theage.com.au/politics/federal/liberal-candidate-linked-same-sex-marriage-to-paedophilia-20190426-p51hij.html
    Homeowners hoping that a widely-predicted cut in the Reserve Bank’s cash interest rate will breathe some life into the property market will be disappointed writes Elizabeth Knight.
    https://www.smh.com.au/business/the-economy/don-t-expect-rate-cuts-to-end-the-housing-slump-20190426-p51hlp.html
    John Wren takes a walk on the campaign trail, uncovering scandals, misinformation and a PHONy or two.
    https://independentaustralia.net/politics/politics-display/wrens-week-watergate-happy-clappers-and-hansons-dinosaurs,12612
    The parent of a man who has been in gaol explains why there is next to no rehabilitation occurring. Unsurprisingly privatisation gets a mention.
    https://www.smh.com.au/national/nsw/wonder-why-prisoners-re-offend-here-s-my-son-s-story-20190426-p51hgy.html
    Jim Bright explains why social media platforms are no place to conduct work disputes.
    https://www.smh.com.au/business/workplace/social-media-platforms-are-no-place-to-conduct-work-disputes-20190423-p51gm7.html
    Some journalists have launched a defence of mainstream reporters suggesting that social media criticism of journalists is coming from a “Trumpian” perspective. This view has also been expressed in a recent column by ABC talking head, Michael Rowland.
    https://independentaustralia.net/business/business-display/twitter-wars-mainstream-media-versus-the-fifth-estate,12614
    What is it about dogs and fleas?
    https://www.smh.com.au/federal-election-2019/photographer-hurt-in-scuffle-at-fraser-anning-press-conference-20190426-p51hig.html
    US Deputy Attorney-General Rod Rosenstein has hit back hard against politicians and the press, and warned that hacking and social media manipulation are “only the tip of the iceberg” when it comes to Russian efforts to influence American elections.
    https://www.smh.com.au/world/north-america/rod-rosenstein-fires-back-at-critics-over-mueller-report-20190426-p51hk3.html
    Anne Summers looks at the choices facing the US Democrats as they commence the convoluted process to select a presidential candidate.
    https://www.smh.com.au/business/the-economy/don-t-expect-rate-cuts-to-end-the-housing-slump-20190426-p51hlp.html
    The London Telegraph opines that Trump has just taken the biggest economic gamble of his presidency.
    https://www.smh.com.au/business/markets/trump-has-just-taken-the-biggest-economic-gamble-of-his-presidency-20190426-p51hdi.html
    Trump has announced that the US will withdraw its support for a United Nations treaty regulating the multibillion-dollar global arms trade.
    https://www.theguardian.com/us-news/2019/apr/26/trump-nra-united-nations-arms-treaty-gun-control
    Of all people Fox News senior judicial analyst Andrew Napolitano has argued that Donald Trump did obstruct justice, with “unlawful, defenceless and condemnable” behaviour related to the investigation of Russian interference in the 2016 election.
    https://www.theguardian.com/media/2019/apr/26/fox-news-andrew-napolitano-trump-obstruct-justice-op-ed
    This upstart doctor has earned nomination for “Arsehole of the Week”.
    https://www.theguardian.com/australia-news/2019/apr/26/doctor-suspended-over-remarks-also-shared-patient-x-rays-online

    Cartoon Corner

    Alan Moir takes us to the election battleground.

    A bit of ying and yang from David Rowe.

    Andrew Dyson at the polling booths.

    Matt Davison on infrastructure promises.

    Zanetti on Palmer’s preference deal.

    I think Sean Leahy is on the money with this one.

    Jon Kudelka looks ahead to the possible MPC debate.
    https://cdn.newsapi.com.au/image/v1/4eb9f40c2385e8286650f364e2828076?width=1024

    From the US





  16. Media likely loves the seat polling as it adds to the sense of uncertainty.

    The fact that most people thinks this election is a shoe in makes for a boring election as far as they are concerned. Not good for the mastheads – they NEED controversy to sell dead trees and generate clicks.

    So the more “noise” that can be created by these polls the better as far as they are concerned.

    And even though many know better, that uncertainty (even if it is unfounded) makes people nervous. Which stables like Limited News all the happier as they feel like they still have “the power”.

  17. I hope Bill doesn’t think giving 50bn bucks to MONA is going to save Braddon…
    despite it being one of the state’s greatest tourist asserts most regular punters think it’s a freakfest for elites….

Leave a Reply

Your email address will not be published. Required fields are marked *