Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

  1. Busy morning, late to PB. Good morning if I type fast enough. G’day if not. 🙂

    Thoughts on Morrison Shorten debate(s): The benefit of a debate for the challenger is that some of the leader-gloss reflects on them. If you consider the reflected ‘glory’ you get posing with soldiers and fire-fighters, Morrison knows this well. Perhaps Morrison wants some of what Shorten has.

    Thoughts on Palmer: Part of the UAP strategy with blanket advertising is to appear to be a major player. Paper Tiger springs to mind.

  2. Bowen has said he is looking forward to debating Fraudenberg.
    That would be worth listening to – depending on the moderator of course.

  3. Thug Fraser Anning brings his goons with him…

    Some of the supporters took a dislike to the line of questioning reporter @ElizaJBarr took with Fraser Anning. When the conference ended they proceeded to follow her out of the park. sbs.com.au/news/man-arres… via @SBSNews

    This is disgusting. Fraser Anning supporters don’t like @ElizaJBarr questioning and follow her out of Cronulla park. Man arrested after alleged assault of photographer with her

  4. The Country Party aka National will have fewer seats than the independents after May 18 thanks to Watergate, General corruption & Scott’s Palmer deal

  5. Its impressive that even after the examples of Trump, and Theresa May (who did the same closely managed vacuous campaign style to universal journalistic acclaim until other factors saw it unravel), our MSM journos have still been sucked in by Morrison down to fawning over how easily Morrison avoids questions on corruption in his ministry.

    And they wonder why people don’t treat them seriously.

    On the subject of preference deals, Kevin Bonham had something to say on Twitter about the discipline of ON voters : https://twitter.com/kevinbonham/status/1121426247102259200

    29 ( of 7613 votes) followed their how to vote card. 99 made it to ‘2’. 14 parties, and no ‘2’ vote at all were more common than their preferred two vote. But we need more discussion of the value of these deals. It fills column inches.

    Also William, thanks for the analysis of seat polls. Good to keep it in mind.

    (You may now all return to your regular discussion of who was in the wrong in 2009)

  6. Guardian

    The ALP’s First Nations justice package has been welcomed by Change the Record, the coalition of leading Aboriginal and Torres Strait Islander, human rights and community organisations.

    The announcement includes , over four years, $40m for Aboriginal and Torres Strait Islander Legal Services, $4m for their peak body NATSILS, $21.5m to family violence prevention legal services, and $20 million for refuges and safe houses.

    “Aboriginal and Torres Strait Islander women are the fastest growing prison population, and most of these women are victims of violence,” Change the Record co-chair Cheryl Axleby said.

    “We welcome this strong commitment, as critical community-controlled services are needed to end these injustices.”

    But Amnesty says the ALP’s policy is a “missed opportunity”.

  7. The Greens are not to blame for Labor destroying two decent PMs in one go and handing power to Abbott.

    Oh. Yes. They. Are. Very much so.

    The Greens truly are the Lady Macbeth of politics.

  8. Michelle Landry is intending to preference the Anning candidate above Labor.

    Morrison said it was a qld LNP decision.

  9. But Amnesty says the ALP’s policy is a “missed opportunity”.

    And I imagine Amnesty’s spokesperson is a Whitefella, putting politics first and Indigenous People second.

    Frankly, if The Greens and wahmbulance chasers like Amnesty haven’t realised it yet, it’s quite obvious to me that Labor are announcing policies that they think will get through the Senate via the more moderate members of that place. Especially as The Greens have made it plain they intend to be sanctimonious spoilers again.

  10. Some of the supporters took a dislike to the line of questioning reporter @ElizaJBarr took with Fraser Anning. When the conference ended they proceeded to follow her out of the park. sbs.com.au/news/man-arres… via @SBSNews

    The police were holding his hand as he smiled his way to the station. He should have got the egg boy treatment – jumped on and scuffed up a little.

    Any leader of a party that preferences this goon should get the same treatment from voters – metaphorically of course.

  11. Heard Greg Jennet say on ABC Radio words to the effect that the Watergate issue had emerged over Easter when people weren’t focussed and would drop off the radar.

    Isn’t his bloody job to apply the blowtorch to corruption in government?

  12. The debate on wage exploitation & penalty rate should be interesting following Reserve Bank analysis

    Ewin Hannan
    Sunday penalty rate cuts have not created one new job or prompted business to give workers any extra hours, admits small business lobby. This was the business rationale for the cuts. Now says whole debate has been a “waste of time”. #AusVotes19

  13. Gee who’d a thunk it, the effing the sainted small business peasants just pocketed the money they ‘took’ from their employees.

  14. I don’t recall ever seeing morrison debate, can someone remind me of when and who he actually debated with in a political forum.
    I know that Mr Shorten has appeared at numerous ‘town hall’ meetings and would therefore be well prepared to answer unscripted questions but as to morrison, never seen him holding a ‘town hall’ meeting, never seen him debate. Only ever seen him speaking (poorly) to his trained media and any difficult questions sees him wandering off.

  15. Guardian

    Q to Shorten: Could you just clear up the situation with Clive Palmer? You said a couple of things about him in this press conference. Did Michael O’Connor or anybody else negotiate with Clive Palmer on your behalf to share preferences with Labor?

    “There’ve been no formal negotiations.”

    (Which does not rule out O’Connor having quiet conversations.)

    Q. Given your views on Clive Palmer, can you then therefore rule out doing any deal with his party to pass legislation through the Senate if you’re elected?

    “Well, my aim is to get as many of our people elected first.”

    (That sounds like they are talking.)

  16. It’s amazing C@t
    “Oh. Yes. They. Are. Very much so.”

    Labor will claim the Greens have no power, yet blame them for the destruction of two PMs and the arrival of Abbott.

    It’s actually bloody stupid.
    Labor were the cause of their own demise. Rudd wouldn’t negotiate with the Greens on the CPRS, and then chickened out on going to a double dissolution. He lost because he chickened out, then got stabbed by his own party because the PARTY couldn’t work with him.
    Gillard messed up with the Carbon Tax comment, and then Tony won because there were enough misogynists to boot Gillard out.

    The Greens worked well with Gillard, and helped pass a lot of fine legislation. That Rudd constantly sniped her is not the Greens fault.

    It may make you feel better to blame someone else, but your logic is shit.

  17. I suspect the non existence of Senior Cabinet Minister Peter Dutton on the campaign trail is reflective of the focus group and private polling results

  18. Anning deliberately provokes violence.

    Police are questioning a 19-year-old man after a photographer was punched at a news conference called by Senator Fraser Anning at the scene of the Cronulla riots.

    Senator Anning chose the provocative location to introduce NSW candidates for his party, Fraser Anning’s Conservative National Party.

    Anning supporters heckled journalists during the news conference, then one made a derogatory remark to local reporter Eliza Barr.

    When News Corp photographer Dylan Robinson attempted to take photos of the supporter, the man lunged at him, throwing punches.

    Mr Robinson’s shirt was badly torn in the altercation.

  19. Sam Maiden tweets..

    Called Eliza to see if she’s OK and she’s fine. Pretty scary though. He said some predictably sexist and offensive things to her too. Really disgusting.

  20. I sorta know a news cameraman. Generally not to be messed with. And they will often send the rougher and tougher ones out with the petite journo types.

  21. Astrobleme,
    Frankly I don’t give a damn what sort of revisionist interpretation of history you want to put about here.

    Sure, Kevin Rudd sniped at Julia Gillard, sure Julia Gillard said, ‘Call it a tax, if you like’. However, what is also the case is that The Greens went MIA when it came to supporting and defending Julia Gillard and Labor from Tony Abbott and the Coalition’s attacks.

    In fact I seem to remember reading at the time that one of the reasons Christine Milne got the elbow was because she wouldn’t go out and advocate forcefully for the Climate Change Action Plan. She left Julia Gillard standing there like a shag on a rock.

    Maybe she was too afraid of going up against Tony Abbott? It’s so much easier to criticise Labor, after all.

  22. but as to morrison, never seen him holding a ‘town hall’ meeting, never seen him debate. Only ever seen him speaking (poorly) to his trained media and any difficult questions sees him wandering off.

    Doesn’t matter. If he gets hit with something he doesn’t know he’ll just lie his ass off as per usual. And the moderator won’t call him on it.

  23. BK says:

    Penny Wong has just said that the Palmer preference deal was a marriage of convenience between an ad man and a con man.

    😆 Now the hard part,working out which one is the ad man and which one is the con man.

  24. This comment (by Lawrie Griffith) on the Guardian campaign blog bears repeating:

    “The elephant in the room with all these L-NP promises is that all the money they ‘promise to spend’ on training, health and so on, will all be done through private providers.

    By using that methodology, the money that Morrison is promising all over the place will actually be used to dismantle government services.

    The taxpayer monies get sucked out sideways by private interests and the supposedly intended recipients get the dregs under increasingly onerous, intrusive and controlling corporate regulation of the citizen.
    This is undemocratic and socially destructive.

    This is a massive issue. Central to the very nature of the society we live in.

    Why is the commentariat not making it part of the election campaign? Is the media going to let this sleight of hand by conservative interests go unchallenged?


  25. The West Australian’s election coverage goes from bad to worse, merely confirming their status as a fully paid up member of the Liberal Party cheer squad.

    This morning they managed to have yet another story about Morrison’s supposed love of WA. Apparently he would “call Perth home if he didn’t live in Sydney”. Truly pathetic from their political editor.

  26. Shorten astutely pointed out that Morrison’s ministers are in witness protection. Environment minister, women’s minister, industrial relations minister. Compared and contrasted the presence and ability of his respective shadows.

    He also pointed out that all Morrison has in the locker is to attack him, because he has nothing positive to say about the country, no policies and wants to distract from the chaos and division in his own party. Pointed to Liberal deals with Palmer and Palmer’s record of chaos; the Nationals deal with ON, Fraser Anning etc.

    Oh, also just saw on the news that a Fraser Anning supporter assaulted a female journalist at a presser this morning.

    Read that former WA Liberal premier Colin Barnett has strongly criticised the preference deal between Liberals and Palmer:

    “Scott Morrison will damage relations with China and risk an electoral backlash if he inks a preference swap deal with controversial businessman Clive Palmer, former WA Liberal premier Colin Barnett said..”


    I note that The West hasn’t reported Mr Barnett’s comments.

  27. There are countries overseas who rely on nuclear power.

    Take that away, by denying them access to uranium, and in most cases they’ll turn to coal.

    Still good for Australian exports, I suppose.

  28. I’ve been trying to find out how the audience for the debate on Monday will be chosen. Just came across this.

    Due to the changing logistics of the election campaign, we have modified the debate structure, and will now be holding it on Monday 29th April at 5pm.
    The audience for the debate will be determined by YouGov. As a result tickets are no longer available.
    The West Australian is proud to bring you the first-ever Perth Leaders Debate in WA.
    You can watch the debate Live from 5pm on Channel 7 WST, and read all about it in The West Australian on Tuesday 30th April.
    Seven West Events


  29. Martin….yep, the West has outdone itself so far in this election campaign.
    That the loveable rogue Scotty would come and live in Perth – if he could – cheers me up no end.
    I think we have enough used car salesmen here already.
    One touts to having been around for 50 years, and recently gave of his opinion that electric cars will be a disaster. That he also has been financial backer of the Liberals in most of that time, and has no qualms selling cheap Chinese imported models does not seem to be any contradiction for him. Anyway, he is “just across the Causeway” as most people in Perth will know.

  30. C@tmomma@12:07pm
    According to Firefox he/she was once an ALP voter/member.
    No wonder ALP right and ALP left fought physically until 80s so that there were serious injuries to people. All those ALP Left people must have been from Australian Communists (which was disbanded), who must have infiltrated ALP. Now they have joined the Greens.

  31. The MSM fights back

    3h3 hours ago

    Hard not to conclude that in grabbing WA debate on Monday on a digital 7 channel & Sky News one, Bill Shorten has decided debates aren’t really his bag so he’s picked two with hardly any ratings only/ pay tv subscribers & then he can say “I’ve done two that’s enough”

  32. Here is some footage of Anning’s thug – the cameraman certainly protects the reporter, and she has go also..

    I didnt see that in the footage. Looks like the guy was walking away and the cameraman and journo were in pursuit. The journo got our her phone to record just as the cameraman was approaching the alleged Anning supporter. Then the violence.

    Perhaps the verbal aggression was from earlier.

Leave a Reply

Your email address will not be published. Required fields are marked *