Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

  1. Ven
    says:
    Friday, April 26, 2019 at 1:12 pm
    C@tmomma@12:07pm
    According to Firefox he/she was once an ALP voter/member.
    No wonder ALP right and ALP left fought physically until 80s so that there were serious injuries to people. All those ALP Right people must have been from DLP (which was disbanded), who must have infiltrated ALP. Now they have joined the ALP.

  2. So true Tricot! The fact The West managed to give over the causeway bloke’s comments on electric cars a front page says it all really. I hope that very few people pay much attention to the paper these days but even still, their bid this time around is still incredibly irritating. I always vow to ignore it for the goodnod m blood pressure, but will often glance at it as I pick up my coffee before work.

  3. Matthew Canavan
    @mattjcan
    2h2 hours ago

    Great to be in Rocky today for the announcement of the naming of the Adani Arena basketball stadium. Supporting mining means we support the community.

  4. My American friends are really shocked by the racism, sexism and general right wing nuttery coming from our political and even media classes. I have even told them that mainstream journalists attend conferences held by a White Nationalist Senator (Anning) whose freely associates with Neo-Nazis. Although they know our media fails to openly call out Neo-Nazis such as Blair Cottrell.

    Therefore; I am convinced there are only some minor differences between Trumpism and the ideology which has dominated the Abbott-Turnbull-Morrison government.

  5. AntonBruckner11

    Cormann is a SkyNews go to bloke. His new buzz word for both Bill shorten and the Greens is ‘extremists’. Rinse and repeat.

    Janine Perrett on Sky likes Morrison’s latest ‘Cranky Bear’ for Bill. No doubt it will be used constantly by her grossly repulsive sky mate Paul Murray but best to avoid his program altogether.

  6. C@tmomma @ #301 Friday, April 26th, 2019 – 1:27 pm

    Matthew Canavan
    @mattjcan
    2h2 hours ago

    Great to be in Rocky today for the announcement of the naming of the Adani Arena basketball stadium. Supporting mining means we support the community.

    Bullshit. It’s feel good free publicity for Adani.

    Free? Someone royally screwed up then.

    Usually if a corporation wants branding rights to a stadium they have to pay millions of dollars for the privilege.

  7. zoomster
    There are countries overseas who rely on nuclear power.
    Take that away, by denying them access to uranium, and in most cases they’ll turn to coal.

    There are usually more detailed reasons behind ALP decisions, but the media tend to make bald statements so that the detail is lost. Forever.

  8. Martin

    🙂 And yet ‘just over the causeway’ man has ads that go……………..

    Four reasons to buy the Mitsubishi Outlander PHEV and start driving on the new Electric Highway to Margaret River

  9. BH @ #302 Friday, April 26th, 2019 – 1:28 pm

    AntonBruckner11

    Cormann is a SkyNews go to bloke. His new buzz word for both Bill shorten and the Greens is ‘extremists’. Rinse and repeat.

    Janine Perrett on Sky likes Morrison’s latest ‘Cranky Bear’ for Bill. No doubt it will be used constantly by her grossly repulsive sky mate Paul Murray but best to avoid his program altogether.

    Thanks, BH. I don’t watch TV anymore, so I get a bit ignorant!

  10. Looked to me like the crew didnt get footage of the earlier set-to so chased him down to incite some more.

    Newscorp news crews wouldnt do that, would they?

  11. Great to be in Rocky today for the announcement of the naming of the Adani Arena basketball stadium. Supporting mining means we support the community.
    ___
    Exactly what you’d expect to hear from a conflicted dill like Canavan!

  12. yabba @ #308 Friday, April 26th, 2019 – 1:35 pm

    sprocket_ @ #292 Friday, April 26th, 2019 – 1:11 pm

    Here is some footage of Anning’s thug – the cameraman certainly protects the reporter, and she has go also..

    https://mobile.twitter.com/cvbyllaardt/status/1121611066444640256/video/1

    The reporter was the tall one in black. The shorter lady with the bag was with the attacker and was supporting him There is a longer Ch 9 video..

    Bit hard to fight with a $10,000 camera slung around your neck

  13. lizzie says:
    Friday, April 26, 2019 at 1:24 pm

    Matthew Canavan
    @mattjcan
    2h2 hours ago

    Great to be in Rocky today for the announcement of the naming of the Adani Arena basketball stadium. Supporting mining means we support the community.

    The point is that Adani is not just a coal mining company.

    It also has interests in solar farms to name one.

    Of course the Minister For Mining wouldn’t want to promote that. 🙂

  14. zoomster @ #289 Friday, April 26th, 2019 – 1:05 pm

    Take that away, by denying them access to uranium, and in most cases they’ll turn to coal.

    Skeptical about that. A nation that’s technologically advanced enough to have a nuclear industry is also advanced enough to have a renewables industry, and would well know that coal is dead.

  15. frednk – I though exactly the same thing, had he wanted Dylan might well have given the young smartarse thug a lesson or two.

  16. A reminder of what sort of ratings SkyFoxNews gets – as a benchmark, Insiders gets up to 700,000 and MAFs topped out at 1,960,000 viewers..

  17. Ven says:
    Friday, April 26, 2019 at 1:12 pm
    C@tmomma@12:07pm
    According to Firefox he/she was once an ALP voter/member.
    No wonder ALP right and ALP left fought physically until 80s so that there were serious injuries to people. All those ALP Left people must have been from Australian Communists (which was disbanded), who must have infiltrated ALP. Now they have joined the Greens.
    ——————————–

    What utter rot you write. I have been an active member of the ALP for close to 50 years across 2 states and have never seen any physical fight between party members. I have heard of one alleged instance in NSW in the 80s . The ALP has always covered a broad ideological spectrum. There has been a strong left grouping within the party right from its formation, long before the Australian Communist Party (ACP) was formed. In fact many of the original members of the ACP moved from the ALP when the ACP was first formed. Those who like to describe themselves as the left of the Greens came from the various Trotskyist groups that like to set up commercial stalls at community protests. It constantly amuses me how a middle class trendy elitist mob like the Greens think they are left wing.

  18. During this (deliberate?) poll free period, media proprietors are attempting to manipulate

    In the 9 Network (Costello) we see a headline “Critics slam plan for mega-lane Eastern Freeway”

    The critics are the Liberal Opposition and 2 Liberal aligned Councils (Manningham and Booroondara)- plus an “expert” who refers to the project being “designed around the car” instead of public transport

    What is not mentioned is that this project (and others including public transport works including the Metro Tunnel 2) were taken to the most recent State election, last November where the ALP government was returned in a landslide 58-42 result

    I have no doubt the absence of polling is deliberate as is media reporting (including the ABC unfortunately) in the absence of polling

    Trying to establish a narrative to their design and sought outcome

    So a fraud on the Australian population at large aided and abetted by the current “government”

    We also have a total and absolute neglect of stagflation which the Nation heads to with Zero inflation and of the GDP being at 0.2% and possibly now negative

    The 10 Year Bond Yield sits at 1.788, down a further 0.67% today, off an intraday 1.782

    This is the 10 Year Bond Yield

    The Australian Public are being taken as fools of the first order by this government and its media (including the ABC where you once got better)

    I trust the election result delivers as it has in Queensland, West Australia, the Northern Territory and Victoria

    In spite of biased media coverage

  19. Simon² Katich® @ #296 Friday, April 26th, 2019 – 1:21 pm

    Here is some footage of Anning’s thug – the cameraman certainly protects the reporter, and she has go also..

    I didnt see that in the footage. Looks like the guy was walking away and the cameraman and journo were in pursuit. The journo got our her phone to record just as the cameraman was approaching the alleged Anning supporter. Then the violence.

    Perhaps the verbal aggression was from earlier.

    The journo is the taller lady in black jumpsuit with sunnies. The photographer runs past the journo abuser to get a photo of him. The abusive one attacks the photographer. The short lady, who is with the abusive one attempts to restrain the photographer. The reporter retreats out of right frame.

    This little vignette is a classic example of how people (including you, SK) can jump to entirely incorrect conclusions, based on preconceptions, and misidentification.

  20. Scotty and Bill will get more eyeballs on 7TWO than Sky, but not many..

    “One of Seven’s four multi-channels has achieved a higher share than SBS, with 7Two capturing 4.6% of total audience.

    The multi-channel’s two most popular shows were Father Brown, with 196,000 metro viewers and Murdoch Mysteries, which 143,000 metro viewers, boosting the channel’s share ahead of SBS’ mere 4.3%.”

    https://mumbrella.com.au/7two-achieves-higher-share-than-sbs-560068

  21. So by Labor agreeing to do Leaders Debates on niche TV outlets, only the motivated will find a way to view. It seems like a deliberate ploy from the front runner to not give the mug an even break.

  22. Re Sprocket @1:47 PM.

    Adding up numbers there, Peppa Pig, a well known Communist, with 102,000 viewers across 3 time slots , outrates the Far Right talking heads by about 2 or 3 to 1.

  23. Afternoon folks.

    Saw Shorten’s presser – very strong …. but what will the scribes report? That is the effin question.

    Heard a little of Morrison’s hysteria …. sigh – scribes will say “good copy”

  24. Peter Stanton @ #318 Friday, April 26th, 2019 – 1:47 pm

    Ven says:
    Friday, April 26, 2019 at 1:12 pm
    C@tmomma@12:07pm
    According to Firefox he/she was once an ALP voter/member.
    No wonder ALP right and ALP left fought physically until 80s so that there were serious injuries to people. All those ALP Left people must have been from Australian Communists (which was disbanded), who must have infiltrated ALP. Now they have joined the Greens.
    ——————————–

    What utter rot you write. I have been an active member of the ALP for close to 50 years across 2 states and have never seen any physical fight between party members. I have heard of one alleged instance in NSW in the 80s . The ALP has always covered a broad ideological spectrum. There has been a strong left grouping within the party right from its formation, long before the Australian Communist Party (ACP) was formed. In fact many of the original members of the ACP moved from the ALP when the ACP was first formed. Those who like to describe themselves as the left of the Greens came from the various Trotskyist groups that like to set up commercial stalls at community protests. It constantly amuses me how a middle class trendy elitist mob like the Greens think they are left wing.

    http://netk.net.au/Whitton/Worms26.asp

    Baldwin was bashed in the kitchen of his house early in the morning of Thursday 17 July 1980. He was taken to hospital with a fracture running down the middle of his face and gashes to his head that needed fifty stitches. Meissner and Domican were interviewed by police; both had alibis. The left embarked on another vigorous recruiting campaign. The right claimed the left were stacking branches; the left replied that at least they were providing five bodies, not forged signatures.

  25. C@t

    Not revisionist
    At the time, Labor people here were fighting tooth and nail over who to support, Gillard or Rudd.

    The loss in 2013 was entirely self-inflicted.

    “The Greens went MIA when it came to supporting and defending Julia Gillard”
    This is just a made up line.

    Sure, it may make you feel comfortable to blame the Greens, but the Rudd-Gillard wars that played out here were a microcosm of the larger dispute within the Labor party at the time.

    Rudd, chickened out and then sniped from the sidelines. Much as Abbott did to Turnbull.

  26. You’re welcome, antonbruckner11

    There’s some scarey stuff relating to politics on TV atm. I thought Abbott/Credlin held the record for numbing the masses but Morrison is a whole new game. No wonder he was sacked from the only real jobs he had.

  27. Monetary policy is a steering wheel that isn’t connected to anything. The central bank should simply set the overnight interbank lending rate at zero and leave it there. The central bank’s focus should be on running the payments system and enforcing strong underwriting standards. It does not have the capacity to make a positive difference to employment and price stability. That is the job of fiscal policy.

    The evidence of the effects of monetary policy is that short of jacking up interest rates sky high to induce a recession, as Paul Volcker of the Federal Reserve did to the US economy in the early 1980s, monetary policy has negligible effects on business decisions. Firms make investment decisions on the basis of profit expectations. The cost of credit is a relatively minor consideration for them. If wages are stagnating, causing consumer demand to stagnate, even zero interest rates won’t induce firms to borrow more and produce more.

    Only fiscal expansion can increase the net financial assets of the non-government sector. All that monetary policy can accomplish is a wealth portfolio reshuffle.

  28. This little vignette is a classic example of how people (including you, SK) can jump to entirely incorrect conclusions, based on preconceptions, and misidentification.

    I didnt misidentify anyone. It was obvious who the reporter was just from the clothing let alone that I had already seen other footage.

    Did you see the reporter hold up her phone to record the cameraman about to ‘take a photo’ of the departing Anning supporter? Why did you think she was doing that?

    I do not doubt that this so called Anning supporter has said or done something to create this situation. But that footage shows he was leaving the scene and they were chasing him before the recorded violence broke out. I suggest they were chasing him to incite him so they could get some juicy pics.

  29. Steve777 says:
    Friday, April 26, 2019 at 1:55 pm

    Re Sprocket @1:47 PM.

    Adding up numbers there, Peppa Pig, a well known Communist, with 102,000 viewers across 3 time slots , outrates the Far Right talking heads by about 2 or 3 to 1.

    No wonder young voters lean left. 😆

  30. Is MAFS still going?
    No. One of them had a terminal STD, so they’re all dead now.

    Giggles.
    My lot are looking forward to MasterChef returning. I am worried it is a gateway drug to Survivor then MAFS…. then The Voice.

  31. A clarification to my previous post – watching the video of the Anning supporter, it looks like he assaulted the male photographer, not the female journalist . The Anning supporter did direct sexist and abusive comments at the female journalist.

  32. sprocket_ says:
    Friday, April 26, 2019 at 2:02 pm

    Scotty Stunt for the Day ™ is at a moulding works in Townsville – and the money photo is….

    Morrison’s prime ministership up in flames!

    or
    Dancing with the Devil!
    Preference deal with Palmer.

  33. The way reporters & cameramen behave these days it’s a wonder more of them don’t get punched. They actively provoke people looking for ‘newsworthy’ reactions.

  34. Shorten did say – let us see how many questions the pm answers in the first two scheduled debates.

    Like this approach, Shorten is the leader. ….Morrison the scrambler

  35. The Toorak Toff @ #265 Friday, April 26th, 2019 – 12:23 pm

    Heard Greg Jennet say on ABC Radio words to the effect that the Watergate issue had emerged over Easter when people weren’t focussed and would drop off the radar.

    Isn’t his bloody job to apply the blowtorch to corruption in government?

    You are confusing him with a journalist.

  36. Yep Adrian…. the ABC notmuch better at this stage, they just stated that Shorten has refused debates on the ABC and nine.

    This is not the case and not what Shorten has said.

Leave a Reply

Your email address will not be published. Required fields are marked *