Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

Comments Page 16 of 17
1 … 15 16 17

  2. @jackol

    I thought as much. THAT is the very best you could respond to about Get UP??!! THAT is what you carried out of my comment? How evasive is that??

    “For sure. I mean it takes a lot of continuous effort on the part of the ‘coalition’ to ensure that we do absolutely nothing or go backwards.”

    Is that the best anyone can do to support GET UP with that advertisement??

    Anyone with any fibre of conscience left ought to be eating crow and condemning GET UP. Anything less is tacit support for this grubby organisation.

  4. Labor-Greens wars. Check

    Bemused causing ructions. Check

    Labor supporters limbering up to tear holes in each other. Check

    Just a normal Friday night on PB. Just waiting for Kezza to get pissed and pick a fight with everyone and we’ll have the full set.

    Mind you, Wayne having a meltdown and threatening litigation against all and sundry is a new one.

    Dog, we need some new polling.

  5. Can I just say as the survivor of a heart attack on Friday 13th March 2015 @ 4.30am, don’t accept indigestion and heart burn as normal if it doesn’t go away. I didn’t realise it was angina and about 15 months later the big event occurred. Be active (I was a regular gym goer) and had no other signs other than heartburn/indigestion – cholesterol was normal, blood pressure slight elevated but not high, non-smoker. Stress does play a part. #secondchance

  6. Rob Oakeshott
    ‏ @RobOakeshott1
    18s18 seconds ago

    Rob Oakeshott Retweeted

    DebbieSpillane

    happy to report we are at 973 volunteers/supporters. I think a t-shirt/sticker/corflute bonus pack for lucky 1000. What more could anyone want this fashion season….:). #cowpervotes

  7. Yes I agree… New polling would be nice.

    This should be an interesting election… but it really isn’t yet.

    Sadly Palmer, the buffoon will control too many idiots with wayyyy too much money. Imagine what we could do with the combined broadcast advertising spending of Palmer, Libs, ALP, GET UP, ACTU and Greens… Hundreds of millions….

    Maybe ALL paid advertising in Australia for elections ought to be banned.

  12. Well said Kirky. I myself am finally getting around to getting a few aches and twinges checked out. Probably just too many years of heavy lifting without due care and attention, but age, weight, current high stress and a certain amount of family history have managed to persuade me to be cautious.

  13. The value of Palmer to labor is not in regional Queensland although I think his influence in Herbert etc is being over rated by the MSM. The value to labor is in the rest of the country.

    It does not matter if labor spoke to him or not. Morrison and Palmer are joined at the hip now.

    Not a good look. Just think if the Tories come out with bullshit of how Shorten short changed workers 15 to 20 years ago. Palmer owes his workers $70 million and Morrison is giving him preferences knowing Palmer has fucked his workers over.

    Great look Scott.

  14. Kirky @ #752 Friday, April 26th, 2019 – 9:31 pm

    Can I just say as the survivor of a heart attack on Friday 13th March 2015 @ 4.30am, don’t accept indigestion and heart burn as normal if it doesn’t go away. I didn’t realise it was angina and about 15 months later the big event occurred. Be active (I was a regular gym goer) and had no other signs other than heartburn/indigestion – cholesterol was normal, blood pressure slight elevated but not high, non-smoker. Stress does play a part. #secondchance

    Now you’ve got me worried!

    (makes mental ‘worried well’ note to go to the doctor next week! 😯 )

  15. Maybe ALL paid advertising in Australia for elections ought to be banned.

    The Hawke government pretty much tried that. Reasonably sure the other side was opposed. Certainly the High Court was.

  20. Rex Douglas@9:23pm
    What will last till the end of this campaign is preference deals between LNP, PALMER, LIBS and ON.
    What is your opinion on those preference deals?

  22. I think it is a bit early to judge if the Palmer spend is value for money or not.

    All we have so far as evidence is four seat polls. Nothing else.

    We will only know after the election how effective his spend has been.

    Morrison is really gambling on a unknown with his decision to preference Palmer. Aim to save two or three seats in Queensland v the possibility of pissing off a large chunk of the rest of the country.

    Go Scotty.

    BTW, I think anyone sho7ld be able to spend what they like in a election campaign. Just play the cards you are handed.

  25. imacca @ #776 Friday, April 26th, 2019 – 9:53 pm

    https://www.smh.com.au/federal-election-2019/bill-shorten-flags-third-election-debate-in-row-with-scott-morrison-over-scrutiny-20190426-p51hf3.html

    Seeyah on that one ScoMo, and raise you …. 🙂

    Shorten pretty good at the timing of these things wot?? 🙂

    So depressing these debates will be.

    Morrison and Shorten the two most uninspiring major party leaders since…. well not that long ago Abbott and Rudd.

    Ughh…

  26. Regarding the “Liberals” approach to addressing climate change, their two factions disagree:

    – The Right says that climate change is happening, but that we don’t have to actually do anything about it beyond slinging money to mates to plant a few trees.

    -The Far Right denies there’s a problem and says that meteorologists are fudging the numbers.

  31. National Press Club chief executive Maurice Reilly said: “The National Press Club has made a submission to hold a leaders’ debate. The club was the traditional venue for the leaders’ debate in previous elections, and we did the first one back in 1984.

    “In the absence of a debate commission, which is well-established in other countries like the US, the National Press Club is the most trusted neutral venue to hold the debate because it would provide it to every network in prime time.”,

    I position i agree with.

  32. Wow, I didn’t know that Scott Morrison walked out of the chamber so that he couldn’t vote for the SSM Plebiscite!?!

    I didn’t either and didn’t remark upon it because I figured it was just me working fulltime and not being across the detail at the time.

  33. C@tmomma@9:39pm
    I did not have even heartburn/ indigestion.
    I was over weight. My cholesterol was normal with a slightly elevated BP, for which I was taking pills.
    I had a stressful period. Then on 27th Jan, 2019 I had pain through out my left arm and congestion in my chest, which was diagnosed as mild heart attack. It was because one of the arteries was 90% blocked. I was fortunate that it was 90% blocked. Hence, my heart was not damaged. The blockage was cleared and stent was inserted. Now, I changed my diet and go on daily walks.

  34. the National Press Club is the most trusted neutral venue

    It is also heavily dominated by news corp which is running a propaganda rather than a journalism campaign.

  35. Wow, I didn’t know that Scott Morrison walked out of the chamber so that he couldn’t vote for the SSM Plebiscite!?!

    Wasn’t that how he earned his nickname, as in NohomoScoMo shortened to ScoMo.

  36. “It is also heavily dominated by news corp which is running a propaganda rather than a journalism campaign.”

    But,…….. i think NPC is an environment where they will be less inclined to go as heavily partisan as they would like. Really, i am not particularly concerned at Shortens ability to handle all comers in the debate / town hall format. He has had a LOT of practice and i’ll put that up against ScoMo’s upping the decibels any time.

  37. “Wow, I didn’t know that Scott Morrison walked out of the chamber so that he couldn’t vote for the SSM Plebiscite!?!”

    Really? Publicise that – that should kill him stone dead in the polls.

    (I mean – even more stone dead than presently)

  38. imacca @ #774 Friday, April 26th, 2019 – 9:23 pm

    https://www.smh.com.au/federal-election-2019/bill-shorten-flags-third-election-debate-in-row-with-scott-morrison-over-scrutiny-20190426-p51hf3.html

    Seeyah on that one ScoMo, and raise you …. 🙂

    Shorten pretty good at the timing of these things wot?? 🙂

    I wonder why The Press Club has been given the cold shoulder? Maybe the ALP is not too impressed by the media and is not going to support their institution? Or perhaps the open question segment is more likely to be a footrub for Morrison and a branding iron to the toes for Shorten?

  40. As far as I can tell there’s not been much discussion here about this but as a Labor voter I’m a bit pissed about them backing fracking in the NT.

  42. I think a NPC debate would lead to partisan questions to Bill from the lib-loving journos, particularly using the word TAX as much as possible. At the same time they will probably ask Morrison about his dog or cat.

    I am happy to trust the ALP tactics mob on this one.

  44. As far as I can tell there’s not been much discussion here about this but as a Labor voter I’m a bit pissed about them backing fracking in the NT.

    I can only guess they have some idea what fracking is, and you well, don’t.

  45. “particularly using the word TAX as much as possible. ”

    I suspect the journo’s will go the route of “wots the exact $ cost of…….”.

    Its the most likely way to trip someone up and have the “election losing gaffe” gotcha moment so dear to some in the press.

  46. Confessions @ #780 Friday, April 26th, 2019 – 10:05 pm

    Wow, I didn’t know that Scott Morrison walked out of the chamber so that he couldn’t vote for the SSM Plebiscite!?!

    I didn’t either and didn’t remark upon it because I figured it was just me working fulltime and not being across the detail at the time.

    It wasn’t publicised at all I don’t think. Which would be exactly how Scott Morrison would have preferred it to be. One of those ‘Canberra Bubble’ things he doesn’t like talking about. 😐

  48. “I did not know that. I thought it was between the ABC and one of the evil channels.”

    ScoMo wanted it to be. 🙁 Shorten has re-framed it. 🙂

    Shorten obviously an annoying man who does that kind of thing AND JUST WONT GET WITH THE PROJECT!!!!. 🙂

  49. I think the NT proposition is more along the lines of a Santos SA gas extraction project, rather than the pock-marking type that fracking represents. But I could be wrong. I often am. 🙂

Comments Page 16 of 17
1 … 15 16 17

Leave a Reply

Your email address will not be published. Required fields are marked *