Wide of the mark

A consideration of whether the poor reputation of seat polling is really deserved (short answer: yes).

Still no new polls, so let’s take a look at some old ones instead. After the 2016 election, I wrote an article for Crikey on the performance of the pollsters, particularly in regard to seat polls, and published here a chart showing the distribution of their errors. After being asked if the findings bore up over the seat polling conducted since, I have now conducted a similar exercise on seat polls conducted since the 2016 federal election, of which I identified 25 conducted in the final fortnight of various state elections and federal by-elections. However, rather than use the two-party results, which have separate issues of their own, I have produced separate results from Labor and Coalition primary votes. These can be found at the bottom of the post.

In the 2016 analysis, I concluded that the polls behaved more like they had a 7% margin of error than the 4% margin theoretically associated with polls sampling 500 to 600 respondents, as is typically the case with seat polls. It turns out that this chimes quite well with the polls conducted since. The mean error for the Coalition was +1.9%, which is to say the average poll had the Coalition that much too high high, while for Labor it was -0.5%. The difference is just inside statistical significance (the p-value on a two-sample t-test coming in at 0.047).

However, this does not mean you can confidently treat any given seat poll as biased to the Coalition, because their record is so erratic that any given poll could fall either way. The charts below record the spread of pollster errors (i.e. their result for a given party’s primary vote minus the actual result) as histograms, with two distribution curves laid over them – a thinner one in black, showing what the curve should theoretically look like with a 4% margin of error, and a thicker one in blue, showing their actual distribution. The lower and flatter the blue curve, the more erratic and unreliable were the results. As such, the charts show seat polls have been particularly wayward in predicting the Coalition primary vote. They have been somewhat nearer the mark with Labor, but still below theoretical expectations. The distributions suggest an effective margin of error for Labor of 6.5%, and for the Coalition of fully 9.5%.

It should be acknowledged, however, that a lot can happen over the last fortnight of an election campaign, and pollsters can always defend an apparent misfire by asserting that the situation changed after the poll was conducted. Perhaps significantly, the two worst performing polls in this analysis only barely fit within the two-week time frame. These were YouGov Galaxy polls from the Victorian “sandbelt” seats of Mordialloc and Frankston at the state election in November last year, crediting Labor with two-party votes of 52% and 51% in seats where the final results were 62.9% and 59.7%. If these cases are removed, the mean Coalition error comes down to +1.1% and the effective margin of error to 8.4%; while for Labor, the mean becames +0.1% and the margin of error 5.3%.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

831 comments on “Wide of the mark”

  1. Faaarrrqqqqqqq! I am going to lose it with our media companies before long.

    Labor releases landmark Women’s Policy today. What do the media reports lead with? Freakin who is willing to debate whom. I mean seriously?

    Debatergate is sure a good reason for voters to choose which party to vote for.

    I’m appalled by the lack of any kind of integrity in our media.

    They snigger that voters are not engaged. That they are ill-informed. And this shit is what they serve up to us?

  2. EGW

    As I said, he may have been in the care or supervision of family members, or his now deceased girlfriend he killed.

    He may have led them to believe he was taking his meds.

    We just dont know.

  3. Victoria @ #454 Friday, April 26th, 2019 – 5:13 pm

    EGW

    As I said, he may have been in the care or supervision of family members, or his now deceased girlfriend he killed.

    He may have led them to believe he was taking his meds.

    We just dont know.

    I am quoting what has been reported.

    I would not be at all surprised if he had been discharged into the care of his girlfriend. Yes, that is speculation.

  4. And we all know that if the Coalition had declined debates the media would have been licking their boots and saying how smart they were.

  5. Bill Shorten will pledge to bring down the soaring cost of daycare for parents on Sunday but will mandate that childcare workers are guaranteed a pay rise under the deal.

    The New Daily understands Labor will confirm the policy over the weekend, in one of the biggest announcements of the 2019 campaign.

    The spending increase will deliver direct Commonwealth funding for wage increases for some of Australia’s most low paid workers in childcare centres across Australia.

    However, it will include a compliance system to ensure that workers get a wage increase and that childcare centre operators do not simply pocket the profits.

    https://thenewdaily.com.au/news/election-2019/2019/04/26/election-2019-labor-pledges-payrise-childcare-workers/?utm_source=Adestra&utm_medium=email&utm_campaign=PM%20Update%2020190426

  6. Jack Aranda says:
    Friday, April 26, 2019 at 4:43 pm

    From the AEC’s FAQs about Voting in Australia:

    The Commonwealth Electoral Act 1918 (Electoral Act) contains specific requirements that must be met by voters in marking their ballot papers to be included in the count for a federal election. When casting a vote it is important that voters follow the instructions specified on the Senate and House of Representatives ballot papers to ensure that their vote is able to be included in the count.

    I think the problem here is that this one paragraph should be two.

    There are things you must do, like using numbers to indicate your preferences not ticks or crosses or letters.

    It then goes on to talk about the specific ballot papers, this should be a new paragraph.

    So,

    The Commonwealth Electoral Act 1918 (Electoral Act) contains specific requirements that must be met by voters in marking their ballot papers to be included in the count for a federal election.

    When casting a vote it is important that voters follow the instructions specified on the Senate and House of Representatives ballot papers to ensure that their vote is able to be included in the count.

    I agree it’s not well written and that first sentence leaves you hanging , expecting more information to follow, but I think inserting the paragraph makes the second part clearer. 🙂

  7. The next Newspoll will have the Great ALP ahead 62-38.
    SlowMo should resign now and let Adam Bandt lead the Fibs to destruction.

  8. EGW @ #435 Friday, April 26th, 2019 – 4:46 pm

    Victoria @ #432 Friday, April 26th, 2019 – 4:42 pm

    EGW

    It’s a difficult one. Many people say they are taking meds, but are not being honest. People take what they say on face value.
    What do you do?

    If they are thought to be at risk of being non-compliant, then they need to be supervised by their local community mental health service.
    De-institutionalisation was supposed to provide for this. It has been a sham.

    EGW,

    I provide in-home, and in nursing home support to older people who have been discharged from mental institutions on the Central Coast, via what is called SMHOPS, administered by our local State Government Mental Health office. We had our monthly meeting this morning. Can I note that the people involved in our local government health services are, without exception, hard working, innovative and admirable.

    In NSW, when a person is released from a mental institution, whether they have been admitted on a voluntary or involuntary basis, the release is overseen by a group within the hospital comprising psychiatrists, social workers, and senior mental health nurses. A visiting nurse will be assigned to the patient, whom they will have met several times while in hospital, and this nurse will visit up to 3 times a week until they are confident that the client is stable and coping, and that necessary support is in place. After that, a patient is ‘discharged’, and, if recommended, people like me and my colleagues take over. Our role is to encourage clients to fully recover, by re-engaging in society, taking up new activities, setting goals and working towards them. Generally we visit weekly, for an hour or two, for four to six weeks, or longer if it is apparent that this will be beneficial. This has been a local initiative, and is now being rolled out elsewhere.

    Regarding medication, it is not possible, outside hospital, to ensure that any client actually takes prescribed medication. In hospital, the nurse will deliver the medication, at the prescribed times, and watch while the patient swallows it. (Drink the whole cup!). Outside the hospital, no such coercion is legally possible.

    The question is vexed. Where does personal freedom and personal responsibility begin and end? The degree of personality change which occurs in some bipolar and schizophrenic clients is difficult to comprehend. Some have supportive and understanding families. Many do not. A HARD problem.

  9. Labor releasing policy after policy.

    LNP have absolutely f-all to offer, except a dodgy tax cut for the rich mates, and a massive record of failure over the last 6 years, including two unexplained PM knifings.

    You do the sums….

  10. Wayne

    Flat and recessionary wages growth (refer the comments of Lowe, Governor of the RBA)

    The Cash Rate at 1.5%, the lowest in history and projected to reduce further from these emergency settings which have maintained since the sub-prime lending crisis, noting the projection of further falls)

    The 10 Year Bond Yield at 1.77%, the lowest ever, and losing value with basis points declines daily.

    House prices in decline, now for 18 months and no stabilization then recovery in sight.

    It has taken the ASX until this week (week ended 26th April), to improve on its 30/06/2018 close (due to currency devaluation and the outlook for interest rates and any wages increases)

    Inflation at ZERO, the very real prospect for deflation and the terminal damage that inflicts (see Japan over the last 30/40 years).

    The most recent quarterly GDP at o.2%, trending down and possibly now negative leading to a recession (2 Negative quarters)

    Labour market conditions, in regards Terms and Conditions of employment on offer for part time and casual employment

    Increasing cost of living, Private Medical Cover premiums, utility costs, home and MV Insurance costs, Council Rates, petrol off the increases in the Global Costings

    Increases in Education costs as identified in the Inflation data.

    Previously part self-funded retirees now fully self-funded due to changes in Pension eligibility rules (part of the right wing “austerity leads to confidence” doctrine)

    Royal Commissions into vital Services, being Aged Care and banking, both beneficiaries of the right wing “the most effective form of regulation is self regulation” doctrine of this dysfunctional government, which heads into an election with Ministers who are resigning with this election (so what “team” are they taking to the electors?)

    Just for starters, before we get to Climate Change, the MDB, the Barrier Reef etc etc etc and the scandals of misappropriation including the misappropriation of the authority of government (and it was such a list as to the number of factors that eventually saw Howard off – read the summary of Hugh Mackay concluding “Few voters will tick all those boxes, but many will tick at least one. And that accumulation of disappointment, disillusionment and disenchantment has turned the political tide

    Wayne, as you may expect from a fully self funded, retired former banker to Corporate Australia over 30 years of a career in banking, I have in the past voted Liberal.

    BUT, that was before Howard as treasurer (where I had personal knowledge because of certain circumstances in the banking industry which involved my responsibilities of the time), then as Leader.

    As Malcolm Fraser said, the Liberal Party is no longer the Liberal Party, and he resigned.

    I hold the front page of The Sydney Morning Herald with the headlines being “Graeme Thorne’s Body Found”

    Then

    “Budget to ease spending. Income and Company Taxes Up; Pensions 5/- Higher” AND “Budget Speech Draws Few Interjections”

    The pm was Menzies, the founder of the Liberal Party.

    The treasurer was Holt.

    The upper marginal tax rate went to 60 cents in the $1-, which rate I was subject to from 1980 (without complaint, so definitely NOT a deterrent to working for and achieving promotion/s)

    Perhaps, just perhaps, you may wish to respond and debate?

    On the FACTS

  11. Margo Kingston
    @margokingston1

    @ramklar
    and 3 others
    it’s not about that for me. Morrison’s people can do his PR shots and post direct to social media. Press Gallery in the social media age should be value adding and if they can’t they should get off the damn bus.

  12. jenauthor @ #451 Friday, April 26th, 2019 – 5:09 pm

    Faaarrrqqqqqqq! I am going to lose it with our media companies before long.

    I’ve cut right back on TV and so called ‘news’ online – for a couple of months.

    Also I’m scrolling etc quite a bit here.

    Guess what ? Life goes on. Let them play their games, I can still stay reasonable well inform without it constantly coming at me.

    I’m probably not the only one?

    PrePolling from Monday – lets see Labor rollout their moves soon.

  13. Jack Aranda @ #441 Friday, April 26th, 2019 – 5:03 pm

    Formatting tip for anyone who uses (blockquote) (where the brackets are angled of course) but doesn’t like the look of the quotes being all in italics (assuming I’m not the only one who’s so fussy). After the blockquote tag do an end-italics tag and then if you want anything emphasised within the quote you can use the usual start and end italics tags. Just tried it in my quote from the AEC above. Also did a start italics before the end-blockquote but not sure if that was necessary.

    Thanks for the tip. (This comment was done with balanced start/end italics within the blockquote.)

  14. Sleeper issue today that bears returning to later is the admission from business groups that penalty rate cuts would create more jobs was a lie. Anyone who could do basic maths knew it but to admit it is a different thing altogether.

  15. Greensborough Growler says:
    Friday, April 26, 2019 at 5:33 pm

    Nats are denying this was Barnaby Joyce.

    Sounds more like Abbott.

  16. You do the sums….

    Our media certainly have. That’s why they’re so insistent on focusing on how many debates Shorten will do and whatever else the Libs decide they should be distracted by…

  17. “They snigger that voters are not engaged ”
    You would have to say that it is the MSM that is not engaged in any purposeful and meaningful way.
    The voters are engaged – and just waiting patiently (or not so in some cases) for May 18th.

  18. yabba @ #464 Friday, April 26th, 2019 – 5:30 pm

    EGW @ #435 Friday, April 26th, 2019 – 4:46 pm

    Victoria @ #432 Friday, April 26th, 2019 – 4:42 pm

    EGW

    It’s a difficult one. Many people say they are taking meds, but are not being honest. People take what they say on face value.
    What do you do?

    If they are thought to be at risk of being non-compliant, then they need to be supervised by their local community mental health service.
    De-institutionalisation was supposed to provide for this. It has been a sham.

    EGW,

    I provide in-home, and in nursing home support to older people who have been discharged from mental institutions on the Central Coast, via what is called SMHOPS, administered by our local State Government Mental Health office. We had our monthly meeting this morning. Can I note that the people involved in our local government health services are, without exception, hard working, innovative and admirable.

    In NSW, when a person is released from a mental institution, whether they have been admitted on a voluntary or involuntary basis, the release is overseen by a group within the hospital comprising psychiatrists, social workers, and senior mental health nurses. A visiting nurse will be assigned to the patient, whom they will have met several times while in hospital, and this nurse will visit up to 3 times a week until they are confident that the client is stable and coping, and that necessary support is in place. After that, a patient is ‘discharged’, and, if recommended, people like me and my colleagues take over. Our role is to encourage clients to fully recover, by re-engaging in society, taking up new activities, setting goals and working towards them. Generally we visit weekly, for an hour or two, for four to six weeks, or longer if it is apparent that this will be beneficial. This has been a local initiative, and is now being rolled out elsewhere.

    Regarding medication, it is not possible, outside hospital, to ensure that any client actually takes prescribed medication. In hospital, the nurse will deliver the medication, at the prescribed times, and watch while the patient swallows it. (Drink the whole cup!). Outside the hospital, no such coercion is legally possible.

    The question is vexed. Where does personal freedom and personal responsibility begin and end? The degree of personality change which occurs in some bipolar and schizophrenic clients is difficult to comprehend. Some have supportive and understanding families. Many do not. A HARD problem.

    What you describe is much better than I have observed over the years in Victoria. (Cue zoomster to jump in and say all is perfect.)

    The staff may be mainly good, but there are simply not enough of them to cover the case load.

    There are few ‘mental institutions’ as such, most in-patient services are dealt with in a psych ward which is part of a general hospital. Premature discharges are the norm as beds are always required for someone even sicker. Proper discharge plans are rarely made.

    The Vic Mental Health Act only covers Involuntary Patients, the section dealing with Voluntary Patients was removed by Kennett.

    I could say much more but this gives some idea of the problems.

  19. I think Jane Norman, ABC reporter travelling with the Scotty Express, has developed Stockholm Syndrome.

    Also, the Scotty mystery tour has landed in Dubbo for an overnight stay. Dubbo you say? What margin is that on?

  20. Bill Shorten will pledge to bring down the soaring cost of daycare for parents on Sunday but will mandate that childcare workers are guaranteed a pay rise under the deal.

    ‘If you wish to be a success in the world, promise everything, deliver nothing.’ Napoleon Bonaparte

  21. Jack Aranda, re your italics tip. I discovered that WordPress checks for and removes unbalanced italics tags. (Maybe it does this for all tags to protect the rest of the blog.) I also discovered that a start-italics tag immediately following the start-blockquote tag cancelled italics and the next end-italics tag started italics. It was as if start and end tags had reversed meaning. Shaking my head, but there’s a logic to it I suppose.

  22. lefty e @ #465 Friday, April 26th, 2019 – 5:30 pm

    Labor releasing policy after policy.

    LNP have absolutely f-all to offer, except a dodgy tax cut for the rich mates, and a massive record of failure over the last 6 years, including two unexplained PM knifings.

    You do the sums….

    Early next week, Bill’s going to have a huge anti-corruption commission launch. Big dollars attached; lots of staff. It will be the knock-out blow.

  23. Bevan Shields
    4m4 minutes ago

    Bill Shorten has proposed a third debate at the National Press Club for May 8, with a journo representative from Nine and the ABC

  24. nath @ #482 Friday, April 26th, 2019 – 5:50 pm

    Bill Shorten will pledge to bring down the soaring cost of daycare for parents on Sunday but will mandate that childcare workers are guaranteed a pay rise under the deal.

    ‘If you wish to be a success in the world, promise everything, deliver nothing.’ Napoleon Bonaparte

    More BS from you Nath.

    But there are problems in increasing staffing ratios, qualifications and other standards and simultaneously expecting costs not to rise.

    The only way this can happen is if Government provides substantial subsidies.

  25. Bill Shorten has proposed a third debate at the National Press Club for May 8, with a journo representative from Nine and the ABC #auspol  #ausvotes 

    Good work Bill, middle of the day – nobody will be watching!

  26. Re mental illness, latest figures on numbers affected by psychotic illness in Australia who have come into treatment is about 80,000, of whom about 50% are schizophrenic. There are of the order of ten bad schizophrenic attacks, leading to serious injury or death, each year. Nobody involved knows how to pick which of the 40,000 unfortunates will be the ones who become dangerous, although those who are also drug users are more prone to.

    The proper scientific studies that have been carried out indicate that psychotic disease results in an incidence of violent offending of between 1.2 and 2 times the ‘normal’ population.In other words, just like the rest of us, only a very small proportion of those with a psychotic illness will ever deliberately hurt someone. Supposedly sane men and women are responsible for more than 50 times more deaths each year than are mental patients (suicides excluded).

  27. Not surprising that Shorten is trying to avoid television debates with Morrison, he saw what a disaster the Gladys vs Daley debate was for Labor and he’s worried the same will happen to him. Shorten knows he will lose to Morrison badly.

  28. I wonder if Morrison has realised he has just had his pants pulled down by Shorten.

    Morrison stamping his foot and complaining and Shorten delivers.

    A bit of role reversal.

    Cheers.

  29. yabba @ #489 Friday, April 26th, 2019 – 5:53 pm

    Re mental illness, latest figures on numbers affected by psychotic illness in Australia who have come into treatment is about 80,000, of whom about 50% are schizophrenic. There are of the order of ten bad schizophrenic attacks, leading to serious injury or death, each year. Nobody involved knows how to pick which of the 40,000 unfortunates will be the ones who become dangerous, although those who are also drug users are more prone to.

    The proper scientific studies that have been carried out indicate that psychotic disease results in an incidence of violent offending of between 1.2 and 2 times the ‘normal’ population.In other words, just like the rest of us, only a very small proportion of those with a psychotic illness will ever deliberately hurt someone. Supposedly sane men and women are responsible for more than 50 times more deaths each year than are mental patients (suicides excluded).

    Any breakdown of treated vs untreated?
    I understand the danger mainly come from psychotic patients who are untreated or not taking their medication.
    Unfortunately a lot of civil libertarians seem to be OK if patients die with their rights on. Sometimes a degree of compulsion is necessary when a patient lacks insight.

  30. ‘What you describe is much better than I have observed over the years in Victoria. (Cue zoomster to jump in and say all is perfect.)’

    Ah. Definitely bemused.

  31. four questions/mysteries
    1) why is the opposition leader so unpopular and for so long (dont blame murdoch – ask your neighbour) ?
    2) why is result not 55-45 despite murdoch and because of dearth of ideas and people in govt
    3) when do labor attack ads appear (it’s getting late)
    4) why is union affiliation such an advantage in leading labor – why can’t party be lead by someone with business or even education background, likewise liberals with someone from union?
    5) why can’t we have a dynamic younger even leader?

Leave a Reply

Your email address will not be published. Required fields are marked *