The polls and the sum of the parts

An overdue appraisal of the pollsters’ performance at the May federal election.

The 2022 federal election was a much happier experience for the polling industry than 2019, with each of five pollster producing election eve primary vote numbers broadly suggestive of the actual result. However, there was a collective error in favour of Labor, whose actual primary vote came in 2.3% below the pollster consensus while the Coalition landed 0.4% higher. While not at the standard Australian consumers had come to expect before 2019, such errors were fairly moderate by historic standards, particularly in the international context.

The pollsters in the national voting intention game before the election: Newspoll, conducted by YouGov for The Australian; Ipsos, for the Financial Review; Resolve Strategic, for the Nine Newspapers; Essential Research, for The Guardian; and Roy Morgan, which tacks voting intention on to its market research survey for its own amusement and, in the pre-election period at least, regularly published its results. The charts below show their final pre-election poll results for the Coalition, Labor and the Greens as black dots that lie at the centre of the span of their margins of error, with the relevant party’s actual result shown as a thick vertical line.

In the case of the three who are members of the Australian Polling Council (Newspoll, Ipsos and Essential Research), what is shown are the pollsters’ effective margins of error, which account for the fact that their results are weighted to emphasise or de-emphasise demographic cohorts who are over-represented or under-represented in their samples. Since Resolve Strategic and Roy Morgan don’t provide this detail, the margins of error have been calculated from their raw sample sizes, making them somewhat smaller than they would be otherwise.

Some caveats must be applied here: the final survey periods ranged from over a week before the election in the case of Roy Morgan (from May 9 to 15) to the course of the final week (Saturday to Wednesday from Ipsos, Thursday to Thursday from Newspoll), and pollsters may always plead they were caught out by late shifts in voting intention that it was beyond their power to foresee. There’s also a certain injustice in evaluating a pollster’s performance entirely by its pre-election poll, as the unavoidable randomness of the exercise means an element of luck is involved in who gets the honours. The best we can do is keep that in mind in the analysis that follows.

Essential Research did not have a great result, having systematically understated the non-major party vote (results here exclude a 7% undecided component). Other than that, three results are clearly outside the margins of error: Newspoll’s 36% and Ipsos’s 35.8% (after exclusion of 5% uncommitted) for Labor, whose actual result was 32.6%, and Resolve Strategic’s 14% for the Greens, whose result was 12.3%. None of these final poll results look like outliers for the pollster concerned. Newspoll’s four previous polls, which were published weekly during the campaign period, had Labor one to three points higher than the final result; Ipsos’s polling consistently had Labor clear of their actual result even without excluding their uncommitted component; and Essential Research’s numbers bounced a few points at a time within a consistent range throughout 2022.

Resolve Strategic’s evident inflation of Greens support was peculiar to its last two polls, which had it at 15% and 14% compared with 10% to 12% in its earlier polling. It should be noted here that margins of error are tighter for parties with lower vote shares, and that it’s conceivable that bad luck with rounding put the final poll outside the error margin at 14% rather than within it at 13%. State breakdowns show the inflation of the Greens vote arose from New South Wales and Queensland, which was respectively balanced by unduly low results for Labor and the Coalition. The latter was a peculiarity of the final poll, which had the Coalition at 31%, down from 41% at the previous poll a fortnight earlier and comparing with an actual result of 39.6%.

Essential Research’s issues were consistent across the board, landing too high for the Coalition and Labor and too low for the Greens in each state. The Victorian numbers were particularly far off the mark, perhaps reflecting their unusually high reading of 11% undecided in this state. It should be noted that the results for Western Australia and South Australia are three-week rolling averages, which means they ought not be regarded as strictly election eve results.

By process of elimination, it might be thought that the honours belong to Roy Morgan, whose reputation never really recovered from a series of poor federal election performances two decades ago. However, state breakdowns suggest a certain amount of luck was involved here, in that every one of its results during 2022 put Labor much higher in Victoria than they managed at the election. The element of luck was that the least askew of these results came in the pre-election poll, which had Labor’s lead in Victoria at a relatively narrow 57-43 compared with an actual result of 54.6-45.4. UPDATE: Adrian Beaumont also points out that they always had the United Australia Party implausibly low, the final result coming in at 1% compared with an election result of 4.1%, and this was matched by an excessive reading for independents/others.

Unfortunately there are state breakdowns available for Newspoll, for which we disappointingly did not see any of the usual aggregated breakdowns during the campaign period, and Ipsos, which did not maintain the finely detailed breakdowns it offered for its first few polls a the business end of the election. I’ll have more to say though about pollster performance at the election during quiet moments over the weeks or months to come, so stay tuned.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics.

13 comments on “The polls and the sum of the parts”

  1. I would argue anyone who votes other than.alp or conservative is a swing voter. But if you mean by will shift their vote. Then there will be people who will always vote uap onp or gr.. then the number is much lower.

  2. If you eliminate Morgan the ‘poll of polls’ two pp was spot on at 52%. Even with Morgan it’s within 0.5%. So I think it was a great result for pollsters this time. Although seat polls are obviously still useless.

  3. History, only because primary vote and preferencing errors sort of cancelled. Pollsters overstated Labor’s primary, but used the 2019 preference flows, which understated Labor’s preference share.

  4. Adrian has pointed out to me that I was failing to account for Essential’s undecided component, which made their results look better than they were, so bits of this post have been redone.

  5. Pollsters ask voters who they intend to vote for. In many cases – perhaps 7/10 – a more relevant question is who they intend to vote against. They could also be asked which party or parties will provide the “best” government.

  6. Historical comparisons of ALP primary should take into account distortions from strategic voting for independents, there is no precedent for the scale of it in the last election AFAIK.

  7. bug1 says:
    “Historical comparisons of ALP primary should take into account distortions from strategic voting for independents …”

    Kevin Bonham:

    “Incidentally, a widespread theory that Labor’s low primary was a result of strategic voting for teal independents is also false. There only were about 17 seats with new teal independents (depending on definition) and these had an average primary vote swing against Labor of 5.2%, meaning there was still a 0.2% primary vote swing against Labor on average in the rest. Labor’s failure to increase its primary vote despite the Coalition’s primary vote losses would have had more to do with Labor losing votes to the Greens and One Nation (the latter purely on account of One Nation contesting nearly every seat).”

Leave a Reply

Your email address will not be published. Required fields are marked *