The polls and the sum of the parts

The 2022 federal election was a much happier experience for the polling industry than 2019, with each of five pollster producing election eve primary vote numbers broadly suggestive of the actual result. However, there was a collective error in favour of Labor, whose actual primary vote came in 2.3% below the pollster consensus while the Coalition landed 0.4% higher. While not at the standard Australian consumers had come to expect before 2019, such errors were fairly moderate by historic standards, particularly in the international context.

The pollsters in the national voting intention game before the election: Newspoll, conducted by YouGov for The Australian; Ipsos, for the Financial Review; Resolve Strategic, for the Nine Newspapers; Essential Research, for The Guardian; and Roy Morgan, which tacks voting intention on to its market research survey for its own amusement and, in the pre-election period at least, regularly published its results. The charts below show their final pre-election poll results for the Coalition, Labor and the Greens as black dots that lie at the centre of the span of their margins of error, with the relevant party’s actual result shown as a thick vertical line.

In the case of the three who are members of the Australian Polling Council (Newspoll, Ipsos and Essential Research), what is shown are the pollsters’ effective margins of error, which account for the fact that their results are weighted to emphasise or de-emphasise demographic cohorts who are over-represented or under-represented in their samples. Since Resolve Strategic and Roy Morgan don’t provide this detail, the margins of error have been calculated from their raw sample sizes, making them somewhat smaller than they would be otherwise.

Some caveats must be applied here: the final survey periods ranged from over a week before the election in the case of Roy Morgan (from May 9 to 15) to the course of the final week (Saturday to Wednesday from Ipsos, Thursday to Thursday from Newspoll), and pollsters may always plead they were caught out by late shifts in voting intention that it was beyond their power to foresee. There’s also a certain injustice in evaluating a pollster’s performance entirely by its pre-election poll, as the unavoidable randomness of the exercise means an element of luck is involved in who gets the honours. The best we can do is keep that in mind in the analysis that follows.

Essential Research did not have a great result, having systematically understated the non-major party vote (results here exclude a 7% undecided component). Other than that, three results are clearly outside the margins of error: Newspoll’s 36% and Ipsos’s 35.8% (after exclusion of 5% uncommitted) for Labor, whose actual result was 32.6%, and Resolve Strategic’s 14% for the Greens, whose result was 12.3%. None of these final poll results look like outliers for the pollster concerned. Newspoll’s four previous polls, which were published weekly during the campaign period, had Labor one to three points higher than the final result; Ipsos’s polling consistently had Labor clear of their actual result even without excluding their uncommitted component; and Essential Research’s numbers bounced a few points at a time within a consistent range throughout 2022.

Resolve Strategic’s evident inflation of Greens support was peculiar to its last two polls, which had it at 15% and 14% compared with 10% to 12% in its earlier polling. It should be noted here that margins of error are tighter for parties with lower vote shares, and that it’s conceivable that bad luck with rounding put the final poll outside the error margin at 14% rather than within it at 13%. State breakdowns show the inflation of the Greens vote arose from New South Wales and Queensland, which was respectively balanced by unduly low results for Labor and the Coalition. The latter was a peculiarity of the final poll, which had the Coalition at 31%, down from 41% at the previous poll a fortnight earlier and comparing with an actual result of 39.6%.

Essential Research’s issues were consistent across the board, landing too high for the Coalition and Labor and too low for the Greens in each state. The Victorian numbers were particularly far off the mark, perhaps reflecting their unusually high reading of 11% undecided in this state. It should be noted that the results for Western Australia and South Australia are three-week rolling averages, which means they ought not be regarded as strictly election eve results.

By process of elimination, it might be thought that the honours belong to Roy Morgan, whose reputation never really recovered from a series of poor federal election performances two decades ago. However, state breakdowns suggest a certain amount of luck was involved here, in that every one of its results during 2022 put Labor much higher in Victoria than they managed at the election. The element of luck was that the least askew of these results came in the pre-election poll, which had Labor’s lead in Victoria at a relatively narrow 57-43 compared with an actual result of 54.6-45.4. UPDATE: Adrian Beaumont also points out that they always had the United Australia Party implausibly low, the final result coming in at 1% compared with an election result of 4.1%, and this was matched by an excessive reading for independents/others.

Unfortunately there are state breakdowns available for Newspoll, for which we disappointingly did not see any of the usual aggregated breakdowns during the campaign period, and Ipsos, which did not maintain the finely detailed breakdowns it offered for its first few polls a the business end of the election. I’ll have more to say though about pollster performance at the election during quiet moments over the weeks or months to come, so stay tuned.

Author: William Bowe

William Bowe is a Perth-based election analyst and occasional teacher of political science. His blog, The Poll Bludger, has existed in one form or another since 2004, and is one of the most heavily trafficked websites on Australian politics. View all posts by William Bowe