From my paywalled article in Crikey yesterday:
In the wake of its most unambiguous failure at a federal election since at least 1980, Australia’s polling industry is licking its wounds.
The Nine/Fairfax papers have announced the Ipsos poll series will be put on ice, and those pollsters who do return to the field shortly will face catcalls whether they persist in recording a Labor lead we now know doesn’t exist, or only now start detecting a Coalition lead that eluded them through the entirety of the past parliamentary term.
Despite it all though, the pollsters’ performance hasn’t been without its defenders.
Spoiler alert: the latter refers to David Briggs and Nate Silver. But Peter Brent can now be added to the list, up to a point, following a review of the issues raised by the polling failure in Inside Story. Specifically, Brent observes that the primary vote miss was less severe than the two-party preferred; that the difference arose from a stronger-than-anticipated flow of minor party and independent preferences to the Coalition; that herding was less apparent on the primary vote (most markedly in the case of Ipsos’s reading of the balance of support between Labor and the Greens); and that the result was, if nothing else, no worse than the Victorian state election.
Another point noted is the strange consistency with which polls have pointed to extravagant gains for Labor in Queensland before and during election campaigns, only for them to fall away at the end. On this occasion, the falling away as recorded by pollsters wasn’t remotely on the scale needed to predict the result, with statewide polling published towards the end of the campaign landing at least 7% shy of what looks like being the Coalition two-party vote in the state.
The question of geographic variability in the pollster failure seemed worth exploring, so I have put together a table of state and electorate level polling published in the last fortnight or so of the campaign, available below the fold at the bottom of the post. Almost all of this polling was conducted by YouGov Galaxy, whether under its own name or as Newspoll. The only exception was a set of state-level two-party preferred totals from Ipsos, published at the tail end of the campaign by the Age-Herald (which performed rather poorly).
Below all this is a list of “average bias” figures, consisting of straight averages of the observed errors, be they positive or negative, rather than the absolute errors. This means combinations of positive and negative results will have the fact of cancelling out — although there were actually very few of those, as the errors tended to be consistently in the one direction. The national and state-level two-party results are estimates provided to me by Nine’s election systems consultant David Quin. With no Coalition-versus-Labor figures available from 15 electorates, this inevitably involves a fair bit of guess work.
A few points should be observed. Given that poll trends pointed to a clear long-term trend to the Coalition, pollsters may be excused a certain amount of Labor bias when evaluating polling that was in many cases conducted over a week before the election. This is particularly true of the Newspoll state aggregates, which cover the full length of the campaign.
Another issue with the Newspoll state aggregates is that One Nation was a response option for all respondents in the early part of the campaign, despite their contesting only 59 out of 151 seats. Their vote here accordingly comes in too high, and as Peter Brent notes, at least part of their failure could be explained by stranded One Nation supporters breaking in unexpectedly large quantities to the Coalition, rather than other minor party targets of opportunity like Clive Palmer.
In seat polling though, where the issue did not arise, the polls were remarkable in having understated support for One Nation, and overstated it for the United Australia Party. This was one face of a two-sided polling failure in Queensland, of which the other was a serious imbalance towards Labor in support recorded for the major parties. While Queensland has caught most of the attention on this score, the polls were just as far out in measuring the primary votes of the major parties in Western Australia. Things were less bad in Victoria, but Coalition support was still significantly underestimated.
The only bright spots in the picture are New South Wales and South Australia, where Newspoll just about nailed the Coalition, Labor and Greens primary votes, and got the big things right in four seat polls. While Labor’s strength was overstated in Macquarie, it does now appear Labor will pull through there – for more on that front, stay tuned to the late counting thread.