Of swings and misses: episode two

Talk of a new industry body to oversee polling standards gathers pace, even as international observers wonder what all the fuss is about.

The Sydney Morning Herald and The Age – or the Herald/Age, to adopt what is evidently Nine Newspapers’ own preferred shorthand for its Sydney and Melbourne papers – have revealed their opinion polling will be put on ice for an indefinite period. They usually do that post-election at the best of times, but evidently things are more serious now, such that we shouldn’t anticipate a resumption of its Ipsos series (which the organisation was no doubt struggling to fund in any case).

This is a shame, because Ipsos pollster Jessica Elgood has been admirably forthright in addressing what went wrong – and, importantly, in identifying the need for pollsters to observe greater transparency, a quality that has been notably lacking from the polling scene in Australia. In particular, Elgood has called for the establishment of a national polling standards body along the lines of the British Polling Council, members of which are required to publish details of their survey and weighting methods. This was echoed in a column in the Financial Review by Labor pollster John Utting, who suggests such a body might be chaired by Professor Ian McAllister of the Australian National University, who oversees the in-depth post-election Australian Election Study survey.

On that point, I may note that I had the following to say in Crikey early last year:

The very reason the British polling industry has felt compelled to observe higher standards of transparency is that it would invite ridicule if it sought to claim, as Galaxy did yesterday, that its “track record speaks for itself”. If ever the sorts of failures seen in Britain at the 2015 general election and 2016 Brexit referendum are replicated here, a day of reckoning may arrive that will shine light on the dark corners of Australian opinion polling.

Strange as it may seem though, not everyone is convinced that Australian polling really put on all that bad a show last weekend. Indeed, no less an authority than Nate Silver of FiveThirtyEight has just weighed in with the following:

Polls showed the conservative-led coalition trailing the Australian Labor Party approximately 51-49 in the two-party preferred vote. Instead, the conservatives won 51-49. That’s a relatively small miss: The conservatives trailed by 2 points in the polls, and instead they won by 2, making for a 4-point error. The miss was right in line with the average error from past Australian elections, which has averaged about 5 points. Given that track record, the conservatives had somewhere around a 1 in 3 chance of winning.

So the Australian media took this in stride, right? Of course not. Instead, the election was characterized as a “massive polling failure” and a “shock result”.

When journalists say stuff like that in an election after polls were so close, they’re telling on themselves. They’re revealing, like their American counterparts after 2016, that they aren’t particularly numerate and didn’t really understand what the polls said in the first place.

I’m not quite sure whether to take greater umbrage at Silver’s implication that Antony Green and Kevin Bonham “aren’t particularly numerate”, or that the are – huck, spit – “journalists”. The always prescient Dr Bonham managed a pre-emptive response:

While overseas observers like Nate Silver pour scorn on our little polling failure as a modest example of the genre and blast our media for failing to anticipate it, they do so apparently unfamiliar with just how good our national polling has been compared to polling overseas.

And therein lies the rub – we in Australia have been rather spoiled by the consistently strong performance of Newspoll’s pre-election polls especially, which have encouraged unrealistic expectations. On Saturday though, we saw the polls behaving no better, yet also no worse, than polling does generally.

Indeed, this would appear to be true even in the specifically Australian context, so long as we take a long view. Another stateside observer, Harry Enten, has somehow managed to compare Saturday’s performance with Australian polling going all the way back to 1943 (“I don’t know much about Australian politics”, Enten notes, “but I do know something about downloading spreadsheets of past poll data and calculating error rates”). Enten’s conclusion is that “the average error in the final week of polling between the top two parties in the first round” – which I take to mean the primary vote, applying the terminology of run-off voting of the non-instant variety – “has been about five points”.

Photo finishes

Progressive updates on late counting in the seats that will determine whether Scott Morrison governs in majority or minority.

A full display of the election results, with complete booth figures, swings and probability estimates, can be found here.

Thursday, May 23

Macquarie looks like going right down to the wire, with the first batch of absents favouring Labor 476-444, and the latest batch of postals reversing the earlier tide in breaking 259-194 to Labor. That cuts yesterday’s Liberal lead of 196 to 131. In Cowan, Labor’s lead is out from 748 to 839 as the first absents break 941-843 their way. Conversely, the first absents from Bass have broken 836-787 to Liberal, which, together with rechecking, pushes the Liberal lead out from 497 to 561. In Lilley, Labor’s lead slips slightly from 842 to 817 as the latest postals break 875-787 to the LNP, outweighing a rather hefty 275-197 Labor advantage on the first absents.

Wednesday, May 22

Only rechecking today in Cowan, where Labor ended the day 748 ahead, compared with 762 yesterday. Elsewhere:

Chisholm. Another batch of postals breaks 1064-905 to the Liberal, increasing their lead from 1220 to 1379.

Macquarie. The latest postals have broken 524-396 to Liberal, exactly the same proportion as those already in the count, increasing their lead from 68 to 196.

Bass. The Liberal lead nudges from 453 to 497, with the latest postals breaking 517-483 to Liberal together with some slight ordinary vote adjustments on rechecking.

Lilley. A big batch of postals breaks very much like the first, going to 2551-2093 to the LNP, which reduces the Labor lead from 1288 to 842, but doesn’t change the impression that Labor should be able to hold on.

Tuesday, May 21

My election results facility, linked to above, has ended the day less buggily than it began. Developments from today’s count:

Chisholm. Another 3963 postals have gone similarly the first 5413, and in doing so have increased the Liberal lead from 591 to 1220. The trend of absent votes in 2016 suggests Labor should only be able to claw back about 250 there.

Macquarie. Only rechecking done today, nudging the Liberal lead from 50 to 68. Labor should only make slight gains on absent and out-of-division pre-polls, which I think more likely than not to be outweighed by the Liberal gain on outstanding postals.

Bass. I thought the first batch of postals surprisingly strong for Labor, but it turns out postals behaved no differently from ordinary votes in 2016 as well, where usually they lean conservative. Today’s batch, however, went 767-706 in favour of the Liberals, increasing the lead from 392 to 453. Absent votes were likewise bang on the ordinary votes in 2016; out-of-division pre-polls favoured Liberal. So Labor would need to pull a rabbit out of the hat here.

Lilley. A rare bit of good news for Labor, in that it looks an error had been made in the Geebung booth that had it favouring the LNP 1046-830, but which has now shows as 1033-862 in favour of Labor. That boosts their lead from 901 to 1288, which you’d think would be enough.

Cowan. Labor’s lead has reduced from 1006 to 762 on the back of a second batch of postals, which went 1223-1040 to the Liberals – slightly less favourable for them than the first, of which the Liberals got 56.9% rather than 54.0% – and ordinary vote rechecking, which boosted them by 61. However, absents and out-of-division pre-polls in 2016 behaved very much like ordinary votes, and there shouldn’t be a huge mass of postals outstanding, so I would think it likely Labor will hang on.

Monday, May 20

As you can see above, I now have an election results facility in business, albeit still with a few bugs to be ironed out. With that more-or-less accomplished, I should be able to follow the final stages of the count in more detail. It now appears clear that the Coalition has secured a majority, the most likely result being 77 seats out of 151. Counting of postal votes is still at a fairly advanced stage, and these reliably lean conservative, so the trend of yesterday’s counting was in their favour. However, no absent votes have been counted, and these can sometimes go the other way. Furthermore, Kevin Bonham believes he has observed a tendency of the first batches of postals to be more conservative than later ones. With that in mind, here’s the latest mail from those undecided seats where counting progressed yesterday, i.e. all of them other than Indi and Boothby, where I’m probably being overly cautious in not calling them for independent and Liberal respectively.

Chisholm. Liberal candidate Gladys Liu extended her lead yesterday from 166 to 591. Postals have so far recorded a smaller swing to Labor, of 0.8%, than ordinary votes, which swung 2.5%. Still in doubt.

Macquarie. The Liberals now lead here by 50 votes, after trailing by 312 yesterday. Postals have so far recorded a 2.2% swing to the Liberals, not much different from ordinary votes. Still in doubt.

Bass. Better news for Labor here, with the Liberal lead narrowing from 437 to 392, and postals surprisingly swinging slightly in Labor’s favour after ordinary votes swung over 6%. Still in doubt.

Lilley. Labor’s lead narrowed yesterday from 1110 to 901. Postals haven’t swung much differently from ordinary votes so far, so Labor seem likely to hold on.

Wentworth. Kerryn Phelps conceded defeat to Liberal candidate Dave Sharma yesterday as a strong trend on postals blew the lead out from 1751 to 2864.

Sunday, May 19

This post will be used to provide regularly updated coverage of late counting in seats that remain in doubt, of which I count seven: the marginal Liberal seat of Chisholm in Melbourne, where newcomer Gladys Liu leads by 166 votes (0.11%); Macquarie on Sydney’s western fringe, where Labor incumbent Susan Templeman is 312 votes in front (0.18%); Bass, where Liberal candidate Bridget Archer holds a lead of 437 votes (0.36%) over Labor incumbent Ross Hart; Indi, where independent candidate Helen Haines holds a 2781 lead (1.6%) over the Liberals in her bid to succeed retiring independent Cathy McGowan; Labor-held Lilley in Brisbane, where Labor’s Anika Wells hold a 1110 vote lead (0.69%) as she seeks to succeed the retiring Wayne Swan; and, stretching it a little further, Wentworth, where Liberal candidate Dave Sharma now holds a 1751 lead (1.16%) over independent incumbent Kerryn Phelps, and Boothby, where Liberal incumbent Nicolle Flint leads Labor by 2183 votes (1.18%). Hopefully tomorrow I will finally get the time to fix the bugs in a results reporting facility that will report results and swings at booth level.

Of swings and misses

The Coalition’s parliamentary majority looks secure, as the polling industry starts facing up to what went wrong.

The latest – or some of it at least:

• It is now reckoned beyond doubt that the Liberals have held on in Chisholm, thereby guaranteeing a parliamentary majority of at least 76 seats out of 151. As related in the latest update in my late counting post, I think it more likely than not that they will supplement that with Macquarie and Bass, and wouldn’t write them off quite yet in Cowan. You are encouraged to use that thread to discuss the progress of the count, and to enjoy the reguarly updated results reporting facility while you’re about it.

• If you only read one thing about the collective failure of the opinion polls, make it Kevin Bonham’s comprehensive account. If you only read two, or don’t have quite that much time on your hands, a brief piece by Professor Brian Schmidt in The Guardian is worth a look.

• The three major polling companies have each acknowledged the issue in one way or another, far the most searching example of which is a piece in The Guardian by Peter Lewis of Essential Research. A statement released yesterday by Ipsos at least concedes there may be a problem with over-sampling of the politically engaged, but Monday’s offering by David Briggs of YouGov Galaxy in The Australian was defensive to a fault.

• Note the guest post below this one from Adrian Beaumont on tomorrow’s European Union elections in Britain.

UK’s European Union elections minus one day

The Conservatives head for their lowest ever national vote share in European Union elections, as Boris Johnson firms as a strong contender to be the next Prime Minister. Guest post by Adrian Beaumont.

Guest post by Adrian Beaumont, who joins us from time to time to provide commentary on elections internationally. Adrian’s work on electoral matters for The Conversation can be found here, and his own website is here.

Elections to the European Parliament in the various European Union countries will be held from May 23 to 26, on each country’s national election day. The UK uses Thursday for its elections, so its EU elections will be on May 23. There are a total of 751 seats in the EU Parliament, with 73 allotted to the UK. No results will be released until all countries have finished voting on May 26 (early morning May 27 Australian Eastern Standard Time).

For the EU elections, the UK is divided into 12 regions, with each region electing between three and ten members. The 12 regions are nine English regions, plus Scotland, Wales and Northern Ireland.  Except in Northern Ireland, members are elected by proportional representation at the regional level. But due to the limited number of seats per region, larger parties will earn a disproportionate share of the seats. Northern Ireland uses Australian-style Senate voting for its three seats.

Nigel Farage’s new Brexit party has surged to a clear lead in polling for the EU elections. Polling has the Brexit party in the low 30’s, Labour in the low 20’s, the Liberal Democrats between 12 and 17%, the Conservatives between 9 and 12% and the Greens between 6 and 11%.  While most polls have Labour in the 20’s, two YouGov polls gave Labour just 15%.  YouGov has been the worst poll for Labour for the last year. The Lib Dems and Greens gained after their success at the May 2 local elections.

If the Conservatives do as badly as polls suggest, it will be their worst ever vote share at a national election, and follow on from their loss of over 1,300 councillors at the local elections. Such an outcome will increase pressure on Theresa May to resign.

May will attempt to pass Brexit legislation through the Commons in the week beginning June 3, but this legislation’s prospects look bleak given Labour and hard Leaver opposition. Although May cannot face a no-confidence vote until December, this rule could be changed if she refuses to resign after losing yet another crucial vote. If May resigns or is forced out, Conservative MPs will winnow the candidates down to two, and those two will go to the hard Brexit-supporting membership. Boris Johnson would be a strong contender to be the next PM.

If Johnson becomes PM, he is likely to attempt to Leave the EU, deal or no deal. The Australian election was a massive setback for progress on global climate change – see this Conversation article for why. In my opinion, the only way left-wing parties will start consistently winning elections across the Western world is if there is a global economic disaster that is blamed on right-wing policies. A no-deal Brexit could be such an economic disaster.

Term three, day three

Anthony Albanese emerges the clear favourite to assume the Labor leadership, as the emergence of the party’s internal pollling belies the notion that it had any clearer an idea of what awaited it than the rest of us.

Some notable links and developments, as the Coalition inches closer towards a parliamentary majority in the latest counting:

• A few bugs remain to be ironed out, but I now have an regularly updated election results reporting facility in business that provides, among other things, booth results and swings in a far more accessible format than anything else on the market. If you would like to discuss the facility or the progress of the count in general, you are encouraged to do so on the late counting thread.

Samantha Maiden at The New Daily has obtained the full gamut of tracking polling conducted for Labor throughout the campaign, which is something I can never recall being made public before. The overall swing shown at the end of the campaign is of 1.5% to Labor, just like the published polls were saying. The polling was conducted by YouGov Galaxy, as indeed was much of the published polling during the campaign, this being the organisation responsible for Newspoll and the polls commissioned by the News Corp tabloids.

• Nathan Ruser of the Australian Strategic Policy Institute has produced fabulously revealing maps showing the distribution of two-party swings.

• Ladbrokes (no doubt among others) has a book open on the Labor leadership, which, with the withdrawal of Tanya Plibersek, has Anthony Albanese a clear favourite on $1.28, Jim Chalmers on $3.00, Chris Bowen on $5.50 and Tony Burke on $10.

The second morning after

A second thread for discussion of the post-election aftermath, as the Coalition waits to see if it will make it to a parliamentary majority, and Labor licks it wounds and prepared to choose a new leader.

I had a paywalled piece in Crikey yesterday giving my immediate post-result impressions, which offered observations such as the following:

Unexpected as all this was, the underlying dynamic is not new, and should be especially familiar to those whose memories extend to Mark Latham’s defeat at the hands of John Howard in 2004. Then as now, the northern Tasmanian seats of Bass and Braddon flipped from Labor to Liberal, with forestry policy providing the catalyst on that occasion, and Labor performed poorly in the outer suburbs, reflected in yesterday’s defeat in Lindsay and its failure to win crucial seats on the fringes of the four largest cities. There were also swings to Labor against the trend in wealthy city seats, attributed in 2004 to the non-economic issues of the Iraq war and asylum seekers, and touted at the time as the “doctors’ wives” effect.

So far as this blog is concerned though, other engagements have prevented me giving the post-election aftermath the full attention it deserves. I will endeavour to rectify that later today, so stay tuned. In the meantime, here is a thread for discussion of the situation. Note also the post below this one, dedicated to updates and discussion on progress in the late count.