Thursday, January 15, 2015

Britain has a 56% chance of not returning a government after Election Day

"Oh look, a squirrel. How quaint." - Tory PM David Cameron (right), to Labour leader Ed Miliband (left) and Lib Dem leader Nick Clegg (center). No pun intended.*

Ever since this blog was started in March last year I had been fairly certain that there would be quite a bit of a hiatus at least for the beginning of 2015. After all, there's not much election that happens in odd-numbered years, save for some scattered state elections in the South (two of which, Mississippi and Louisiana, already seem to favor Republicans) and some mayoral elections. There's also the pre-presidential primary scramble, which promises to be interesting, and of course 2016 sees a totally unclear presidential election as well as a Senate map that's almost as favorable for Democrats as the map the Republicans had in 2014. But of course, 2016 is just under a year from now, and for the primaries only two major candidates, former Florida Gov. Jeb Bush and former U.S. Senator from Virginia Jim Webb, have even formed exploratory committees. (That's if you exclude joke candidates like Jeff Boss, who unsuccessfully ran for the Democratic nomination for governor of New Jersey in 2013, and then successfully petitioned to get on the ballot as the "NSA did 911" candidate.)

As it happens I get bored easily, and an off term from college almost certainly doesn't help. (For me, it's either watching elections or playing TF2 all day anyway.) So what's there to watch in 2015? A lot, as it turns out. This year, America's two closest allies have scheduled national elections: the United Kingdom election is on May 7, and Canada's tentatively scheduled its elections for October 19.

How much does it matter to the U.S. domestically? Probably not that much. But hey, it's something to do, so here's how it works.

A crash course in parliamentary politics
Recycling the model from the 2014 elections doesn't work all that well because it applies specifically to and was developed specifically for the U.S. elections. But most English-speaking countries elect their parliaments wildly differently from the way the U.S. elects its governments. Specifically, the U.S. is run on a presidential system, while our fellow Anglophone nations are run on parliamentary systems. The crucial difference between the two is that in a presidential system, the executive is elected independently of the legislature. In a parliamentary system, only the legislature is elected directly--the legislature then elects someone, almost always the leader of the majority party, as the prime minister. Like a president, the prime minister acts pretty much as an executive: he directs the national policy agenda and has a cabinet to advise him and implement the laws passed by the legislature.

This difference leads to some phenomena that Brits, Aussies, Kiwis, and Canucks (it's not racist if they have a sports team with that name, right?) would find almost alien, but which Yanks see as pretty much par for the course. Specifically, the U.S. has seen divided government, where a party other than the president's party controls at least one house of Congress, for more than 26 years of the 34 years since 1981--13 of the last 17 Congresses. In the UK and Canada, by contrast, such an arrangement is pretty much impossible because it's the majority party in the legislature that chooses the prime minister (and there's only one elected house of Parliament in each country; the British House of Lords and the Canadian Senate are unelected bodies). In the UK, for example, a Conservative Parliament run by a Labour PM would be like if House Republicans elected Nancy Pelosi as Speaker of the House.

With that in mind, the goal of predicting the party that wins the premiership isn't as simple as stacking polls up against each other. The intermediate goal is to predict which party wins a majority--if any party wins a majority at all, that is, since both Canada and the UK, unlike the U.S., have third and even fourth parties that are alive and kicking, if only among certain segments of the electorate. In some relatively straightforward models, what I've done is regress the percentage of seats won by a party in the House of Commons (it's the name in both countries) on the vote a party received as a share of the vote received by all parties that won seats in the election. 

Because there's so much detail, Canada gets its own post next; this one focuses on the UK.

The current British situation
Current PM David Cameron's Conservative Party (or "Tories", colloquially, for reasons I can't explain) won 36% of the vote (39% of the vote if you count only those parties that won at least one seat in the House). That was more than any other party, but it was only enough to win the Tories 306 of the House's 650 seats, 20 short of the 326 needed for a majority. With no party receiving a majority, the result of the election was a hung parliament, in which no majority was able to elect a prime minister. It fell to the 57 Liberal Democratic MPs, under the leadership of Nick Clegg, to decide which party to form a governing coalition with. In combination with 11 seats won by minor parties, Clegg could easily have given Brown a second shot at the premiership. However, it was not so; Clegg allied with the Conservatives to become Deputy Prime Minister, with Conservative leader David Cameron at the helm.

That's the background. The aforementioned regression of historical seat share on vote share for the Conservatives is summarized in the following table:



Coefficient
Std. Err.
t
P
Intercept
-51.7572
12.8680
-4.022
0.0069
ConsVote
2.5421
0.3385
7.510
0.0003
R2
0.9038




And here it is for Labour:



Coefficient
Std. Err.
t
P
Intercept
-22.7786
13.8379
-1.646
0.151
LabVote
1.9955
0.3942
5.062
0.002
R2
0.8103




While the data used for these regressions only go back to the 1979 election that propelled Margaret Thatcher to the post of prime minister (as an arbitrary date for the beginning of "modern British politics" to which any models applicable today might apply), the results nevertheless contain predictive power, explaining 80-90% of the variation in seat share by popular vote share alone.

What should the takeaways from these tables be? Let's take the Conservatives as an example. The intercept is -51.8; speaking very abstractly here, this means that if the Conservatives were to win 0% of the vote, they would be expected to win -51.8% of the seats in the House of Commons. Similarly, if Labour were to win 0% of the vote, the party would be expected to win -22.8% of the seats in the House.

Plainly this makes no sense, but the main idea is that the Conservatives start at a much lower "baseline" vote share than Labour does. At most levels, Labour would be expected to win more seats than the Conservatives do even at identical levels of support in the popular vote. At 35% of the vote, for example, Conservatives are predicted by the model to win 2.542 x 35 - 51.7572 = 37.2% of the seats; Labour would be expected to win 1.9955 x 35 - 22.7786 = 47.1% of them at the same share of the popular vote. That translates into a 60-seat bonus in the House of Commons. 

Clearly this model has its limitations. In particular, it doesn't work for the Conservatives if their vote share drops below 25% because the number of seats predicted would be negative, nor does it work for them above 58% support because the number of seats predicted would be over 100%. (Similarly, the model breaks down for Labour below 14% and above 63%.) Even close to those bounds the model seems to fail miserably, seeing as it produces results where one party has almost no seats--and that hasn't really ever happened in British history and is difficult to picture happening today. Fortunately, however, the data for this election don't appear to be anywhere close to breaking the model. Since 1979 the Tory vote has never dropped below 30% and never risen above 45%; historical Labour vote shares lie within a similar range. So this model should work reasonably well if the election doesn't deviate wildly from elections in the past.

If polling is any indication, it won't. This year's popular vote shares appear to be well within the range in the sample. Labour is polling at about 34% in our average (about 36% as a percentage of estimated seat-winning parties) and Conservatives at about 32% (34% as a percentage of estimated seat-winning parties). From these numbers the model currently predicts that Labour will win about 316 of the 650 seats in the House of Commons, while the Tories are expected to send about 224 candidates to Parliament. Both major parties are short of the 326 required to win a majority, but the confidence intervals are fairly broad. Labour's 95% confidence interval is between 316 ± 50 seats, putting the magic 326 number squarely within reach. (The Tories' confidence interval, 232 ± 57 seats, does not overlap with 326.)

Spanners in the works
Two factors might hamper the applicability of this model. First, regression on Liberal Democratic seat share has much less predictive power than the regressions on Labour and Conservative seat shares do: 


Coefficient
Std. Err.
t
P
Intercept
-1.9481
4.5062
-0.432
0.681
LibVote
0.4150
0.2416
1.718
0.137
R2
0.3296




With an R-squared of 0.33, this regression has much less explanatory power for the variation from the mean, and not least due to the fact that the Liberal Democrats, as the usual distant third place finisher in British elections, tend really just to scrape up whatever leftovers the two major parties and various local narrow-but-deep parties haven't picked up. As a result, the number of seats actually won by Lib Dems varies wildly compared to the share of the vote they win nationally. In 2010, for example, although the Lib Dems increased their vote share by one point to a record-breaking 23%, they actually lost five seats in that election.

More importantly, though, is upending of the 2 + 1 party system in Britain: two clear major parties (Labour and Conservative), followed by a third party that's way ahead of all other parties in the nation (Liberal Democrats). Since the spring of 2013, however, the Lib Dems have consistently trailed the upstart UK Independence Party (UKIP) in polls. This really is tough for modeling to handle, since the UKIP has never won a single seat in Parliament in a general election (although two special elections last year gave them two seats, and they currently control a plurality of the British delegation to the European Parliament).

Ideally, since the UKIP's Eurosceptic stance (leader Nigel Farage has been one of the most outspoken advocates for British withdrawal from the EU), conservatism on social issues, and monarchism place the UKIP squarely to the right of the Tories, we could use the Conservative regression model for the UKIP as well and be done with it. And that would probably work reasonably well, if not for the fact that the UKIP is currently polling at around 14-17%--pretty high, but still too low for the Conservative regression model to generate anything except negative seat counts. Instead, what I've had to do is pool all party-elections together and regress seat share on vote share for all parties across all elections since 1979--so the Labour result from 1979, the Conservative result from 1979, the Liberal/SDP Alliance result from 1979, the Labour result from 1983, etc. would all be put in the same sample. The benefit of this is that the sample size is enlarged, which allows for more precise estimations of the coefficients, but the drawback is that pooling the data assumes certain generalizations across the different parties results that may not actually apply. Still, it's the best we can do. Here's the result of that regression:


Coefficient
Std. Err.
t
P
Intercept
-13.49422
2.61852
-5.153
< 0.0001
VoteShare
1.46498
0.07613
19.244
< 0.0001
R2
0.8769



 

We can't know how well this model predicts the UKIP performance in the House of Commons, but we can sort of guess that the model errs on the side of overestimation of the UKIP performance. Third parties, after all, always tend to underperform their popular vote when it comes to the number of seats they actually win under first-past-the-post, winner-take-all voting systems as we have in most of the U.S. and in the UK. It's also likely that in a close election (which this one definitely qualifies as, at this moment), a good portion of the people who say in surveys that they will vote UKIP will essentially chicken out on Election Day for fear of wasting votes to the benefit of the major party candidate they most agree with. 

At any rate, once we've got these models all estimated, all we need to do is calculate the popular vote share needed to attain a majority of the seats in Parliament. This we can do with some simple algebra; after that we can turn to our old simulator from the 2014 elections to determine the probability of each party actually receiving that much of the vote. The results are:
  • a 44% chance of Labour winning a majority;
  • a 56% chance of a hung parliament;
  • less than 1% chance of any other party winning a majority. 
*The unintended pun in the caption was about their political positions.... Miliband (Labour) on the "left", Clegg (Liberal Democrat) in the "center", and Cameron (Conservative) on the "right".... groan, I know.

No comments:

Post a Comment