Friday, October 31, 2014

Republicans are now a nearly two-to-one favorite to take the Senate

Two-to-one: not quite what I'd call betting odds. Republicans don't actually have a 66% chance of taking the Senate, but the 64% chance they have in our latest forecast is closer to it than any other forecast we've done since we'd started. Previously Republicans consistently had a slightly better than even chance of winning, usually in the mid-50s. The lowest they had gone was actually slightly worse-than-even chances of taking the Senate:

Friday, October 24, 2014

Close-Up: Overtime in Georgia

"Stalemate! You're all losers."

By "you all", I of course mean the vast majority of Americans living in one of the two states where runoff elections could decide control of the Senate. With less than two weeks before Election Day, people living in one of the dozen or so most competitive races are almost certainly sick of attack ads, faux-ksiness, largely cosmetic disagreement with President Obama, forced pride in Kentucky coal, alarmist and vaguely creepy Democratic fundraising emails, and door-to-door canvassers with a persistence rivaling that of a Mormon Jehovah's Witness selling a really good set of encyclopedias. Hell, I live in New Jersey and I'm already sick of hearing about how Aimee Belgard is a dishonest tax-and-spend liberal and how "T-Mac" (do not confuse him with "Mac T") is a North Jersey carpetbagger who hates women (the first part is true, however). Thank God I don't live in, say, Arkansas.

But don't be surprised if, for all the billions and billions of dollars thrown at this election, we still reach a stalemate on Election Night. Election Night. While it's likely that Republicans will pick up at least 6 seats on the night of November 4, it's also far from impossible that neither party reaches the finish line. In fact, one of the more likely scenarios come Election Night looks like this:


This is actually quite plausible since all we've done here is marked down the most likely outcome in each state. If a candidate wins more than 50% of the time in the 100,000 simulated elections, he or she is declared the winner. If both candidates receive under 50% of the vote in the relevant states (Georgia and Louisiana), the election is declared a runoff. In short, if we were to bet on any one combination of Senate results right now, this would be our top pick. But this sort of scenario is also the reason our forecast doesn't say January 3 (when the 114th Congress is sworn in) on it: it says "Democratic seats held on January 7, 2015" to account for the fact that Georgia doesn't hold its Senate runoff until January 6 next year, three days after the 114th Congress is sworn in. (It holds its gubernatorial runoff, should one be necessary, on December 2 this year.)

Thursday, October 23, 2014

No Cellphones, No Callbacks, No Neutrality

We've hoped for a while that people have learned, through hearing about polls, that one poll by itself doesn't constitute a game-changer. It should only be regarded as part of a larger trend--because absent a major, substantive scandal or the death of one candidate, polling as a whole rarely swings suddenly in a new direction. However, even in this context the latest New England College poll does give us something to think about: is Sen. Jeanne Shaheen really down against Scott Brown?

In its survey of 921 New Hampshire likely voters, New England College found that Sen. Shaheen was just barely behind the former Massachusetts senator by less than one percentage point, well within the margin of error. But then again, a month ago Sen. Shaheen routinely led by 6 or 7 points. "Within the margin of error" is not a phrase she wanted to hear. And it's not as though this poll is a truly isolated incident. Recent polls from UMass - Amherst, the University of New Hampshire, and SurveyUSA all show a close race. So should we accept this New England College survey as part of the story? 

Tuesday, October 14, 2014

To put it in terms that we can actually understand

A lot of the probabilities I give, especially in the most competitive races, are pretty easy to understand. Rep. Bruce Braley's odds of beating Joni Ernst in the Senate election in Iowa (48%) are no better than a coin toss. Secretary of State Alison Lundergan Grimes's chances of unseating Sen. Mitch McConnell in Kentucky (12%) are about the same as the odds of flipping a coin three times and getting three heads. But some of them are a bit tougher to grasp, especially in the safest seats. If we say, for example, that Shenna Bellows has less than a 1% chance of defeating Sen. Susan Collins in Maine, you should be getting two things from that statement:
  1. Because the page is only so wide, we can't include all of the decimal places needed before we start seeing numbers that aren't zero. That's how low her chances are.
  2. Her chances, however, are not zero. There is always the outside chance that she can win; very little in the world is 100% certain. The only candidate who is 100% assured of victory this year is Sen. Jeff Sessions, who is unopposed by any candidate. Even if he drops dead in the time between when I finish typing this sentence and when I click "publish", Alabama voters don't have the option to vote for anyone else. (Someone would be appointed to fill his place and then a special election would take place, probably next year, but the now-late Sen. Sessions would still be elected.)
But again, it's hard to get a real-life grasp on how low those chances are. FiveThirtyEight gets around this by using sports analogies in its "A Probabilist Equivalent From The Sports World" segment, where Messrs. Silver and Enten use a (presumably familiar) sporting statistic, such as "The GOP’s current chances of taking the Senate, 57.6 percent, are the same exact chances the New England Patriots had of winning Super Bowl XLVI against the New York Giants with eight seconds left in the first half." 

I have two problems with this, though. Problem #1 is that I'm not a sports guy, so I understand what he's saying even less than I did before he gave the analogy (although that's more a personal problem). Problem #2 is that the probability is near-meaningless unless it's A) greater than the arbitrary 95% confidence threshold statisticians and social scientists agreed on somewhere down the line or B) reporting a statistic that can be reproduced in repeated experiments. That sports analogy is neither; presumably that example means that in situations resembling whatever it is he just said, the team in the Patriots' position won 58% of the time. It can't be repeated, and even if some games come close to it the sample size isn't large enough to make that kind of probabilistic statement down to the decimal point.

I do, however, love myself some poker, so I find it easier to express these probabilities in terms of poker hands. This has some nice side effects, the first of which is that poker hands, being drawn from a fixed deck of cards, have expected probabilities predicted purely by math; we know that the probability of drawing any particular poker hand is 1 in 2,598,960. Second, poker as a game can be simulated and has in effect been simulated over the millions and billions of hands that have been played in its history. Not only can you calculate probabilities mathematically, you could also theoretically observe them, given the patience. 

For your amusement, here are some results I picked out:

Wednesday, October 8, 2014

Notes on the expanded three-way simulator

In simulating elections, we've so far been able to mostly ignore third-party candidates in determining the chances of victory--they've been so far behind that there isn't any need to calculate their probability of victory because it's essentially zero. In R code (the language under which our simulations are run) the relevant section of the simulator looks like this:
for(i in 1:100000) {
...
        if(D > R) {
              result <- 1 ## Democrat wins
        } else {
              result <- 0 ## Republican wins
        }

        cumulative <- cumulative + result
    }
    cumulative / 1000 ## Democratic win percentage

Close-Up: Independent Candidates in Sparsely Populated Prairie States Are Ruining Our Probabilities

Polls in the Great Plains delivered another defiant middle finger to our conventional wisdom about the Senate races this year. SurveyUSA released its latest survey of the state, and while Democratic no-name Rick Weiland remains at his sub-30% support, the major change is that this poll is the first one to put independent and former Republican Sen. Larry Pressler in a clear second place behind former Gov. Mike Rounds.

There's a bit of a paradox here, though. When we talk about third party candidates we inevitably wonder: is the candidate spoiling votes for the Democrat or the Republican? In South Dakota, before ever looking at the polls, we would guess that Pressler was spoiling votes for Rounds, the Republican, since Pressler represented South Dakota in the Senate as a Republican for 18 years before losing the seat to the man who currently holds it, retiring Sen. Tim Johnson. The trends seem to confirm this--Pressler's support has generally grown since the summer, while Rounds still remains ahead but seems to have trouble breaking the low 40s in the polls compared to the summer, where his best performance was 59% in the first iteration of the YouGov panel survey. (Note, however, that that survey did not include Pressler as an option.)

Larry Pressler, then (1979) and now (2013).