New Polls Out, Have Political Junkies Questioning Meaning of Existence

I said something a while ago about one poll not being news. What about, I dunno, four?

Campaign junkies are (or at least, I am) either excited, disgusted, or perplexed by the latest set of polls out from Upshot, a new 2014-midterxtravaganza section from The New York Times, and the Kaiser Family Foundation. You've got David Lauter at the Los Angeles Times giving a positive "always look on the bright side of life" spin, especially on Sen. Mark Pryor's chances in Arkansas. Then on the other side you've got the always-prescient Karl Rove at Fox News raising some admittedly reasonable objections to the poll methodology. And then you've got non-partisans wondering why these polls bucked trends so hard.

That's me, the last one. (I'm also a bit excited, but only because these are the first polls to come out of some of these states in weeks.) 

So, first things first: what do the polls say?

Arkansas. The biggest surprise for me was Arkansas Sen. Mark Pryor's sudden boost in his race against Republican Rep. Tom Cotton. According to the NYT/Kaiser poll, Sen. Pryor now leads his opponent 46-36 where our previous average (albeit one that was last updated a couple months ago) had the incumbent trailing by about 3 points. Where Sen. Pryor's approval ratings were supposed in the negative double-digits, this poll has his approval rating above his disapproval rating, 47-38.

Kentucky. The Kentucky result, with a statistical dead heat of 44-45 between Democratic Secretary of State Alison Lundergan Grimes and Republican Sen. Mitch McConnell, was completely in line with the average we've had for several months now. 

Louisiana. Unfortunately we can't add the Louisiana result to our average, since this poll lists every candidate on the ballot due to Louisiana's unusual "jungle primary" system, where the top two candidates on a giant nonpartisan slate of candidates move to a runoff election if no single candidate wins a majority. We're comfortable in only doing the averages for the runoff because we're confident at this point that the election will proceed to a runoff between Sen. Mary Landrieu and Rep. Bill Cassidy: Sen. Landrieu has consistently polled under 50%, while the Republican field is split at least three different ways. The NYT/Kaiser poll has Sen. Landrieu at 42%, Rep. Cassidy at 18%, state Rep. Paul Hollis at 5%, and a misspelled version of retired USAF Col. Rob Maness at 4%. Again, totally consistent with a runoff ensuing in December.

North Carolina. The Tar Heel State has actually had pretty consistent polling from Public Policy Polling, a Democratic firm headquartered in Raleigh. However, it's pretty refreshing to see what a non-PPP firm has to say about the race. Apparently, it's nothing new--both Sen. Kay Hagan leads 42-40. In our average, it's still a statistical tie. What was surprising from North Carolina was that apparently Sen. Hagan's approvals have gone from cold to tepid, with her approval and disapproval ratings at 44-44.

Should Democrats be excited?

We're not sure. There seem to be several contradictory figures in the report of the poll that may be a bit complex to explain away.

First and most obvious is what Mr. Rove had to say at the Fox News editorial page: what's with the sample? To be specific, in North Carolina, for example, President Obama lost by about two points to Gov. Romney. But in the sample breakdown for the poll, Obama voters outnumber Romney voters 38-31. It's even worse for the other states--in Arkansas, where the president lost by 24 points, Obama voters in the sample are outnumbered only 27-26. 

Incidentally, that's another thing I'm concerned about--why the total number of Obama and Romney voters was so low, considering that this was (ostensibly) filtered through a likely voter model. While Mr. Rove has an interest in making the environment look as poor as possible for Democrats (he needs to make those campaigns look worthwhile so donors will give to his SuperPAC), it certainly looks like Democrats were way, way oversampled. According to their methodology page, the percentages given in that sample breakdown were just the raw percentages and were re-weighted later--although even then, it's pretty difficult to draw a random sample of roughly half Obama voters from a state where President Obama lost by 24 points.

But that isn't the only thing up with this poll. The religious breakdown gives me pause as well. In particular, the evangelical numbers in Arkansas, Kentucky, and North Carolina look all right when compared to this handy interactive map from Pew's Religion and Public Life Project. However, where the numbers go awry is Louisiana, which, due to a strong French influence during the colonial era (the same reason they celebrate Mardi Gras down there), has a much higher proportion of Catholics than the rest of the deep South, and a correspondingly low proportion of evangelical Protestants. According to the Pew map, Louisiana's about 31-28 in favor of evangelicals. However, the NYT/Kaiser poll makes that composition out to be more like 44-23, which doesn't align with geographical data or historical trends.

What it comes down to is that I'm concerned about the sample: it doesn't align well with what previous data would predict. Ms. Marjorie Connelly, of Upshot, writes that it's a simple random sample throughout the four states using random digit dialing with regular calls back at different times throughout the day on different days, with appropriate demographic weights put in at the end to ensure a representative sample based on sex, race, age, the usual. I don't doubt that that was their method: I just see a contradiction between the randomness of their sample and the skew of the demographics in the polls, and if the sample isn't really all that random, the entire poll is questionable. Until we find out more, we're going to hold these polls from our average.