Cell Phones and Polls

I have been a skeptic of the argument that the polls are undercounting Obama supporters due to a “cell-phone” affect. The idea here is thus: most polling outfits use only landlines to contact potential respondents. A growing number of people do not have landlines, but use only cell phones. These cell-phone only folks are likely to be younger, and therefore likely to be Obama supporters. Excluding cell-phone voters means a biasing your sample in favor of McCain.

People started making this argument back in 2004 and as I said, I have been very skeptical of it. For one thing, I doubted that the number of cell-phone only voters was nearly large enough to make any kind of difference. For example, there must be some number of voters out there who don’t have phones at all. These people also are likely to back the Democratic candidate (based on the assumption that they don’t have phones due to financial constraints), but we never heard much about their impact, or lack thereof, on public polling because there just aren’t enough of them. Secondly, I figured that even if pollsters were worried about those cell-voters, they could easily reweight their sample to add slightly more young people since there was no reason to believe that, if voters who only have cell phones are disproportionately young, they are any different politically from other young voters. Finally, and most importantly, I was skeptical because I didn’t see actual hard evidence supporting the claim.

As recently as this weekend I maintained my skepticism as the effects of cell phone only voters. But I have to admit that, this week, I have begun to reevaluate my opinion based on two new analyses. First, Nate Silver at fivethrityeight.com did some analysis comparing polls taken by firms that include cell phones in their samples and those that do not. He found that adding cell phones improves Obama’s relative standing by about 2.8 percentage points on average (meaning if he was ahead by 2 before cell phones, he’s ahead by about 5 afterwards). Then, today, we have a new report from Pew about cell phones and polling and they find that adding cell phones “results in slightly more support for Obama and slightly less for McCain, a consistent difference of two-to-three points in the margin.” Nearly identical findings to Nate Silver. Furthermore, a big part of the reason for this impact is that, contrary to my assumptions stated above, cell-phone only voters are actually different from young people in general. More specifically, they appear to be even likelier to back Obama.

So what does this mean? Should we all just add two points for Obama to every poll we see? Hmm, probably not. For starters, some polls actually are already including cell phones in their samples (see here ) so there’s no reason to adjust those polls. Also, it’s not at all clear that the effect is the same across states. There might be good reason to believe, in fact, that the cell phone affect would be higher in some places than others (New York probably has a higher percentage of cell phone only voters than Iowa, for example).

This is probably a good place for me to reiterate my belief that looking at the day to day fluctuations of individual polls isn’t a very good way to evaluate the state of the race. We all tend to treats poll as if they were censuses (“Obama IS two points ahead,” “McCain gained three from last week”) when of course they are samples. Sampling contains error, both inherent random error and human introduced error. Cell phones are only one part of that.

I think the lesson here is less about what the polls tell us than about what they don’t. Polls are very good tools, but they can’t give us a perfect picture of the electorate’s preferences. Cell phones and weighting by party id (a topic for another post) and Bradley affects are part of the package. My bet is that 39 days from now, the polls will still show a close race with about 5% undecided, and even then all will be reasonably sure of is that no one’s gonna run away with it.