Fudging Poll Numbers 101

Rush Limbaugh recently accused the Gallup Poll of "upping the sample [of] black Americans to keep [Obama] up at 50%" in the polls. Now, if you were a pollster and really did want to cheat and over-represent Democrats, there is a much easier way to do it.

Mark Blumenthal links to Rush Limbaugh accusing Gallup of "upping the sample [of] black Americans to keep [Obama] up at 50%" in the polls.  (For the context, see the last paragraph of the transcript.)

Frank Newport of Gallup responds here.  Newport denies it all, but he would, wouldn't he?

Seriously, though, it's hard to believe that Limbaugh really believes that Gallup is fudging the numbers.  As a big-time radio host, he's gotta know all about marketing surveys, right?  I'm just assuming he said that "upping the sample" bit as more of a joke or an off-the-wall speculation.  It did raise two interesting questions in my mind, though:

1.  The assumption behind Limbaugh's argument -- as with many arguments about polls -- is that the published poll results have an effect of their own, beyond the president's underlying popularity.  For example, maybe some senator would vote for the healthcare bill if he read that Obama's approval rating was 51% but would vote no if he read that Obama only had 49% approval.  This might very well be true -- it makes sense -- I just don't really know.

2.  What if you were a pollster and really did want to cheat and over-represent Democrats?  How would you do it?  Contra Limbaugh's suggestion, I don't think you'd oversample blacks.  Gallup does telephone surveys, and it's not like there's a separate telephone directory for blacks.  Also, as several commenters to Newport noted, the percentage of blacks among the survey respondents is easy enough to check.  And, for that matter, many survey organizations (possibly including Gallup) do post-sampling weighting adjustments for race, anyway, in which case oversampling blacks won't do anything for you at all.

If you're doing a telephone poll and want to oversample Democrats, you can just call states and area codes where more Democrats live.  Call New York, L.A., Chicago, etc.  You can even call people in Democratic-leaning white areas if you want to mix things up a bit.  That'll do the trick.  Bury it deep enough in the sampling algorithm and maybe nobody will notice!