Do It Wrong?
Here are some of the most common errors of polling: Sampling error (defined)
Does the sample (defined) really represent the population?)

Oddly enough, says Fritz Scheuren, professor of statistics at George Washington University, "This is the only parameter the polling organization can clearly control."

Harry Truman
Harry Truman AP/Wide World Photos
The most obvious -- and most reliable -- sampling technique is to select respondents in a truly random fashion.

Coverage errors
Does the list used to select the population actually include everybody of interest?

Usually not. In political polling, random dialing of phone numbers gives pretty good coverage since about 95 percent of the population has phones. But affluent people often have two phones, which increases their chance of being picked. (Pollsters can adjust for this factor by asking whether the respondent has two phones). That's the good side. But the "coverage" can also include members of a group (who generally have one bias or another), people in a certain location, etc. One of history's worst polling fiascoes 1936 is blamed on a coverage error.

Response rate errors
(The proportion of potential respondents who actually answered the questionnaire.)

Were the people in the sample too busy, not home, sick of being annoyed on the phone, or using their answering machines to screen calls? Calling back is one way to improve response rate, but it's expensive, and in quick political polls there may not be enough time for it. Still, this may be the most serious problem with polls, Scheuren says. "Many people just don't want to be bothered." With response rates down to about 50 percent for many polls, he says, "you have to assume that the people you didn't reach are the same as the ones you did. It's a serious problem, and getting worse."

Pollsters don't like to reveal their response rates, says James Fishkin of the University of Texas, so they often don't report households that fail to answer their phones as non-respondents. Want to read a really nifty way to get around problems with non-response?

Measurement errors
(What was asked, how was it asked, even the nature of the previous question can all impair accuracy.)

Before you file the results of a poll in the "truth" file, here are some tough questions for the pollsters.

More! home back
NISE/NSF