20 Questions A Journalist Should Ask About Poll Results

Third Edition
Sheldon R. Gawiser, Ph.D. and G. Evans Witt

Polls provide the best direct source of information about public opinion. They are valuable tools for journalists and can serve as the basis for accurate, informative news stories. For the journalist looking at a set of poll numbers, here are the 20 questions to ask the pollster before reporting any results. This publication is designed to help working journalists do a thorough, professional job covering polls. It is not a primer on how to conduct a public opinion survey.

The only polls that should be reported are "scientific" polls. A number of the questions here will help you decide whether or not a poll is a "scientific" one worthy of coverage – or an unscientific survey without value.

Unscientific pseudo-polls are widespread and sometimes entertaining, but they never provide the kind of information that belongs in a serious report. Examples include 900-number call-in polls, man-on-the-street surveys, many Internet polls, shopping mall polls, and even the classic toilet tissue poll featuring pictures of the candidates on each roll.

One major distinguishing difference between scientific and unscientific polls is who picks the respondents for the survey. In a scientific poll, the pollster identifies and seeks out the people to be interviewed. In an unscientific poll, the respondents usually "volunteer" their opinions, selecting themselves for the poll.

The results of the well-conducted scientific poll provide a reliable guide to the opinions of many people in addition to those interviewed – even the opinions of all Americans. The results of an unscientific poll tell you nothing beyond simply what those respondents say.

By asking these 20 questions, the journalist can seek the facts to decide how to report any poll that comes across the news desk.

The authors wish to thank the officers, trustees and members of the National Council on Public Polls for their editing assistance and their support.

1. Who did the poll?

2. Who paid for the poll and why was it done?

3. How many people were interviewed for the survey?

4. How were those people chosen?

5. What area (nation, state, or region) or what group (teachers,lawyers, Democratic voters, etc.) were these people chosen from?

6. Are the results based on the answers of all the people interviewed?

7. Who should have been interviewed and was not? Or do response rates matter?

8. When was the poll done?

9. How were the interviews conducted?

10. What about polls on the Internet or World Wide Web?

11. What is the sampling error for the poll results?

12. Who’s on first?

13. What other kinds of factors can skew poll results?

14. What questions were asked?

15. In what order were the questions asked?

16. What about "push polls?"

17. What other polls have been done on this topic? Do they say the same thing? If they are different, why are they different?

18. What about exit polls?

19. What else needs to be included in the report of the poll?

20. So I've asked all the questions. The answers sound good. Should we report the results?

Gawiser, S. R. and Witt, G.E. 2006. 20 Questions a Journalist Should Ask about Poll Results. National Council on Public Polls. Accessed at http://www.ncpp.org/?q=node/4 on January, 10, 2007.