START A PETITION 27,000,000 members: the world's largest community for good
2,100,400 people care about Politics

Should We Believe Public Opinion Surveys?

Should We Believe Public Opinion Surveys?

When a new survey tells you that one in six Americans believe that President Obama’s a Muslim, do you believe it? Most of us do, especially if the organization is one of a handful of widely trusted and respected public opinion research centers like Gallup or the Pew Research Center. These polls are important barometers for public opinion and often (as we’ve seen this past week with same-sex marriage) shape the debate around a particular issue.

Should we really trust public opinion surveys, though, given that fewer than 1-in-10 Americans participate in them if contacted? A fascinating new report from the Pew Research Center sheds light on one of the biggest problems that its own researchers face: a falling response rate. At Pew, the response rate of a typical telephone survey was in 1997 was 36%, meaning that 36% of the people who were called at random to participate in the survey actually stayed on the line and answered the questions. Today, the response rate is just 9%, which means that fewer than 1-in-10 Americans stand in for the entire population on the most trusted public opinion surveys. The question then becomes: how representative of everyone else is this tiny minority?

This change is in large part due to the rise of cell phones. Back in 1997, people still relied on landlines, which remain the primary method of reaching respondents for public opinion surveys. Now, large numbers of households have cell phones but no landlines, which meant that polling organizations had to change their methodology to include calls to cell phones, which are generally more expensive for polling field houses to reach. However, cell phones have their problems too: for example, they have caller ID, which allows potential respondents to recognize an unknown number and ignore the call. Many cell phones also belong to people under the age of 18, who cannot participate in most public opinion surveys. Back when landlines were the norm, that wasn’t a problem — children or teens could just give the phone to an adult in the household. Not so with cell phones.

The result is that some populations are more likely to participate in public opinion surveys than others. Senior citizens are more likely to participate than younger Americans, and minorities are more difficult to reach than white Americans. Women also tend to be overrepresented.

Does this skew the survey’s results? Interestingly, Pew found that the people who respond to their surveys are, for the most part, representative of the general population — except for their level of civic engagement. Respondents to Pew surveys are markedly more likely to have contacted a public official in the past year, volunteered for an organization in the past year, or talked to a neighbor in the past week.

There are, of course, no shortage of problems with public opinion surveys. Take, for example, the famous “Bradley effect” — in which white respondents are less likely to express their discomfort with a black candidate to a person on the phone. Nonresponse bias is just one of these issues, and it doesn’t mean that polls are completely unreliable. The fact that Pew took the time to conduct this research indicates that they care about reducing as much of the bias in their surveys as possible. But at the same time, reports like these are important reminders that the people who are responding to surveys are not always representative of the average American — and that every survey should be taken with a grain of salt.

Related Stories:

Is the CBS Poll “Fatally Flawed”?

Poll: Americans Still Support Environmental Protection

McCaskill Leads All Challengers in New Missouri Poll

Read more: , , , ,

Photo Credit: aflcio2008

quick poll

vote now!

Loading poll...

have you shared this story yet?

some of the best people we know are doing it


+ add your own
9:45AM PDT on May 23, 2012

A good friend of mine recently visited relatives in a southern state I won't name, and somehow the talk turned to politics. One of my friend's cousins stated that she definitely will not vote for Pres. Obama--because he's an atheist AND a Moslem! (She did not explain how he could be both, which suggests limited knowledge of both categories.) We agreed: you can't fix stupid.

7:39PM PDT on May 22, 2012

If one in six Americans believes that President Obama is a Muslim, then all that proves is that one in six Americans is stupid and ignorant. I think the people who pay for these surveys are
just trying to find out exactly how many Americans are stupid so they can pander to them for votes.

7:33PM PDT on May 22, 2012

That would depend on who is taking the survey. Surveys are all faked up nowadays so you can't really trust any of them.

6:49PM PDT on May 22, 2012

Pay attention to the questions and the options you have for answering...note that most often the answer you really want to give isn't available. That's a slanted poll. Plus it's always nice to know who's paying for the poll and then follow the money.

4:26PM PDT on May 22, 2012

Since nobody can really verify the results or determine who exactly was asked, what's the point of these polls?

4:25PM PDT on May 22, 2012

Since nobody can really verify the results or determine who exactly was asked, what's the point of these polls?

4:12PM PDT on May 22, 2012

So called "public opinion" polls are bogus. There is really no way to know what part of the population has been polled or if the information is even true. More people should take more interest in the issues and campaigns and follow them more closely throughout the year and not just at the last minute when it's time to vote (if they even vote). There's alot of apathy out there.

2:27PM PDT on May 22, 2012

I generally don't listen to polls and statistical data on the news because most, if not all of it, is concocted, developed and used in so many ways to manipulate the public. Even when it looks/sounds unbiased, if you really dig deep to find out who funded it, it often turns out to be some corporation that will benefit politically/financially from the data, and thus the bias is there right at inception.

10:56AM PDT on May 22, 2012

If instead of opinion polls we could have voter initiatives and referendums like the Swiss do, with the help of IT, America could have as good a government as Switzerland has. It has one of the highest per capita incomes in the world despite having few natural resources and no wars in over 200 years despite being surrounded by warring nations:

Direct Democracy In Switzerland Ch. 16-20

10:47AM PDT on May 22, 2012

public opinions are nothing more than tools to control the masses.

add your comment

Disclaimer: The views expressed above are solely those of the author and may not reflect those of
Care2, Inc., its employees or advertisers.

Care2 - Be Extraordinary - Start a Care2 Petition
ads keep care2 free

Recent Comments from Causes

I didn't need any reminders, but it seems those who are desperately concerned for their own right to…

Al.... Darlene likes to twist words and ignore what omnivores post unless it is to tell us we are wrong…

I am an Australian. I wonder if I can sue this dipstick for defamation or something?

ads keep care2 free

more from causes

Animal Welfare

Causes Canada

Causes UK


Civil Rights


Endangered Wildlife

Environment & Wildlife

Global Development

Global Warming

Health Policy

Human Rights

LGBT rights


Real Food

Trailblazers For Good

Women's Rights

Select names from your address book   |   Help

We hate spam. We do not sell or share the email addresses you provide.

site feedback


Problem on this page? Briefly let us know what isn't working for you and we'll try to make it right!