Biased Survey

To get samples, one way is to take survey

But

.

.

.

Survey can also be biased!

Take this example:

Do you think the government should tear down the parking lot to rebuild a playground so kids can have somewhere to play?

This question is biased because it indirectly guides people to vote yes, else they will assume themselves people who do not care about children. And it will distort the results. So the survey will get more “Yes” than it is supposed to get.

An unbiased question should not try to direct people’s opinion. A more fair and unbiased version of this question would be something such as 

Should government tear down the parking lot? 

 

Another type of biased survey question is the question that automatically catergorizes answers. For example:

Do you like or dislike the decision made by City Concil yesterday regarding the relocation of public restrooms?

Questions like this divide answers into “like” or “dislike”. But it is highly possible that people who are surveyed do not have any feelings about this decision. In other words, they do not “like” or “dislike” decisions. But this question may direct people with this feeling to pick one attitude and thus affect the results. A better rephrase question can be:

How do you like the decision made by City Concil yesterday regarding the relocation of public restrooms?

 

How about let people volunteer to submit their responses? Voluntary responses are made for this. In voluntary reponse, people volunteer to leave their opinions.

For example, Rate My Professor

But as you see above, the opinions can be extremely polarized. There are people who favor a professor so much that they cannot wait to express their loves and thus leaves a high rating.

Or the other way around,someone hates a professor that he or she comes here to leave a extremely negative review even though he or she dropped the class in 2 weeks!

Voluntary response tend to invite people with strong opinions and they may not represent the population’s voice. People who come to voluntary responses are mostly people who are eager to share their voices probably because of some excellent or terrible experiences. The most people, who does not have significant feeling towards certain experiences, are ignored in voluntary responses. But in many cases they make up the most of the population and should be considered as well.

Take a look at this CAPE response. (CAPE is the system at UCSD which collects evaluations of courses and instructors).

Notice the Evals Made is only 34 while Enroll is 133, about 1/3 of the people who enrolled in the class made the evaluation. So the results are depend on only 1/3 of people in this class. Can you completely trust the outcomes now?

So what is a trustworthy voluntary response like?

1. Have high percentage of response. Only 10% or 1,2 reponses are generally not accurate reflection of the population.
2. Reponses contain comprehensive and relatively moderate evaulations. 

Want to read more about biased survey questions? Click here