One of the most common problems with survey research is sampling errors. Let’s talk a little about this problem and why it matters in terms of you choosing to participate in Town Opinions database.
Sampling
First of all, let’s understand sampling. Sampling is taking a whole population that you are wanting to study and selecting a smaller amount to actually survey.
Why wouldn’t you just ask everyone in the population to answer the survey? Traditionally when large groups are involved this isn’t done for many reasons: it is expensive, it is hard, and it is not necessary.
Why is it not necessary? It is not necessary because you can get similar accurate responses from a small part of the whole population, if the sample is representative of the whole population.
Is the sample representative of the population?
There are two main types of problems that could happen when using a sample. They are called: Coverage Error and Nonresponse Bias
Coverage Error is when, due to how researchers go about finding their sample, they accidentally exclude certain unique groups of the population. Sometimes they sample based on convenience, say what is close to them. For example, most social science research is performed on students, because it is more convenient to include them.
The following graphic is an example of a coverage error. I want to know about men’s facial hair. I don’t want to survey the entire population, so I survey a portion of the population or 20 % of the population. I surveyed those closest to me and found that:
40% of the population has scraggly beards
20% have old school mustaches
20% have pink mustaches
20% have red weird old goatees
When in reality it is as follows:
20% of the population has dirty blonde scraggly beards
20% have light brown old school mustaches
20% have pink fluffy mustaches
20% have red weird goatees
20% have black barbershop mustaches
In order to combat this we do the following:
- Offer the option to participate via email, phone, or mail. Maybe all of the black barbershop mustaches didn’t have phones and that was why they weren’t included in the sample. We want to avoid this.
- We ask for you to include your demographic information in your profile and that you update it regularly. This will help us check to make sure our samples are representative of the population.
- We are trying to get the word out by several different methods about participation so that we make sure not to exclude any one group of participants. You can help by spreading the word to those around you once you are signed up yourself.
Nonresponse Bias is when the sample isn’t representative because those who don’t participate have certain characteristics about them that incline them to not participate. This error is found when the survey or invitation to participate is given to a representative sample however a certain group or people with certain characteristics don’t respond.
The reverse of this would be they have characteristics that would incline them to participate. And only a certain type of person responds.
The following graphic is an example of nonresponse bias. The invitation to participate is sent out to 25 people. But I only get five responses back. I want to know about the use of hair products on men with facial hair. Even though my survey was sent out to a sample that was representative of the population, only a certain type of participant responded. Most likely my answers will not be representative of the whole population because no one with pink or brown hair responded, no body with a fluffy or more natural mustache responded, and the responses are heavily weighted toward people with black mustaches and red goatees.
In order to combat this we do the following:
- We ask that you respond to every questionnaire that you receive even if you aren’t interested in the topic.
- We will try to sample in a way so that you don’t have to answer very many surveys in a year.
- We ask for you to include your demographic information in your profile and that you update it regularly.
It is essential that the research that is used in board meetings or city councils or state legislatures or anywhere where it will be used to make decisions about our communities is accurate and truly reflects the population not just those who are closest to the researcher or more likely to have time to respond to research.