Carry That Weight: How to Choose Weighting Indicators

The best way to select the right weighting indicators for your survey is to understand what they are and how Veracio choose which ones to use.

How do I choose the best set of weighting indicators for my survey?

It’s a question I hear a lot. The best way to select the right weighting indicators for your survey is to understand what they are and how Veracio choose which ones to use. Let’s look at an example:

Susie is a team member at a social service organization. She’s been charged with finding out if the good people of her state feel her team’s efforts are achieving something. Susie hops online and creates a survey with a single question:

Do you approve of the way our organization is handling X issue?

Respondents are given two (obvious) options: Yes, they approve or no, they do not. All seems simple enough so far, right?

These raw survey results show strong approval on the survey question before weighting.

The Role of Weighting Indicators

Now that we’ve dreamed up Susie and her survey, let’s also imagine the survey respondents are biased. What does that mean?

It means their answers do not represent the views of the state’s population because they do not share the same demographic characteristics.

Maybe more women than men took the survey, and more white community members than people of color. Susie could not predict how respondents would be biased because she did not know in advance which residents would actually complete the survey.

By weighting the responses, Susie can eliminate the bias in her results. Of course, the best demographic factors (or set of factors) to use as weighting indicators for your responses can’t be determined without first knowing the demographic factors of those who completed the survey.

How Veracio Uses Weighting

Let’s assume Susie created her online survey using Veracio. (She’s a smart one, that Susie!) During the survey creation process, Susie asked respondents for their gender, race or ethnicity, and age.

Now Veracio can weight the responses for her using every combination of demographic information to find the most accurate, least biased results.

  1. First Veracio takes only gender, and weights 500-1000 bootstrap resamplings of the survey. (This is a test that relies on random sampling with replacement.)
  2. It calculates the average (or mean) and variance (how far the numbers spread out from the mean) of the results using the gender weighting indicator.
  3. Then Veracio repeats this step using only race or ethnicity, and then age. (If Susie had selected other weighting indicators, it would do those too.)
  4. The tool then continues through all the possible combinations of weighting factors.

Possible weighting indicators include gender, race or ethnicity, age, any combination of two of them, or all three.

Our simulations show that the combination of weighting indicators that results in the lowest variance is the one that matches the real bias in the survey most closely. In simpler terms, the combination of weighting factors that provides the result most consistent with the mean is the most accurate. In Susie’s case, using the weighting indicators of gender and race or ethnicity matched the real bias most closely – providing her team with the most accurate picture of how people in their state felt about their work.

After applying weighting indicators, we see a noticeable difference in results.

Ready to Get More Accurate Survey Results?

Veracio was purpose-built to take the work out of collecting accurate data via online surveys. But you don’t have to take our word for it! You can try it yourself for free.

Create an account now to start constructing your own automatically weighted surveys. Have questions? Check out our frequently asked questions, or get in touch with our team.

Join the discussion 2 Comments

Leave a Reply