More Good Things Come to Those Who Weight (Weighting With Multiple Variables)

By June 14, 2017How To, Weighting
Are you ready to try weighting with multiple variables?

Last week, we spent a bit of time examining how weighting really works, and how Veracio uses weighting to improve the accuracy of your surveys — if you missed it, check out the first part of Good Things Come to Those Who Weight.

But what happens if you want to dig a little deeper into the population you’re studying? In our previous example, we weighted survey responses by gender to ensure the ratio of male to female respondents accurately represented the community we were examining. Of course, we know that all women won’t share the same views on a particular issue — nor will all men. Perspectives will vary based on many other factors, including (but not limited to):

  • Age
  • Ethnic background
  • Income level
  • Education

So what does that mean for the accuracy of your survey? Not to worry – Veracio uses weighting with multiple variables to ensure an honest representation of what your community thinks or experiences. Let’s take a look at how weighting with multiple variables works, shall we?

Improve Results by Weighting With Multiple Variables

If you thought the math was complicated when we talked about weighting survey responses with a single variable, just wait until you try weighting with multiple variables. The calculations get seriously complex very quickly — but fortunately with Veracio, you can sit back and watch while it does all the heavy lifting.

Using the same example survey we discussed last week, let’s imagine we want to weight answers not only by gender but also by education level. Fortunately, our survey included a question about respondents’ education level, so we can happily task Veracio with weighting by gender, education level, or both combined.

With each additional variable we add to our calculations, the variance will increase and the bias will decrease. In laymen’s terms, it means that our results will be closer to the truth on average, but there will be more variability in the answers.

When we apply weighting with more than one variable, we are adjusting our sample in two directions simultaneously. Let’s compare how are population lines up with our sample:

Survey respondents broken down by two weighting variables

Population broken down by two weighting factors

Just like in our previous single-variable example, we can see that our sample does not accurately reflect the population as a whole. For example:

In the sample, 19 of our 100 respondents are men with low levels of education. In the population, only 15% are men with low education levels. This means we need to weight responses from this group slightly lower than one, so their responses don’t count for more than they should.

Veracio Makes It Easy

From here, we can proceed in the same way as when we weighted with only one variable. When you choose multiple weighting variables in Veracio, you have the option to view your survey results using any combination of the factors you chose. So if you include questions on age, gender, and income level, Veracio can weight your results by a number of factor combinations:

  • Unweighted responses
  • Age only
  • Gender only
  • Income level only
  • Age and gender
  • Age and income level
  • Gender and income level
  • Age, gender, and income level

Of course, this raises a critical dilemma — how will you know which set of results is most accurate?

If you were manually weighting your results, this is the point where you’d probably be crushed under the flood of numbers and calculations you’d have to do, but as I explained How to Choose Weighting Indicators, Veracio can make short work of all your data:

  1. First Veracio takes only gender, and weights 500-1000 bootstrap resamplings of the survey. (This is a test that relies on random sampling with replacement.)

  2. It calculates the average (or mean) and variance (how far the numbers spread out from the mean) of the results using the gender weighting indicator.

  3. Then Veracio repeats this step using only race or ethnicity, and then age. (If Susie had selected other weighting indicators, it would do those too.)

  4. The tool then continues through all the possible combinations of weighting factors.

Starting Collecting More Accurate Survey Results

Congratulations, you’re well on your way to becoming a fully fledged statistical weighting expert. But even if you weren’t, you could still create more accurate, representative survey results using Veracio – get started now!

Do you still have questions we haven’t answered about weighting and online surveys? We’re here to help! Check out our FAQ or get in touch with our survey experts today.


Leave a Reply