Category

Weighting

We’re using Veracio’s weighting technology to determine just how big the gap between millennials and boomers really is.

Millennials and Boomers: Just How Big is the Generation Gap?

By | Current News and Trends, Weighting | No Comments

It’s a lifestyle news story that practically writes itself: the ever-widening gap between millennials and boomers. New vs. old. Innovation vs. establishment.

But how much of the Millennials and Boomers love-hate story is based on fact? And how much is creative license drawn from one writer’s experience with a cranky old neighbor who wants everyone off his lawn? (Or, in the case of a more mature writer, a selfish millennial staffer who seems oblivious to the way things have always been done?)

The team behind Veracio has been bouncing this argument around for a little while now. And given that we’ve just launched a tool designed to facilitate accurate data collection online, the answer seemed obvious. We should attempt to answer the question scientifically.

Be the Voice for Your Generation

Let us know if your generation is really as bad — or as good — as they say it is. Answer the questions in our 5-minute survey. We’ve developed questions based on a number of stories about significant issues that Millennials and Boomers differ on. All you have to do is take the survey to have your say about how true the stories are.

How Accurate Will It Be?

Veracio weights all survey responses using current US Census data. This means we can ensure the results will be representational — more reliable and accurate than typical “man on the street” interviews often used for opinion polling. Want to know more about how Veracio uses weighting? Check out our blog posts on:

Take the Survey Now

Let your voice be heard! Take our survey, Millennials vs. Boomers – How Different Are They?

Have questions about Veracio or want to use it to create more reliable, representative surveys for your own organization? Veracio is free for everyone to use. Create an account to start building surveys or get in touch with our team today.

 

Accuracy counts. Do you know how to get accurate survey results from your online survey?

Just How Accurate Are Your Survey Results?

By | How To, Improving Your Survey, Weighting | No Comments

Is bad data better than no data at all?

Not really. In fact, inaccurate, unrepresentative data can do much more harm than a complete lack of data. With that in mind, we should all be considering the question: just how accurate are those survey results we’re relying on?

The answer: it depends. On what? A number of factors.

Does this mean you should abandon the idea of using an online survey to gather data? Not at all. It just means you have to use the right tools (and ask the right questions) to ensure you’re getting accurate survey results.

Online Surveys: Pros and Cons

There are upsides and downsides to many tools nonprofits use, and online surveys are no exception. The first step in getting accurate survey results is understanding how online surveys work — including their strengths and their weaknesses.

Pro: Easy and inexpensive to administer

Unlike door-to-door or telephone surveys that require trained staff to administer, an online survey has the “set it and forget it” magic. Once you’ve created and published your survey, there is virtually no work involved in collecting responses.

Con: Text is more easily misinterpreted than a person asking questions

When you’re asking questions face to face, there’s a good chance you’ll notice if the respondent is hesitant to answer or doesn’t understand the question. Not so with a text-based online survey. (This problem can be solved by careful, deliberate survey development. Ensure all questions are clear and easy for respondents to understand.)

Pro: More respondents

Have you attempted a door-to-door or telephone survey lately? The Los Angeles Times found that fewer than 10% of people contacted for telephone surveys completed them. Mobile phones and do not call lists can make it hard to get the responses you need.

Con: Difficult to make online surveys representative

To be truly unbiased, every member of a population must have an equal chance at answering the survey. Digital discrimination means whole communities who lack access to online data are being overlooked. If the group you’re studying doesn’t have reliable internet access, accurate survey results will be impossible to obtain online.

Follow our tips to get more accurate survey results.How to Get Accurate Survey Results Online

Once you understand the pros and cons of using an online survey to collect data, there are a few ways to ensure you get the most accurate survey results possible. Remember to:

Phrase all questions (and answers) in a way that will be clear to those completing the survey.

Clarity in your survey questions is critical, as you won’t have the same opportunity to explain questions you would have in a survey administered in person. Just because a question seems clear to you and your team, doesn’t mean it will seem as simple to your audience. Put yourself in their shoes — consider all possible ways your questions could be interpreted.

Provide an option for respondents who don’t know or aren’t sure of the answer.

Avoid incomplete or inaccurate answers prompted by multiple choice questions that force respondents to choose an unsuitable answer. Always include an option that allows users to indicate the set responses (or indeed, entire questions) don’t apply to them. Think “I don’t know,” “I’m not sure,” or “Not applicable.”

Weight your responses to ensure they represent the population you’re studying.

Weighting your responses to make sure they accurately represent the community you’re gathering data on is a crucial step towards collecting accurate survey results. Of course, weighting can be a lot of extra work — so choose a survey tool that does the heavy lifting for you.

Conduct a pilot test before you send your survey out into the wild.

Share your survey with a small sample group (ideally from the community you plan to administer it to) and check the results. Are you getting the data you need? Are there odd or inexplicable patterns in your results? Do your respondents seem to understand the questions? A little time on analysis of a test version can save you a lot of time and effort if something is not quite right.

Veracio: For More Accurate Survey Results

Are you ready to take your first step out into the world of online surveys or do you want an online survey tool that will provide you with more accurate survey results? Veracio is at your service. Create a free account and you can build surveys that will automatically weight results using US Census data. (International data coming soon!)

Still have questions? We’ll do our best to answer them. Contact our team today!

Are you ready to try weighting with multiple variables?

More Good Things Come to Those Who Weight (Weighting With Multiple Variables)

By | How To, Weighting | No Comments

Last week, we spent a bit of time examining how weighting really works, and how Veracio uses weighting to improve the accuracy of your surveys — if you missed it, check out the first part of Good Things Come to Those Who Weight.

But what happens if you want to dig a little deeper into the population you’re studying? In our previous example, we weighted survey responses by gender to ensure the ratio of male to female respondents accurately represented the community we were examining. Of course, we know that all women won’t share the same views on a particular issue — nor will all men. Perspectives will vary based on many other factors, including (but not limited to):

  • Age
  • Ethnic background
  • Income level
  • Education

So what does that mean for the accuracy of your survey? Not to worry – Veracio uses weighting with multiple variables to ensure an honest representation of what your community thinks or experiences. Let’s take a look at how weighting with multiple variables works, shall we?

Improve Results by Weighting With Multiple Variables

If you thought the math was complicated when we talked about weighting survey responses with a single variable, just wait until you try weighting with multiple variables. The calculations get seriously complex very quickly — but fortunately with Veracio, you can sit back and watch while it does all the heavy lifting.

Using the same example survey we discussed last week, let’s imagine we want to weight answers not only by gender but also by education level. Fortunately, our survey included a question about respondents’ education level, so we can happily task Veracio with weighting by gender, education level, or both combined.

With each additional variable we add to our calculations, the variance will increase and the bias will decrease. In laymen’s terms, it means that our results will be closer to the truth on average, but there will be more variability in the answers.

When we apply weighting with more than one variable, we are adjusting our sample in two directions simultaneously. Let’s compare how are population lines up with our sample:

Survey respondents broken down by two weighting variables

Population broken down by two weighting factors

Just like in our previous single-variable example, we can see that our sample does not accurately reflect the population as a whole. For example:

In the sample, 19 of our 100 respondents are men with low levels of education. In the population, only 15% are men with low education levels. This means we need to weight responses from this group slightly lower than one, so their responses don’t count for more than they should.

Veracio Makes It Easy

From here, we can proceed in the same way as when we weighted with only one variable. When you choose multiple weighting variables in Veracio, you have the option to view your survey results using any combination of the factors you chose. So if you include questions on age, gender, and income level, Veracio can weight your results by a number of factor combinations:

  • Unweighted responses
  • Age only
  • Gender only
  • Income level only
  • Age and gender
  • Age and income level
  • Gender and income level
  • Age, gender, and income level

Of course, this raises a critical dilemma — how will you know which set of results is most accurate?

If you were manually weighting your results, this is the point where you’d probably be crushed under the flood of numbers and calculations you’d have to do, but as I explained How to Choose Weighting Indicators, Veracio can make short work of all your data:

  1. First Veracio takes only gender, and weights 500-1000 bootstrap resamplings of the survey. (This is a test that relies on random sampling with replacement.)

  2. It calculates the average (or mean) and variance (how far the numbers spread out from the mean) of the results using the gender weighting indicator.

  3. Then Veracio repeats this step using only race or ethnicity, and then age. (If Susie had selected other weighting indicators, it would do those too.)

  4. The tool then continues through all the possible combinations of weighting factors.

Starting Collecting More Accurate Survey Results

Congratulations, you’re well on your way to becoming a fully fledged statistical weighting expert. But even if you weren’t, you could still create more accurate, representative survey results using Veracio – get started now!

Do you still have questions we haven’t answered about weighting and online surveys? We’re here to help! Check out our FAQ or get in touch with our survey experts today.