Survey Response Bias: What it is and How to Minimize it
Administering homeowner surveys is inherently invaluable unless the data gathered is accurate. When customer feedback is transparent it provides actionable insights your business can use to improve operations and increase customer satisfaction, among many other benefits. But survey response biases can make accurate and reliable data difficult to collect.
Understanding response bias and how to avoid it can help your company administer better surveys that get better data, so dependable conclusions can be drawn and smart changes made.
What is survey response bias?
Survey response bias is an overarching term that describes different ways survey respondents can be influenced when answering questions. Response bias can be deliberate or subconscious, but in either case, the accuracy of survey results is impacted, thus affecting the reliability of insights drawn from the feedback.
There are a number of response biases that can occur. Here are a few types homebuilders should be aware of when administering homeowner surveys:
Mood bias and emotional mindsets
A homeowner’s mood or mindset can affect how they answer survey questions. If you give them a survey during their honeymoon phase after just moving into their new home, responses will likely be skewed in the positive direction. That’s not to say they wouldn’t give you positive feedback otherwise, but heightened emotions cause strong opinions that often subside as some time goes by.
The same situation is true for negative emotions. If you administer a survey after telling a homeowner construction will take another two months, you can expect their responses to be somewhat skewed in the negative direction.
Remember, the most useful feedback is transparent and representative of a customer’s true experience. If you base your insights on survey responses resulting from an exaggerated emotional state you’re not getting helpful data.
Extreme response bias
Extreme response bias is similar to mood bias but is not based on emotions. Instead, extreme responses, like only choosing ones and fives on a rating scale, are a result of education or culture factors, and most commonly, the wording of a question. Survey questions that use emotive language frequently cause those extreme responses. Emotive language is the use of certain words to evoke emotion and persuade a person to agree with the question or give a higher rating.
Example:
“On a scale of 1-5, rate your experience with Diva Homes, the top-rated homebuilder in Houston.”
This question is intentionally phrased to persuade the survey respondent to give Diva Homes a higher rating by stating the builder is top-rated. While this fact might be true, it should never be included in a question to “remind” a respondent of who the company is. Gathering customer feedback is not about bolstering the company’s reputation, it’s about understanding the homeowner’s personal experience.
Central tendency bias
Central tendency bias is the opposite of mood bias and emotional mindset. It’s the tendency of some people to avoid responding in extreme ways completely. For example, on a scale of 1-5, they will never respond with a one or a five, and will usually stay at the center of the scale for every question. All three’s can be just as untelling about how a homeowner really feels about their experience as extreme responses.
Non-response bias
Non-response bias occurs when the majority of survey respondents are unwilling or unable to respond, which can skew the results towards the perception of those who gave feedback. Generally, the people who did take the time to respond will have a more extreme opinion, which could lead to mindset and emotional bias.
For example, you survey 200 homeowners 30 days after move-in, and only 40 respond. Those 40 homeowners might be really happy with their home and give great feedback across the board, or very upset – they don’t like the landscaping, they’re disappointed with the items left to correct, etc.
In either instance, feedback will be skewed positively or negatively and not necessarily representative of the entire group of 200 homeowners. This makes it difficult to uncover what aspects of your operations are going well and which need improvement – not great for business or customer experience management.
How to minimize response bias and increase feedback accuracy on homeowner surveys
There are several things that you can do to improve the accuracy of survey feedback. Here are just a few:
1. Time your surveys right
To reduce mood bias and emotional mindset, it’s important to send out surveys in a timely manner but not immediately after a key moment in the homebuyer journey. The reason Avid Ratings sends out move-in surveys 30 days after keys are handed over is to allow the excitement to fade a bit, so survey responses are more authentic of the homeowner’s true feelings and perception of their experience. It’s also important not to wait too long because you still want homeowners to accurately remember their experience.
2. Use carefully crafted questions that won’t mislead
Each survey question should have a purpose, but the wording should be void of bias. If you’re writing your own surveys here are a few best practices to ensure wording won’t persuade survey-takers to answer one way or another:
- Ensure none of the questions are leaning towards a negative or positive response.
Example: “How much did you enjoy the responsiveness of your builder?” v.s. “How did you feel about the responsiveness of your builder?”
- Avoid questions that will cause a strong emotional response.
- Avoid asking about multiple topics in one question.
Example: “How was your builder’s level of caring and communication?”
Ideally, you want a third-party to administer your surveys because their survey programs are tested and validated for reliability. This means each question is written to be completely neutral and non-persuasive.
3. Explain why surveys benefit your company
Emphasizing how important surveys are to your company can help homeowners understand why their feedback matters and why they should take the time to answer truthfully, reducing central tendency and non-response biases. Creating survey awareness well before surveys are administered will improve the chances of your homeowners taking your surveys and answering sincerely. This approach works best for survey participants that trust the company and believe that the company will act on their feedback. This should work better for homebuilders than it might for typical B2C businesses because building a home is a long and personal process.
Work with a third-party you can trust.
Avid Insights Team
