HRoi
Stylish Dinosaur
- Joined
- Dec 28, 2008
- Messages
- 25,315
- Reaction score
- 16,241
I actually am responsible for administering surveys for my company. Online reviews are a slightly different animal but are subject to the same theory. Now I'm not a market research person but I subcontract a lot of them so I think I've learned a bit.
What you're talking about it called response bias, and it doesn't completely invalidate a survey rating. First, response bias works the other way too - really happy people are more likely to answer a survey/give a review as well.
Second, the bias affects the whole industry randomly, so you can assess one company's rating against its peers on a relative basis even with heavy response bias. In other words, a 1 of 5 stars rating may be overly harsh and a product of response bias, but in general it's still valid if all your competitors are 2+.
Third, attempts to influence ratings through bribery generally don't work well unless the survey sample size is very small. That's why I like Tripadvisor because they typically have massive numbers of reviews for a place
Finally, sample size is key. For all the reasons above, one or two reviews are next to useless if you really want a reasonable expectation of the experience you'll get
I actually am responsible for administering surveys for my company. Online reviews are a slightly different animal but are subject to the same theory. Now I'm not a market research person but I subcontract a lot of them so I think I've learned a bit.
What you're talking about it called response bias, and it doesn't completely invalidate a survey rating. First, response bias works the other way too - really happy people are more likely to answer a survey/give a review as well.
Second, the bias affects the whole industry randomly, so you can assess one company's rating against its peers on a relative basis even with heavy response bias. In other words, a 1 of 5 stars rating may be overly harsh and a product of response bias, but in general it's still valid if all your competitors are 2+.
Third, attempts to influence ratings through bribery generally don't work well unless the survey sample size is very small. That's why I like Tripadvisor because they typically have massive numbers of reviews for a place
Finally, sample size is key. For all the reasons above, one or two reviews are next to useless if you really want a reasonable expectation of the experience you'll get
