3 Big No Nos for Rating Scales in Surveys
Surveys provide a rich array of information—when they’re designed well to minimize bias. We at ACET Inc., follow rigorous protocols so that the information revealed in a survey gives your organization helpful information that you can use to become even more effective. We avoid the 3 big no nos for creating rating scales.
1. Not balanced. Carefully designed surveys follow a balance of responses with equally positive and equally negative choices. When surveys are not balanced, the responder becomes influenced by the researcher’s bias before choosing a response. For example, a balanced survey would have options such as Strongly Agree, Agree, Disagree, Strongly Disagree. An unbalanced survey might look like this: Strongly Agree, Agree, Agree Somewhat, Disagree.
2. Having a neutral or an undefined category. Having a neutral or an undefined category (e.g., neither agree or disagree, no opinion) creates measurement error. If you have a neutral or an undefined category, and 80 percent of your responders choose that response, what do you learn? Asking respondents to choose a response that is not neutral will give you results that you can use to make important decisions.
3. Too many options. When you’re presented with tons of options and the options are undefined (e.g., on a scale of 1 to 10), what do you choose? Most people will gravitate toward one of the extremes or to the middle. Researchers say that for most scales, a four-point rating scale will give you helpful results.[1]
Designing rating scales is much more complex than avoiding the 3 big no nos. Many other critical components need to be addressed in addition to creating an unbiased survey instrument.
If you’re interested in having ACET create an evaluation tool with rating scales that will give you effective results, please contact us at info@acetinc.com. We offer a free initial consultation!
[1] Haladyna, T. M. & Rodriguez, M. C. (2013). Developing and validating test items. New York, NY: Routledge, Taylor & Francis Group.
