Survey Design: Bias Caused by Question Order & Routing Respondents Through Survey Questions

When developing your survey questionnaire it is good practice to use filter questions to ensure respondents only answers questions relevant to them.  Whether it’s an online survey, kiosk survey, mobile survey or paper survey, the guidelines remain constant.  In the example below of an Online Customer Satisfaction Survey, examples of routing options are shown:

Bias can be caused by the order of codes in closed questions affecting the responses given.

 1)    Left to Right bias

  • If response options appear horizontally, respondents are more likely to select responses on the left than responses on the right.

 2)    Positive to Negative bias

  • In rating scales, respondents are more likely to select positive responses than negative responses

These 2 biasing effects may be used to counteract each other by putting negative responses on the left in horizontal scales.

If the results are used for comparison with other customer satisfaction survey results, be sure the left to right order is consistent between the customer satisfaction questionnaires.

To assess the level of left – right bias, the order may be reversed for half of the sample, and the results from the two halves compared.

Leave a Reply

Current day month ye@r *