Welcome to the Listening Post!

Matrix Question Best Practices

Tweet

 

You've refactored some matrix questions in your survey but you have one that you simply must present as a grid. Keep the following best practices in mind.

• Reduce the number of rows - Remove low incidence topics from the list. What exactly are you going to do with the answers from the 9 beer drinkers who rated Olde Frothingslosh? While I admire your thoroughness, since such data won't be useful anyway, don't clutter your questionnaire trying to collect it.
• Reduce the number of displayed rows - Use an introductory question that asks respondents to select items that they will then rate in a matrix. Then show them a matrix containing only the rows of the items they selected in the prior question. This is much less visually imposing than showing each item with a "don't know/no opinion" choice, especially for long brand lists.

• Get rid of the "Other" row - I've lost this argument with clients many a time, but having a final row that says "Other" is useless. The aggregate averages provide no information, since each respondent will most likely be rating something different (but typically something very important to that individual respondent).
• Don't require answers - As if the matrix question wasn't tedious enough, many survey authors make answering every row required. This assumes that the respondent has opinions about every row; some respondents may not have an opinion about any row. Better to have item non-response error than abandonment.
• Alternate row backgrounds and repeat the header - Most survey software lets you alternate the background color of rows and repeat the table header a certain number of rows (e.g., every 10 rows). This improves data quality by making it easier for respondents to visually line up the radio button they are selecting with its label.
• Randomize the order of rows - The adjacency of topics introduces bias; respondents often narrow their interpretation of a topic based on the topics seen before it.