Introduction

Every survey is different. While for some surveys you may be looking to gather more in-depth feedback to better understand why people act and feel the way they do, for others you may want to quickly collect as much top-level data as you can to help guide your ongoing decision making.

Therefore, whatever survey you’re planning next, it’s important to choose and create the right survey question types for your needs, if you’re to maximise your response rate and quality of your data.

So, if for your next survey you need to ask as many questions as you can, but in as short a time as possible for your respondents to complete, matrix style questions can offer you a compelling choice.

What are matrix questions?

The matrix question is pretty easy to identify, as it’s a group of multiple-choice questions, which are typically displayed in a grid of rows and columns.

The aim of the matrix question, which belongs to the family of closed-ended questions, is to ask respondents to evaluate one or more row items using the same set of answer choice criteria in the columns provided.

Very often answer choices will be presented as a scale, which requires respondents to select their answer to questions from a range of different rating classification scales. This could involve anything from a ‘1 to 5’ scale rating system, to choosing their answers from a range of options lying anywhere between ‘very dissatisfied’ to ‘very satisfied’.

Pros and cons of matrix questions

As with any survey question, there are both benefits and drawbacks from using matrix questions, so it’s good to be aware of these before you start your survey. That way you can ensure you get the best use out of them and minimise any potential shortcomings from their use.

The advantages

Here’s some key advantages of using matrix questions:

Save space and help shorten your survey: Although it’s a single question, given the fact that a matrix question presents a series of issues for respondents to answer in a single table, this can save space and give the appearance of a shorter survey, which can help reduce drop off rates among survey takers.

Easy for respondents to understand and answer: With the grid style of matrix questions, and only one set of answering criteria to worry about, matrix questions are much simpler and quicker for participants to understand and answer.

Simple to analyse: With only predefined answer sets for respondents to choose from, matrix questions produce clearly defined answers that are easy to analyse, interpret and make decisions with.

Some limitations

Although, we are always keen to know about the benefits, it’s always prudent to be aware of any potential disadvantages before moving forward. And there’s some limitations of using matrix questions that you need to think about.

They can be vulnerable to straight-lining: While offering the same answering criteria for respondents to select from over a series of rows makes it easier for them to answer questions, it can also make these questions vulnerable to abuse.

Straight-lining, where respondents select the same response for each item without careful consideration of what each row is asking them is a good example of this, which can harm the reliability and validity of your survey results.

Incorrect set up can be frustrating for respondents: You must be careful about how you set up matrix questions, if you’re to avoid some of their drawbacks.

If you’ve included too many rows or columns, not only can it be confusing to respondents, but they can also lose interest too, speeding through it without taking due care. You also need to be careful that you’ve optimised these questions for all devices that your respondents could be taking your survey from, particularly smartphones, otherwise it will be difficult for them to see the whole matrix grid, resulting in the same user confusion and frustration.

How to use matrix questions

Similar to regular multiple-choice questions, matrix style questions can be presented as either single-selection or multiple-selection questions.

Single-selection questions

With the single-selection matrix question, an array of possible responses is presented to the respondent in a grid, where only one response can be selected for each row of that grid. Usually this is used for where a single multiple-choice question applies to several related topics.

For example, let’s say you were an organiser of trade exhibitions and regularly ran event surveys, to get feedback from your attendees. For one of your survey questions, you might present them with a matrix question such as the following:

Following your recent exhibition visit, please rate your levels of happiness with each of these criteria.

Multiple-selection questions

In contrast to single-selection, multiple-selection matrix questions provide an array of answers to the respondent, with the respondent able to click multiple answers on each row.

Just as the ‘matrix single answer per row’ question works well to combine multiple connected ‘multiple choice, single answer’ questions together, this ‘matrix multiple-selection’ question type works well to combine many ‘multiple choice, multiple answer’ questions in the same way.

This particular matrix question type can be beneficial for many types of surveys and industries.

Let’s say you worked in marketing. From measuring your brand awareness and testing new campaigns and advertising concepts, to analysing customer loyalty and retention or analysing what your competitors are doing. There can be lots of reasons why you might want to run a marketing survey and many types of survey you could deploy to improve your understanding, with the use of the multiple-selection matrix question playing a big part in this.

For example, if you operated in the mobile phone industry you might employ a multiple-selection matrix style question as part of your competitor research to find out what features people like about your rivals’ products. In such a scenario, one of your questions could look like the following:

Select which features you most like about the following mobile brands?

Matrix with drop-down menu questions

A further option to consider is a matrix with a drop-down menu style question, where respondents have to select their option answer from a series of drop-down menus arranged in a grid.

The advantage of using drop-downs is that they can help conserve screen space. They also prevent users from entering incorrect data, as only legal choices are revealed. And they’re simple to use, as the standard widget they’re based on is one that is familiar with most respondents.

While this question type is quite specialised it can be used in a number of ways. One way in which it could be used is as a feedback survey covering several subjects and topics, with the drop-down menus being used by respondents to reveal the degree to which they agree with what they are being asked.

For example, let’s say you worked in an HR department and ran an employee engagement survey every six months to measure what your staff thought about the encouragement and opportunities you gave them over the past financial year. You could find a matrix with drop-down menu style question a good way of presenting this question, placing a series of questions that you wanted to ask them on the left-hand side of the grid, with a drop down menu of answers available in the columns on the right hand size of your grid for respondents to select from.

So, if you wanted your staff to comment about these two areas in the first two quarters of your current financial year, a matrix with drop-down menu style question could allow you to present this question to them and rate their answer from ‘strongly agree’ to ‘strongly disagree’ as follows:

Looking back on Q1 and Q2, how much do you agree with the following statements?

Best practice when using matrix questions

While you should now have a good basic grasp of matrix questions, their benefits, shortcomings and when it’s best to use them, it’s important to ensure you create and set them up in the right way if you’ve to gain the greatest value from their use.

So, here’s a few best practice considerations to think about when you’re developing your next survey and are thinking about including some matrix questions.

Keep rows and columns to a minimum

There’s a fine balance to providing a question that’s enticing and enjoyable to complete, or risk overwhelming and boring respondents. So, think about limiting your rows and columns to around 5 different options.

Avoid very long questions

Given that you’re working within a table grid format, try to avoid using too many words in your questions, as that could lead to a poorer respondent experience.

Group similar concepts together

Matrix questions work best, when everything that’s discussed is about the same topic area. So, if you’re looking to find out more about your brand perception, keep all your questions related to this subtopic.

Include an opt out category

It may be that some of your respondents are unfamiliar with some of the concepts or questions you’re asking them, so to avoid them abandoning your survey altogether, you many consider including a ‘no opinion’ or ‘neutral’ answer option for them.

‍