By Michael Cordeiro
Humans are analytical in choosing between extremes. Too often, though, collected responses do not provide the best insight into the questions being asked. How frequently in data analysis are grid type or comparison questions answered like this:
It’s impossible to make sense of answers like the one above, where each item is answered as the same value.
One point is for certain though; it leaves Researchers begging the question, “What does this mean?”
To help eliminate uncertainty, Alchemer offers users the Max Diff Question Type!
Max Diff encourages respondents to make tough choices. In this blog, we’ll focus on when to use Max Diff, how to build it in Alchemer projects, and making sense of the responses it drives!
What’s the Use?
Max Diff’s goal is to narrow down a respondent’s scope of what they may find most important in a product, service, or solution. We want to limit respondents to choose between tough choices, thus creating reliable data collection.
One example of when a Max Diff question type is valuable exists in a new restaurant creating their menu. They will want to ask potential customers what items seem most and least desirable to sell at their restaurant.
For our example here, let’s focus on how we offer support at Alchemer. How users receive help on their Alchemer questions can come in a variety of ways, and we want to focus on which avenues for support are most desirable and least desirable for users. The avenues that are asked can be seen below:
o Social Media Support
o Email Support
o Live Chat Support
o Phone Support
The Building Blocks
First, we’ll create a brand-new project by selecting Create New from the projects tab:
Then we’ll add a new question, scrolling down in the question type dropdown to the advanced section, clicking Max Diff:
Next, we’ll go ahead and add a question description, as well as some special customizations. Here we use the title, “Which of the following support channels would you most/least desire?”
We will then select the orientation of how the question displays, the locations of the labeling, and define the labels themselves:
After the orientation fields have been determined, scroll down to the attributes section. These are the items respondents will compare against. Our attributes for this example can be seen below:
Once the orientation section is defined and the attributes have been added, we move to the core of the question type! How many options are we going to display per set, and how many sets of attributes are relayed to a respondent.
In our example, we want to display 3 attributes (social media support, email support, etc.) per set. Additionally, we want to display 3 sets of attributes. Populate each area with the number 3 to achieve this:
Let’s save it and test it! Below is a screenshot of the 3 sets a respondent may be displayed during the experience with the question.
Set 1:
Set 2:
Set 3:
Each set includes a different order or answer options available for the respondent to answer. For market researchers, it enables one to make assumptions on the data provided in a logical way.
When examining set 3, we can infer that email support is less desired than both live chat and video conferencing, and live chat is more desirable than the two other options available. In short, we can limit sampling bias by leveraging several options against one another!
Aggregate Reporting on Max Diff
Where Alchemer can really shine with Max Diff is within the reporting capabilities. Each Attribute displays the percentage of times that it was ranked as most appealing, least appealing, or not chosen.
With Standard Reports, a custom chart is generated and displays as seen below:
Moving from left to right, the attribute is ranked by percentage of most desired, least desired, or not chosen. Based on the chart above, we know that social media support is the most desired avenue based on the options provided for respondents.
Difficult Choices Facilitates the Best Analysis
People struggle when it comes to choices. With more options and methods to tackle a singular question, respondents can become overwhelmed which can lead to speeding through a response, survey bias, as well as inaccurate answers. When Max Diff is utilized effectively, Alchemer users gain access to the most concise comparison analysis possible.
For more information on how to use the Max Diff Question Type, Check out our documentation here!
