Survey pros know the strategies to create a survey that is engaging to the respondent, yet also produces the most accurate and unbiased results. The following strategies are used across the board by survey pros. Read on to find out when to randomize, alphabetize, and use numerical order.
Randomization is one of the best ways to combat some of the downsides of survey research; bias, and speeders (people who complete surveys as quickly as possible). Think of it as shuffling a playlist. When you randomize, you are shuffling the choices so that each respondent sees the list of responses in a different order. This strategy forces the respondent to actually read through all options to select an honest answer, and in addition ensures that bias is spread randomly. When you want to randomize, all you need to do is click on “Randomize Answers” on the top right corner when creating a question.
Randomization is most useful when creating a single choice or grid question. For example: when asking a question about something widely recognized, like a set of popular foods or beverages randomization is very useful.
While randomization is helpful in efforts to remove biases in your data, it is not always applicable, and there are other types of response lists when randomizing is not the best option. For example, avoid randomization when creating a rating scale question, like frequency or importance (not important, slightly important, very important). You’ll want these options to stay in a logistical order.
Along with rating scales, some other lists that should be given a second thought to when randomizing are:
Extensive Response Lists: Some brands lists, depending on the category, can feature a tremendous amount of item choices. For example, the category “Chocolate Candy Bars” can contain enough items that would make it difficult on respondents who are trying to find and select their favorite among a randomized list. In this case, leaving the list in alphabetical order is best.
Chronological Style Response Lists: Similar to scales, some response options that are also best left unrandomized are lists that have a chronological order or flow to the different selections. While we tend to see these more commonly in demographic questions such as “How old are you?” which may contain age ranges such as “18-25” and so on, it is also best to not randomize response lists that appear in questions such as “What time of day would you use this product?” that include response options such as “As soon as I wake up” and “The middle of my day” among other options.
Essentially, alphabetizing provides an organized display of answer options that can reduce confusion for participants. Alphabetization is an additional way to keep choices unbiased as well. Use alphabetization when you have a list of brand names, or a list of options twice “Target Brand Art Supplies, Target Brand Soap, Target Brand Toilet Paper.” See how visually pleasing alphabetization can be:
Response options in numerical order have a number of important uses in survey research. Use numerical order when asking the respondent to write an exact number, like the example below.
Another use for numerical order is for a ranking question. If you want your respondents to rank or compare a number of items from their most to least favorites (for example: Please rank the following types of soda from 1-5, #1 being your favorite).
Test out these strategies by logging into your account today.
Need more info to become a survey pro? Check out our Tutorial and Tips & Tricks pages.