- The longer the survey, the higher the drop-out rate

- Engagement is very important, and participants might be less interested in a survey that is too long or costs too much time.

- Researchers should be careful about redundant questions and only keep the necessary ones.

- If all questions are important, some of them could be randomly assigned into a subset.


University of Michigan - Ann Arbor

Ohio State University, Columbus

Arizona State University

10 common mistakes in a survey design. Avoiding these errors will make the survey more effective.

10 Ways to Get a Horrible Survey Response Rate

Sauro, J. (2014, February 11). MeasuringU: 10 ways to get a horrible survey response rate. MeasuringU. https://measuringu.com/horrible-responserate/


- Although open questions allow participants to explain their thoughts, they are more time-consuming and require more effort. 

- Open-ended questions should not be the focus of the survey and they should be optional.


Many Open-Ended Questions

- Finding very specific survey participants is difficult. One example of the “impossible screening criteria” from the article is “Looking for women between the ages of 25-34 who bought a washing machine on their smartphone in the last month but have never heard of Maytag.”
- Participants who only meet some of the criteria could still provide efficient responses.


Impossible Screening Criteria


- Listing too many items in a matrix could overwhelm the participants and makes the survey more time-consuming.
- Only listing the necessary items and consider re-group them into different questions.


Giant Matrix Questions

- Adding a progress bar or other types of progress indicator helps participants to keep track of their progress and encourage them to finish the survey.


No Progress Bar or Sense of Completion

- Avoid asking too personal questions, especially at the beginning of the survey. 
- While composing demographic questions, be careful about the wording of the questions and response choices.  


Really Personal Questions Right Upfront

- Although participants might be able to put “N/A” or “Not Applicable” as their responses, too many such questions will overwhelm and discourage the participant and make the survey longer than it needs to be.

Redundant and Irrelevant Questions

- Participants could be asked to answer forced rank questions, but if the question contains too many items, participants could be confused and spend too much time.
- Run a pre-test to track the time and decide if all items need to be kept.
- Consider using the Top-task approach if all items must be included.


Confusing and Challenging Questions


- Example: Please rate your level of agreement to the statements about the role of the Federal Reserve’s recent announcement on monetary policy.

- If given the chance to express their opinion, people will have one, even if it’s uninformed or if people don’t have a strong feeling about the topic. For such questions, participants may simply put N/A or Neutral.


Questions That You Have Little or No Opinion On

- Incentive is not a must-have, and sometimes the best response rates come from leveraging the interest participants have in having their opinion heard.
- However, for long surveys, incentives could attract and encourage participants to finish them.


Learn Before

Related