Common survey creation mistakes
We have collected 14 frequently made mistakes that can cause great damage to the quality of the data collected by our survey.
1. Don't make it too long or boring
Both can lead to inaccurate answers. If the person filling in just wants to get through the whole thing, he will neither read the questions and answer options carefully, nor think them through properly.
RECOMMENDATION: The more beautiful you look, the more images and videos you use. Interesting questions - obviously to the extent possible. If we know for sure that it has turned out to be long, we recommend interrupting the filling in the introduction and then completing it in several parts. The system provides a way to interrupt and then continue the filling. The filler can interrupt the filling process at any time. If you open the page again in the same browser, you can continue the survey where you left off.
2. Complicated or ambiguous wording
Since the text of the survey must mean the same thing to the creator of the survey and to all those who complete it, you can only use language and expressions that achieve this.
RECOMMENDATION: Use wording that a 12-year-old child can surely understand.
3. Not specific enough, imprecise wording
Don't take it for granted that a question means the same to the other person as it does to us. Always try to be as precise as possible. For example, this question: "How many children are there in your family?" leaves more options open. Do you mean only children living in the same household as the respondent? Should only those under the age of 18 be included, or should the applicant's older children also be included?
SUGGESTION: Think carefully about each question. In the best case, testing should reveal these errors. Staying with the above example, this is more precise: "How many children under the age of 18 live in your household?"
4. Let's not force an answer!
Give us the option if the interviewer does not want to answer something. Better to lose an answer than a filler.
SUGGESTION: Enter "I don't want to answer!" as an option, or set it so that you can continue even if you leave the question blank.
5. Filtering, or conditional display of questions
With an online survey, it is very easy to set up so that the person filling in only fills in the questions that really concern him. If we ask whether he likes coffee, the following questions can be hidden or displayed according to his answer. That way, if you don't drink coffee, you won't lose patience with the following 5 questions about your coffee habits...
RECOMMENDATION: At every point where it is possible, attach the display of the question to a condition.
6. Emotional manipulation, asking suggestive questions
It is possible to suggest the expected answer with a single question, but it is also a mistake if several questions follow one after the other, which prompt the person to answer not the truth, but what his conscience suggests or perhaps expects from himself. Any form of both manipulation and suggestion will distort the results.
SUGGESTION: Try to write simple and open questions. We should never start like this: "Don't you think so..."
7. Lack of gradual structure
Always progress from the simple to the more complex. This way, the fill-in can also tune in and understand the questions more easily.
SUGGESTION: After simple questions, let's move on to increasingly complex ones. After simpler topics, only the more difficult ones should come.
8. Absence of filling instructions
Where necessary, prepare instructions for filling in or add comments to individual questions.
RECOMMENDATION: If it is a general thing that applies to the entire survey, it should be described in the introductory part. If only some blocks or questions need an explanation for some reason, we will only describe it before the given question or block.
9. Too many or complicated answer options
If we give too many or too complicated options after a question, it is possible that the person filling in - especially if this happens more than once - loses patience and chooses the first acceptable option.
SUGGESTION: After taking the test, interview some people. During the interview, we ask the questions of the survey live. This enables us to receive specific feedback on some questions. In this way, we can either simplify the options offered, or simply omit the unnecessary ones.
10. Too few answer options
If the fill-in does not find the answer option that suits him, he will choose the least bad. This distorts the results obtained.
SUGGESTION: Always give an option that will get out of this situation. For example: "Other", "None.", "I don't know.", "I don't want to answer."
11. Too many good answer options
It is also a problem if the person filling in considers several options to be equally good, but can only mark one. This also distorts the results, because he will simply choose only one of them - because we forced him to.
SUGGESTION: Set it so that more options can be selected.
12. Nesting multiple questions
Many times a closed and an open question are crowded together. This can cause problems when analyzing the data later.
SUGGESTION: If we have such a question, then break it down into elementary questions.
13. Skip verification questions
If there is an answer whose truthfulness is particularly important, it is a good idea to check whether the person filling in answered honestly.
SUGGESTION: Be sure to ask it in a different block compared to the original question. Let's try to act as discreetly and carefully as possible.
14. Failure to test
Many times we don't even bother to go through the survey once, this is a serious mistake.
SUGGESTION: We write more about this in the testing article. It is worth testing the survey in 3 phases:
- View Survey Preview
- Personal interviews
- Online filling with 2-3 testers