Optimal Design of Numeric Survey Questions
Have you ever gone to analyze the results of a survey in which you’ve asked for open-ended numeric information (that is, number based fields where respondents type in a number instead of choosing from a drop down list) only to find that many/some of your respondents have entered things like “10-20″ or “about 15?” You’re not alone. While many survey systems work to defeat these “alphanumeric elaborations,” Marak Fuchs published a paper last year in which he discussed the “optimal visual design of open-ended frequency questions in web surveys in order to reduce the percentage of alphanumeric elaborations and explicit extremes to frequency questions.”
Personally, I think it is just as easy to eliminate the problem by using the validation options of whichever survey package I’m using to ensure that respondents get an error message if they try to enter anything other than a number in a numeric field (and shame on any survey packages that don’t offer such validation).
Validation aside, there is something valuable from a semantic/questionnaire design perspective in understanding what can be done to minimize the number of respondents who enter inappropriate values in open-ended question boxes.
For Fuch’s experiment, he tried putting a default value in the box so that respondents could see the desired format. While it didn’t have a substantial impact on the number of people who put in inappropriately formatted values, it did substantially increased the number of people who put pure numbers into the box (that is, while they didn’t guess as much as they did before, they did at least put the numbers in the right format).
My only concern about putting default values such as those that Fuchs described into the text boxes is that it is very difficult to know what to do if the default value is left unchanged in the box. If the respondent leaves the “0″ does it mean they meant to convey a zero or they just never changed the value?
One solution to this challenge that some survey software packages are doing these days is using pre-filled values that disappear after the user clicks in the text box. It isn’t that hard to implement (just a quick bit of Javascript) but I have to imagine it significantly improves the likelihood that respondents will use appropriately formatted values.
Related posts
Related Categories & Tags
learning Tags: American Statistical Association, default values, open-ended-questions, questionnaire-development, survey usability, usability
