Asking the Right Research Questions

Preparing research questions

Ask any researcher and they will tell you that you can’t always predict user behaviour from their answers to research questions. Human beings are complex, confusing and often frustrating subjects. They don’t always do what they say they will do; they don’t always mean what they say or say what they mean. They don’t act rationally and consistently; they don’t follow strict patterns of behaviour or activity. All in all they are pretty much of a nightmare to research.

Having said all that, they are all we have when it comes to user research and without it (and them) we cannot make any sort of assessment or predictions on usability, functionality and other aspects of online usage.

So how can you make sure that your research reflects users’ attitudes and intentions as accurately as is humanly possible? Obviously, asking the right people in the first place will give you a significant advantage and the importance of audience identification and sample selection are well-documented and I won’t go into detail on them here. What I would like to deal with is question suitability, accuracy and orientation. In other words: are you asking the right research questions in the right way and with the right language, phrasing, clarity etc?

Psychology is an important aspect of any research project and many researchers have a background in psychology or have studied it in some context or other. The way a research question is framed can often lead a respondent to a particular answer. It might be an answer you like but it might not be true or accurate – so it isn’t really much help to you. What you need is research questions that convey what you are trying to discover: –

• Accurately;
• Simply;
• In understandable language;
• Clearly without ambiguity or potential for misinterpretation;
• Unbiasedly without unintentional (or deliberate) attempts to elicit particular responses;
• Transparently and honestly not leaving users second guessing what you might really be getting at or expect from them.

Do’s and don’ts when putting together research questions on surveys

• Keep questions as short and to the point as possible;
• Use simple language and concepts so as many respondents as possible will understand what you mean;
• Be careful with terminology – if something is capable of more than one interpretation explain exactly what it is you are looking for;
• Don’t ask unnecessary or vague questions – they won’t accomplish anything and simply annoy the respondent;
• Keep the respondent interested and focused by varying the way questions are asked ensuring they don’t just fall into a routine pattern of answer;
• Give respondents the chance to answer ‘not applicable’ or ‘prefer not to answer’ to appropriate questions, otherwise they might simply quit the survey;
• Avoid the use of jargon, technical terminology or any other form of language that is likely not to be understood or might confuse or put off answers;
• Watch out for things like double negatives or questions within questions where what the answer actually means is likely to be unclear;
• Don’t ask leading questions, questions which contain value judgements or which require the respondent to make estimations or hypothesise about imaginary situations – people are generally not very good at dealing with any of these things;
• Test your question set – try it out on the sort of people who are likely to be participating in the research and see how much they understand and how they interpret the questions you have compiled.