Moderated Vs Unmoderated Testing
During unmoderated usability testing, there is no direction or guidance given to the users for the duration of the testing. The users are given the task, complete it by themselves and the results are analysed without reference to their thought process or actions. This method is sometimes not favoured over moderated testing due to the fear of losing control of the process and not gaining any useful data through the users going off-track or getting bogged down in details.
Despite the apparent drawbacks, unmoderated testing does have a place and there are some specific benefits associated with the technique (and some situations where it is the only reliable way to get accurate, unbiased data).
For example, when testing an early prototype or quite a simple task-set, you might want to see the results of unguided actions to determine how successful you have been in transmitting simplicity or effect. You might also want to compare results from moderated and unmoderated tests to see to what extent guidance and help are useful and to what degree users can work stuff out for themselves.
Here are some of the potential benefits you can gain from unmoderated testing;
- It is cheaper with no need to engage an expensive professional moderator and can be administered more easily;
- Recruitment is easier as the process is less place-, time- and user-specific.
- Less bias as there is no outside agency to introduce their own views or steer to users and the observer effect (this is the perception that people act differently when they know they are being watched) does not apply as there are no observers.
- Testing can be quick and can be incremental, building on the results of one trial to build another more accurately, and so on until the issue is honed to the necessary degree.
There are, however, some potential shortcomings of the methodology which you need to be aware of before making any decisions on using it.
- There is no opportunity for dealing with unexpected problems or actions by the user. If they go off on the wrong track or get stymied by something you hadn’t anticipated the whole test might be worthless.
- There is no flexibility in the process and therefore no opportunities for other learning outside of the allotted task.
- It is likely to be unsuitable for complex or extensive testing due to the lack of opportunity to provide help or feedback.
- In the absence of context and observation some of the analysis of findings might be hard to interpret.
The idea of unmoderated testing should not be dismissed out of hand; and, as observed earlier, there might even be circumstances where it is desirable. But it does have shortcomings and you need to assess the impact of these limitations before embarking on this course.
If you would like to discuss this method, or any other aspect of usability testing, why not give us a ring on +44(0)800 0246 24 or email us at hello@ux247.com for a discussion of your possible needs.
