It has been more than 12 months since we last conducted design research in a usability lab facility due to the ongoing global pandemic. Prior to the pandemic about 20% of our research was run remotely with the majority of the balance lab-based – there was some contextual research in the field. Even thought we were pretty good at remote research we have become much better as a result of doing it all the time.
Plenty has changed:
- The technology platforms we use
- How we handle quotes and clips
- How we ask participants to join
- Our systems and processes
- Our handling of GDPR
- The way we pay incentives
I could go on. With all that we have learnt, is there any reason to go back to the lab?
What have we gained by going remote?
There are a number of significant advantages that have been gained as a result of design research being delivered online. The benefits are for both customers and researchers and I think the main ones are:
- No need to travel.
- Not being constrained by lab session hours.
- Participants joining from their own home/workplace so they are more relaxed.
- Participants using their own devices, that they are a) familiar with and b) represent their real world.
- Fewer no-shows and easier to reschedule when it occurs.
- Lower cost – no lab and catering fees.
- Easier to recruit hard to reach participant profiles.
There are more but I think these represent the biggest benefits.
What have we lost by not being in the usability lab?
There are a couple of things we thought we would lose that we haven’t. Seeing and recording facial expressions when participants are on a mobile device was a real problem. But innovation through lockdown has given us new ways to mitigate this that are entirely effective.
There are two big losses that I think will ensure labs return to some degree. These are:
Team collaboration in the viewing room: I know some clients that have not carried out research during the pandemic because they value the experience of being in the lab, in market so greatly. Others that have but can’t wait to be able to have the team together for a day collaborating away from the office on what they are seeing in the research. We have had one client during the pandemic who did this quite well remotely, but only one. This will be a key reason for returning to labs.
Researching highly sensitive or confidential prototypes: Quite often the client needs to control how and where their prototype is being used during the research. This can be if it is a particularly innovative solution or includes confidential information. Even though it is possible to run research remotely with these types of assets (it can be very hard) the lack of control may not be satisfactory to the client. Only a lab, where they can ask us to use their device and thereby control the distribution of the asset is sufficient to alleviate their concerns.
What are the wider risks?
I think there are wider risks with remote research that are similar to what we have seen since the emergence of test platforms (like Usertesting.com). These platforms provide easy access to testing and over the past decade have grown in use. They are brilliant at what they do and we use them ourselves, often on behalf of customers.
Frequently, clients ask us to look at what they are doing with an online testing platform. They ask because they have concerns over the way the tool is being used and the decisions they are basing on the data. Very often, almost always in fact, what we find is that it isn’t being used properly: rubbish in = rubbish out.
I think there is a risk that remote research is similarly easy to not carry out properly. If conducting mobile research and you can’t see the participant, only the interface, sessions could descend into a basic Q&A – not observation. If the questions are not then structured well, the results (not insight) are worthless.
How much usability lab research do I think we will do?
I don’t think we will get back to the proportion of usability lab research we did before the pandemic, but I do think it will return. My guess is we will do about 30-40% of research in the lab with the balance remote or in the field. This split could be driven by UX24/7 being a global design research agency where a large part of what we do is multi-market studies. These lend themselves to remote research for all sorts of reasons with cost being chief among them.