Remote Research of prototype usability
Global eCommerce provider evaluates developing prototype in Germany & France
Our client is an eCommerce platform provider with a laser sharp focus on customer experience. Their platform is used in numerous markets around the world and that number is growing all the time. With their own UX Research team supporting their requirements in their primary markets, UX24/7 provides Design Research in the secondary markets.
The Client was considering enhancing their software platform in the area of collections and deliveries. In order to inform these advancements, they wanted to understand how consumers in France and Germany currently use collection options when shopping online. They had also created a prototype to demonstrate a potential proposition and wanted to evaluate that. Finally, they wanted the audience to be recruited from across Germany and France so that there was broader representation.
The research contained a number of elements that needed to be considered and carefully planned. The sessions would be a combination of discovery and evaluative research. They would also need to be run remotely in order to be cost and time efficient – traveling around France and Germany was not practical. The participant would be using either a mobile device or laptop, their own, and the prototype would be customised to reflect their actual location to deliver a realistic experience.
An additional dimension was that the client team wanted to view the sessions and so we would need simultaneous translators for each country and the ability to broadcast the English audio channel alongside the session video.
The key project attributes were:
- Viewing the research: we selected a software platform that allowed the participant to share their screen and camera with us on either mobile or laptop. “Observers” from the client team viewed the research live.
- English audio: A separate audio channel using a cloud-calling service broadcast the translator audio. The client team observers could choose whether to listen in local language or the translated English audio.
- Participant recruitment: Recruitment was completed ahead of the research so that the prototype customisation could be completed. We also needed the participant to set themselves up on the research platform ahead of the session and iron out any technical issues.
The screen sharing interface, shown below, allows the participant’s face and screen to be seen. It also provides a chat window where the client team and share questions and make comments.
- Remote Research methodology for usability testing
- Participants located around Germany and France
- Moderator, Client and Interpreter also remote
- Client able to choose whether to listen in local language or English
Following the session moderation, we carried out an initial debrief with the client team about what we had observed that day, collecting their input also. We also provided a summary of the key findings by email so the client team could start work immediately on changes to the prototypes.
These are some of the key areas of insight we uncovered for this project:
- Participants appreciated the flexibility, convenience and freedom that collection options provided to them, often over store pick-ups.
- People preferred collection points that were along their daily routes and selected location then opening times in order of preference.
- Participants moved smoothly and quickly through the prototype without much support.
- There was feedback about the presentation of instructions and the need to ensure they looked authentic
- There was also a slight mismatch of expectations when using the map
Both the French and the German reports contained nearly 50 pages of analysis. They also provided actionable recommendations for improvements to the prototype and future considerations relating to the overall proposition. Alongside the reports we provided the session video both with native language (French or German) or with the translator audio edited in.
Fundamentally, there is little difference methodologically between face-to-face in-person research and face-to-face remote research. There are additional considerations that need to be addressed during participant recruitment and technology requirements to ensure the viewing experience is good. And there are significant benefits to conducting research remotely, such as:
- Lower costs as there is no need for expensive research facilities
- Wider geographical coverage is possible
- The timeframes are more flexible
- It is easier to recruit hard to reach participant profiles