More people than ever connect to the internet via mobile. As a result, we carry out more searches, use more apps, and spend more time socialising, shopping, and playing while staring into our smallest screens than on any other device.
Mobile screen sizes demand the streamlining of complex navigation banners, information presentation, actions and processes, so these simpler-looking yet complex platforms must operate seamlessly and intuitively.
How do we know if they work as well as they possibly can? Well, we test them with the people most likely to use them.
What is mobile usability testing?
Usability determines the ease of operation of a digital or physical product.
Mobile usability testing shows how our intended users perform necessary tasks and the ease and pleasure they achieve from each process. It caters to all mobile operations, whether an app, a website, a new feature, or other tools.
In UX research, we test through all stages, from prototypes to pre and post-launch products. Re-testing at as many points as necessary ensures the best performance.
Why perform mobile usability testing?
The short, sharp answer: to detect usability issues.
Through testing, we improve the UX of the app, website, or function, suited to what real users want, need, and expect. To understand how users interact with our digital products, we design specific tests to see how they perform, revealing where they’re working well and where they’re letting users down.
These tests show us which functions, stages, or areas are confusing or frustrating to use so that we can help stakeholders, designers, and developers create the app they expect, one that delivers an irresistible product their customers find effortless to operate and can’t put down.
It helps us hone in on where users expect to find information, where they look, and how. We also get to see how they feel when they run into brick walls and dead ends and what they do then.
We all love a little praise when we’ve done something well. Yet, the real value in usability testing is uncovering the stumbling blocks and pitfalls our users may face, delivering an opportunity to make far better products.
Key components and stages of mobile usability testing
Setting goals and objectives
We’ve said it many times: without goals or objectives, you’re testing for testing’s sake.
This is as true as ever for usability testing. In mobile testing, we’re exploring the most utilised devices in the market, so hitting the sweet spot with your users is essential.
Whether it’s a prototype, a native OS product, a new feature or a website, if you don’t compile the goals you expect your users to comfortably achieve and understand why they’d want to perform them, there’s no qualified substance behind your process.
It’s good practice to base testing tasks around specific objectives, creating scenarios that make those operations more realistic and natural. If you order those tasks and scenarios in a realistic sequence, you’ll ensure each session flows as your app operation would be expected during typical use.
Documentation
You need documentation for any testing, so it’s crucial to get it right. For example, you’ll need to provide consent forms and questionnaires, even contracts where payment is offered as a reward.
Your questionnaires must be well written, straightforward, easy to understand, and easy for your participants to provide the information you want.
Setting up tasks
Planning and designing your test methods are where the meat of testing lies. Each task should be considered and crafted to provide the necessary information required from each objective.
There are many testing platforms, each including various ways of gathering your essential data. To monitor and measure each task, you’ll need the appropriate equipment.
- Mobile recorders – Screen recording software or specialist data-capturing video options.
- Screen-capture software – Catering to app operation on-screen and the front-facing camera capturing user emotions and expressions.
- Mirroring – Monitoring user operation by projecting it onto the moderator’s observation screens.
- Camera cradles and slings – With mobile usability testing, camera set-up can be complicated. Specialist hardware is available to hold the mobile phone with a fixed observing camera as a single, fixed unit.
- Webcams – Additional static cameras to record participants’ reactions, expressions, and emotions.
- Extended charging cables – Battery levels can be an issue with prolonged testing and power-hungry applications
- Eye-tracking – Specialist software that monitors where users focus on the screen.
It’s also necessary to conduct your usability testing on mobile devices belonging to the participants. They should have owned them for at least three months to ensure they’re proficient with the operating system and device. Where participants are unacquainted with a device, it can deliver false results down to issues with the basic mobile operation instead of the app or website under scrutiny.
Participant selection
As with any digital product test, the results are only relevant if the users fit the demographics and characteristics of your end user. You won’t gather the data you need from a disinterested or irrelevant participant.
You may already hold a suitable pool of test subjects; otherwise, you can advertise for participants or procure them from a professional service. However, buying in a participant pool leaves you vulnerable to low-quality users or expensive acquisitions. In addition, costs can rise significantly where products are aimed at niche users and industries, where testers may come at a much higher price per head.
Then there’s availability. If you’re conducting lab testing, your participants must be in a suitable location. If you’re operating virtual testing using video calls, you must ensure users have the tools to relay their operations back to you. If they’re international participants, time zones and language barriers are also necessary considerations.
Moderator function
Ensuring the moderator is well-versed in objectives, tests, and procedure is essential. Ideally, it should be a usability specialist, part of the project’s UX team. They’ll clearly understand what each test should achieve and the data necessary to validate (or not) each goal. In addition, they should be able to observe how participants behave and operate the app, site, tool, or feature in real time to ensure correct operation.
Conducting the test
The following shows a generic process for a typical mobile usability test.
- Welcome and documentation
- Pre-test questionnaire
- Pre-test interview
- Performing the tasks
- Post-test questionnaire
- Post-test interview
- Debrief
Pre- and post-test questionnaires and interviews allow you to gather data that your practical tasks might not uncover. During the debrief, you can explain what will happen with the user’s data and whether they might be expected to participate in further tests.
There are all kinds of concerns you may want to consider, but the following are some of the most essential.
Pilot testing – Before you get anywhere near inviting participants to take part, you should perform in-depth pilot testing of the entire process. It uncovers any technology issues, poorly worded instructions, areas of confusion, and unforeseen pitfalls and hiccups in your approach.
Moderated vs unmoderated testing – For most usability testing, it’s better to have a moderator present to ensure the flow of the process while monitoring each user during operation. However, an unmoderated study can be adequate for less demanding goals and objectives. Plenty of tools deliver this type of testing, lending itself to larger test pools and extensive data.
Lab testing vs contextual inquiry – Depending on your goals and objectives, it could also benefit you to test a product on location, monitoring any changes in behaviour it brings and delivering the most relevant data. However, with lab testing, setting up the necessary tools and ensuring perfect operation before testing begins is far more manageable.
Quantifying the results
With the testing complete, it’s time to examine the data. What you learn from your participants will be responsible for the changes your stakeholders and designers must make. Therefore, it is essential to document it in easy-to-understand visuals with clear, concise explanations.
- Essential background information – What was tested, where, and how?
- Methodology – Test scenarios, how observations were carried out, plus participant demographic data.
- Results – The type of data compiled and what it uncovered.
- Actionable outcomes – Including solutions, their costs, and timeframes
Delivering documentation
Longwinded, text-heavy documents aren’t always the easiest to follow, even those with plenty of illustrations, screenshots, graphs, and charts. A presentation allows more opportunity for the clearest delivery and to focus attention on key findings and their resolutions.
A presentation offers ample opportunity to show sections of video that testing has captured, showing precisely where users experienced problems and their reactions. It also allows you to open the meeting to a Q&A discussing details and where you should head next.
Conclusion
Regarding usability testing, mobile operation is crucial to ensuring the performance of our websites and apps. With more people than ever accessing the internet using a mobile device—with less-advantaged countries operating on little else—testing at each stage, from early prototypes to market-ready products, is essential.
By paying close attention to your test participants, you can deliver the experience that they, and thousands like them, will continue to enjoy throughout your product’s lifetime.
If you would like help with your mobile usability testing requirement from UX24/7, email us at hello@ux247.com.
