8 week diary study during beta launch
O2 uses diary study with real customers using beta version of new app
O2 is the second largest mobile network operator in the UK with 25 million subscribers. They are the principal commercial brand of Telefónica UK, which is part of the global telecommunications group Telefónica S.A, headquartered in Spain and operating in Europe, and North, Central and South America.
UX24/7 was commissioned by O2/Telefónica to facilitate a remotely moderated diary study evaluation of the new all-in-one O2 Ask app. The study aimed to capture the behaviour of 60 respondents who were all O2 customers at the time.
The diary study was specifically designed to gain insight from real users, looking into their recorded daily habits. Additional assignments were created to look further into:
- Bills and payments
- Purchasing a bolt-on
- Using the new in-app assistant
- Making use of 4G and Wi-Fi calling
Some of the specific objectives of this research were looking at customer expectations of the app and whether they were satisfied. The study will assist in prioritising feature development, based on identified customer needs, and looking at what local customisations could be made to improve the experience for UK customers.
UX24/7 contacted and recruited 60 participants via a study recruitment agency. 56 individuals completed the full eight-week study. Prior to the study, respondents were briefed and asked to download two apps to be able to take part. These apps were the new O2 Ask app and Nativeye, to feedback their thoughts and feelings.
Respondents needed to complete seven assignments in total, spread over the duration of the study. The last assignment to complete was an evaluation assignment, to help gain overall thoughts and feelings towards the O2 Ask app.
Participants were aged between 25 and 65 and were required to be My O2 app users, which was the predecessor to the new O2 Ask app. Twenty were to have joined O2 within the past 12 months, 20 to have been with O2 for at least 3 years and 20 to be out of their contract and due for renewal.
QUICK FACTS
- 8-week remote diary study
- 57 participants completed study – 85% completion rate
- Digital data capture of day to day activities
- Multi-media deliverable
During the sign up process, 96% of the participants found the method and time it took to be reasonable, but only 56% of the participants found the introductory information to help them get started.
Initial feedback on the app highlighted positive brand recognition and alignment with the expected look and feel of the brand. The app also offered a more detailed level of account information compared to My O2. However, the iconography was not immediately clear to users which meant they were unsure what each of the bottom dock navigation options would give them.
A common first reaction to the app was that it had more information than its predecessor and showed account allowances upfront. When comparing the apps, 67% of participants thought that the new O2 Ask app was better. The most popular uses of the app were to check data allowance, accrued charges and billing related information.
Participants found their desired information relatively easy to locate but expected the digital assistant to assist across the breadth of the application, rather than be restricted to billing related enquiries. Users also found that the digital assistant was easy to locate as a feature owing to the central positioning of the icon. Although responses for questions around billing were positively received, other topics were a struggle for the assistant to handle.
Some also struggled to find information telling them if they were set up for international roaming or not. This resulted in a large number of participants giving up and stating they were unable to find the information.
Having usage and billing information shown upfront on the dashboard is useful but the minimalistic visual design could be revisited to avoid confusion and include more contextually relevant actions. Using stronger visuals to relay introductory information could also help first time users familiarise themselves with basic controls.
The digital assistant came across as an underdeveloped aspect of the app and could use a review to make it more promising as a trouble shooting assistant. Adding a ‘Roaming’ option in the account section would also be helpful to give customers quick-view summary information regarding using their phone abroad.
Introducing the ability to personalise the in-app assistant, like Apple’s ‘Siri’, where you can change the gender and accent of the digital assistant would help to give it a more personal touch.
Informing customers of the benefits of Wi-Fi calling and messaging would help increase the uptake of these features and would encourage O2 customers to use the app more. By the end of the study, 81% off the participants said they were likely to keep using the O2 Ask app.
UX24/7 also recommended planning further qualitative research focusing on end-to-end in-app user journeys.
