Measuring The User Experience

Measuring The User Experience

UX research plays a huge part in product design (where your product is a website, app, or other digital platform or program). You want to ensure that when you launch, there are meticulously well-researched reasons for how it looks, operates, navigates, and how you’re going to direct your visitors and users down the funnels that fulfil your goals and targets.

However, performance and research don’t stop on launch day; it just changes shape. In fact, implementation and analysis should never stop. Your product—due to trends, behaviours, progress, and practices—is an ever-evolving and adapting platform, and with that, its goals, interactions, and drivers will need to transform with it.

What’s the difference between product metrics and UX metrics?

To keep things simple, product metrics are the data collected after we take a new system, site or feature live. Our UX metrics were those defined during development that pre-empts the performance and success of the live site.

Test users are hand-picked through the design and build stages to explore performance, practicalities, and preferences. Once a product is live, users are defined by different types of acquisition—through marketing, SEO, advertising, and more. They are real users, and the information you gather provides tangible data you can use to enhance performance, pleasure, interaction, and, most importantly, your bottom line and ROI.

With masses of metrics available to monitor, it’s too easy to try and track everything, because, in one way or another, they all add to the experience. Improving any metric by even 0.1% is a healthy step forward; do it in a hundred different areas, and you’ll see radical gains.

However, as the old design adage goes, ‘put everything in bold, and nothing’s in bold’. You could monitor thousands of metrics, but you can’t monitor them all well. When looking for gains to your bottom line, you need to pick the essential operators and work out how to improve them. You’ve got to know just how to pick your battles.

Choosing your battles: which metrics are important

It’s all about goals. As with any design project, you don’t start with a system and see what good it might achieve; you start with a solution to a problem. What do you need your product to do to make a task easier, more efficient, or improve its user’s life? How will it do it? Then, for the big money, how will it excel beyond what’s already out there?

They’re some of the goals you’ll consider to deliver a valuable, successful product.

And when you’ve determined what your goals are, the most essential UX analytics metrics to affect funnels and performance will form your user experience KPIs (key performance indicators). It’s your KPIs that most often hold all the answers you’re looking for.

How to determine your primary product metrics

If we’re going to uncover how successful you are by your goals, we need a system that determines what they are and how beneficial to your overall analytics they’ll be.

Several standard frameworks are available for measuring the user experience, monitoring efforts, and defining your most important indicators.

The following examples are just a few of the more popular, used by many leading operations.

The Heart framework

This method works on the acronym HEART.

  • H – Happiness – customer satisfaction, etc.
  • E – Engagement – interaction rates, etc.
  • A – Adoption – product access, etc.
  • R – Retention – churn rate, etc.
  • T – Task success – task level satisfaction, etc.

Our first framework follows five categories; each considers the performance of the product through the actions/reactions of its users. By standardising those categories, we can choose the product metrics most relevant to our operation and the goals we should be most interested in pursuing.

The Pirate framework

This method works on the acronym AARRR.

  • A – Acquisition – tracking sources, etc.
  • A – Activation – time taken and ease to interact with the product, etc.
  • R – Retention – longevity, churn rates, etc.
  • R – Referral – recommendation, user satisfaction scores, net promoter scores, etc.
  • R – Revenue – ROI, sales rates, conversions, etc.

The Pirate framework follows the user journey from end to end, monitoring successes through each different phase.

The North Star framework

There’s no easy-to-remember acronym here, but the principle is elementary to understand and follow. This framework focuses on the most essential metric (your key UX KPI) and leading indicator of long-term business success (our North Star) while examining the relationships between it and four or five complementary metrics.

Focussing on a tight group of product metrics helps to eliminate less essential information that can create a distraction from what should be our priorities.

diary

What product metrics should you track?

In an ideal world, you’d have teams of designers and developers tracking every last figure for optimum opportunities to define and determine our best and weakest performers, building on or fixing them to deliver the continual evolution of the product—and the profit.

However, that’s just not practical.

While we can’t monitor everything all of the time, we can use tools that deliver a snapshot of our process and progress—especially utilising alerts that make us aware of sudden spikes, drops, and irregular activity—keeping us aware of any areas of slow decline, sudden shocks, or indicators that something’s gone awry, allowing us to step in and fix them before they escalate into a full-blown disaster.

Those metrics primarily cater to your site or app performance, but not always. Yes, it could be something as practical as slow-loading pages or poor coding; however, it could just as easily be content that creates discord with specific cultures, confusing gateways, or misleading information/direction.

As we said earlier, the essential product metrics relate to your goals. So first, decide what’s important to you and your users and how you monetise your product; the rest will follow.

What product UX metrics can you track?

Not so long ago, without such innovative technology to lean on, most of how we built and operated our businesses was forged on the old ‘hit and hope’ guesswork unless there were undeniable indicators of why we were doing so well (or so badly!).

Now, we can track almost everything. It’s not always easy to convert our qualitative results into numbers, but it’s rarely impossible. The data we can track is virtually endless, so choosing which has the most value is a big part of UX research.

To simplify matters, we can break the data into several categories.

Real user monitoring metrics

We can monitor much of our data through analytics packages and platforms, and you’ll find a real user monitoring system an ultra-handy tool for tracking new product performance. Typically, you’ll have an easy-to-read and navigate dashboard delivering quick and easy access to all kinds of data, showing spikes, errors, times, scores, and more.

KPIS of real user monitoring include:

  • HTTP requests
  • Page load times
  • Crashes
  • Timeouts
  • User satisfaction scores
  • Locations
  • Devices
  • Errors

User engagement metrics

There’s a lot to learn by measuring how engaged users are with our sites and their content. You’ll learn precisely what they’re interested in, what leads to conversions, the areas that keep them the most engaged and those that keep them coming back, and equally important, what turns them off and drives them away.

KPIs of user engagement include:

  • Unique visitors
  • New visitors
  • Returning visitors
  • Page views
  • Time on page
  • Session length
  • Pages per session
  • Conversion rate

Usability metrics

Usability teaches us how users use your website. We can track their entire journey—from where they came from, their stop-offs around your site, the rabbit holes your links took them down, to where and (hopefully) why they leave. We can also track how easy it was for them and how successfully they completed each task—this bank of product metrics measures effectiveness, efficiency, and satisfaction.

KPIs of usability include:

  • Task success rates
  • Time on task
  • Search vs navigation
  • User error rates
  • Task level satisfaction

User adoption metrics

Adoption metrics uncover how often and which features your users engage with, determining the value they hold. This area is critical for product managers to understand how they can improve the system, enhancing customer management.

KPIs of adoption include:

  • Active users – daily, weekly, monthly
  • Stickiness – returning users
  • Product access – daily, weekly, monthly
  • Adoption rates
  • Time with product
  • Feature adoption rates

User retention metrics

Keeping hold of visitors and customers is key to repeat sales, conversions, and delivering great service. Retention involves ensuring they’re happy and enjoying an ongoing ideal customer and user experience.

KPIs of retention include:

  • Churn rate
  • Net promoter score
  • Customer retention rate
  • Average order value
  • Customer lifetime value

Conclusion

How to measure user experience and precisely what we should be measuring is a big topic. However, defining and understanding the affecting factors in each performance area by monitoring metrics opens up all kinds of opportunities to evolve and improve our products and practices.

We started this piece by suggesting UX was a continual best practice for an ever-evolving product; we’ll round things up by hammering home that very same point.

As we’ve just covered, there are plenty of methods and opportunities to measure and make use of all kinds of guiding information. How you use them is down to you and your most important goals.

If you would like to know more about UX and product measurement, email us at hello@ux247.com.

Related Posts