Why testing hypothesis is the wrong start point
In my recent post about the value of discovery research in saving time and money I touched on usability testing letting teams design. I thought it worth expanding on that in terms of how it manifests in the actual sessions and also, in the preparation. The word “hypothesis” normally enters the conversation quite quickly.
If you are unfamiliar with usability testing, it involves having participants recruited against your target demographic, attempt tasks with your interface, similar to those they would be attempting in the real world. Through observation and moderation we identify issues and make recommendations for improvements. All very straight forward.
The interface we test can be a live website, although that tends to be for optimization, or more commonly a developing prototype. Prototypes are very easy to build now days compared with wireframes that were common in 2000. With some of the tools available and a talented team of designers a prototype can be developed that looks and behaves like the real thing.
Where ‘hypothesis’ comes in, is in the origin of the prototype. Because they are relatively easy to build I am seeing an increasing number of prototypes arrive for usability testing that are built entirely as a result of a teams brainstorming activities. Here’s a quick summary of that process:
- Proposition idea is generated
- Team brainstorm hypothesis of how that idea should be evolved into a design
- Prototype is built
- Usability testing is conducted
This is broadly the way agile teams work: develop MVP (minimum viable product), test it, fail fast, and go again. But if you are not set up in an agile way this approach is flawed. For sure, customers are involved but too late in the process.
Hypothesis should be researched
Prototypes should be tested. Hypothesis should be researched, not tested. If you need convincing, look at the Design Councils Double Diamond framework. The first stage is “Discover” which they define as:
“The first diamond helps people understand, rather than simply assume, what the problem is. It involves speaking to and spending time with people who are affected by the issues.”
It could be argued that testing hypothesis is OK because the team can be open minded about the outcome and go back if necessary. But even if this is the case, surely that is a waste of time and money?
I have also heard people say that they needed the prototype to show the customer what their ideas are. That is to miss the point of discovery research which is to define the problem space that the ideas address. Defining the experience comes later.
If you want to make the most of your budget and time, and build differentiated digital products that deliver an excellent user experience, you need to involve your customers early and throughout the process. Bringing them in late to test your hypothesis will generally result in either a poor product/service or a longer and more costly process.