The Pros and Cons of Heuristic Evaluation

Heuristics are a set of established, empirical norms or principles that are applied to a discipline or process. They are derived from observation, research and experience over a wide range of projects and usually over a long period of time so they have some cogency and credibility in the field to which they apply.

Heuristic evaluation is a process applied to a project or product that employs these principles to assess how the product performs against a clearly defined set of criteria.

It is conducted by experts in the field who test and review the product with the criteria at the forefront of the process. While this is undoubtedly a useful approach it does have its drawbacks and critics and it is very dependent on a number of value judgements and inputs that may or may not be accurate. Here is a brief look at the pros and cons of heuristic evaluation.

Pros

• It is a detailed, technically sound process that assesses the product against very clear criteria.
• Because it is done by several people there is a better chance of getting a range of views and picking up more potential problem areas.
• The very act of setting up the heuristic evaluation is a useful exercise as it forces you to identify the root elements of the product and focuses development on the main issues.
• There are fewer practical and ethical issues attached to heuristic evaluation as testers are testing in a virtual space.
• Heuristic evaluation tends to focus on fewer, more relevant areas so the problems it identifies tend to be important ones.

Cons

• The evaluation is only as good as the people you get to do it. This means you have to spend a lot of time analysing and reviewing experts to make sure they are relevant and experienced in the issues you are concerned with.
• Another disadvantages of heuristics is that a number of experts are required and this can be time-consuming and expensive to research and set up.
• You are getting opinions and personal observation rather than hard, empirical data from the exercise and the experts’ own background, attitudes, and preferences might colour the verdicts.
• You have to do a good deal of analysis and thinking to make sure you choose the right heuristics in the first place. If this is wrong, no matter how good the experts are, you are likely to get less than optimum results.
• Often the problems identified are not critical (or even real in some cases).

Heuristic evaluations are certainly useful in some instances and can provide crucial insights into how your site is meeting its objectives without the time, expense and potential problems of real user evaluation. It can. However, be risky to rely on it as the sole means of testing your concept and product.

If you would like more information on heuristic evaluation (or any other type of user evaluation) why not ring us on +44(0)800 0246247 or email us at hello@ux247.com