Formative and Summative Usability Testing

The two categories are known as formative and summative usability testing.

Formative testing is testing that forms and shapes a design for a user interface. Summative testing is about determining the issues, problems or benefits of an existing or prototype design.

It is not only the objectives of these two types of research that are different. If you really think about it, the methodology is also constructed in differing ways using separate techniques and devices.

For example:

Moderated and Remote testing

The nature of formative testing means that it is crucial to observe and understand the participants thought processes and their actions resulting from them.

The tester is moderating and observing the testing in real time so that how the user deals with the system and what issues they encounter is recorded. This will exclude unmoderated and especially remote unmoderated testing from the equation.

In contrast, summative testing is more about measuring and counting. Examples include; recording the time it takes to understand and complete tasks , counting the number of clicks ,pauses or errors in either the system or the users’ comprehension of it. In this type of testing, the volume and speed of tasks undertaken is more important than the quality of the observation or narrative.

Data Recording in formative and summative usability testing

In a formative test the type of information recorded is much more qualitative. Information recorded can include;

• Confusion or misunderstanding of instructions or functions.
• The thought process when confronted with a particular decision or choice on screen.
• Perceptions of the clarity.
• Usefulness and simplicity of the interface and its navigation and instructions

Analysis and interpretation takes place as the test runs so that learnings are fresh and observations are not lost or changed by memory. These may need to be fed back into the process or used as guidance to refine user experience as the trial is continuing.

Summative testing requires more disciplined data collection.The metrics recorded reflect what actually happens during the test and not perceptions or feelings. It is crucial that the testing is based on scientifically gathered, reliable data so that whether the interface passes or fails the user test is determined.

To this end, a qualified data logger is used in summative testing to record and present the results. Data is compiled and assessed over a longer period as expert statistical analysis and summarisation is required.

If you are interested in learning more about the types of usability test or need help in setting up and running them please contact us today for an informal chat on your requirements on +44(0)800 0246 247, or email hello@ux247.com or contact us.