The critical incident technique is a qualitative research method designed to observe memorable events and analyse the impact of specific human behaviours on the outcome.
The method was initially developed as part of an aviation psychology program to analyse pilot errors when reading and interpreting aircraft instruments. Since its introduction through the social sciences, it has been applied to healthcare, education, organisational business operations, market research techniques, product development, and user experience.
By exploring detailed information about a specific situation, its background, the interactions and critical incidents, practitioners can address common and uncommon issues from the respondent’s perspective. It can be used to identify ideal personnel, as a performance appraisal process, to enhance training programs, avoid routine incidents, and bring major benefits to various systems, staff, and products.
The critical incident technique is typically used where people and machines work together. It is utilised in UX for users to recall specific incidents while using digital products, whether a system, process, feature, or user interface.
The definition of critical incident technique
As described by the Nielsen Norman Group:
The critical incident technique (CIT) is a research method in which the research participant is asked to recall and describe a time when a behavior, action, or occurrence impacted (either positively or negatively) a specified outcome (for example, the accomplishment of a given task).
What makes a critical incident?
For an incident to be considered critical, the user must outline a specific episode or event that had a ‘causal relationship’ with the outcome; in other words, an input that affected the outcome. Such events may be due to human engagement with a product or product application to a situation.
Critical incident technique studies are designed to uncover positive and negative outcomes, major and minor events, from both regular and less common occurrences.
How is the critical incident technique different to a standard interview or survey?
The crucial difference of a critical incident technique interview (or questionnaire) surrounds the type of questions we ask. While qualitative by nature, exploring in open-ended detail the actual events of a particular incident, CIT questions are designed to provoke users into recalling specific incidents featuring the necessary critical outcomes.
While not all questions in a CIT study will contain a direct line of enquiry into the outcomes of critical events, those that drive the discovery of essential data will. For example:
- Can you recall a time when the product/application helped you be effective in your work?
- Can you think of a further instance where the product/application helped you be effective in your work?
- This time, can you recall a time when the product/application prevented you from being effective in your work?
- And now, can you think of a further instance where the product/application prevented you from being effective in your work?
- Now, can you recall an operation within the product/application that helped you with a specific incident, task, or event?
- This time, can you recall an operation within the product/application that caused problems while carrying out a specific incident, task, or event?
These are very general questions, but they can be adapted to deep-dive into specific products, applications, sites, and situations. While conducting research this way requires a particular style of questioning to collect data surrounding such critical incidents, we uncover further information by repeating the same level of exploration to uncover additional events and incidents.
Advantages and disadvantages of the critical incident technique
As with any UX methodology, the critical incident technique has pros and cons, with a distinct place within our research studies.
Advantages of critical incident technique
- It provides detailed information surrounding common and rare events.
- It uncovers patterns and themes that can be used to provide essential solutions.
- It explores critical incidents with both positive and negative outcomes. This provides the opportunity to correct weak areas and enhance systems by introducing more instances of their strongest performing practices.
- It helps us understand the human behaviour behind decision-making and product applications.
- It has a broad range of cross-industry uses, from market research and UX to healthcare and staffing.
Disadvantages of critical incident technique
- It can be heavy on time and budget resources, whereas other methods may deliver the necessary insights at a lower cost.
- It requires experienced, professional practitioners.
- It’s dependent on the quality of the captured data and insights it provides.
- Subconscious biases may influence the data. While not intentional, practitioners and participants will have preconceived ideas and views about a service, product, or practice. This often introduces biases in either direction. Practitioners need to examine their questions to ensure they’re not driving responses in a particular direction but to attain as authentic and accurate a picture as possible from the respondent’s perspective.
- Poor recall and memory can affect the data collected through incorrect recollection or omissions. Users are more likely to recall events that happened recently—those at the forefront of their minds. Unless a historical incident resulted in a more severe and memorable outcome, it’s less likely to be remembered without the time to consider the full range of possible events.
How to deliver a successful critical incident technique study
Whether you carry out interviews or ask participants to complete a questionnaire, it’s crucial to remember that this is a fact-finding mission to examine where specific ‘critical’ problems occur.
CIT interviews yield a more flexible method to attain open-ended, long-form explanations. However, a questionnaire or survey will allow for more structure.
A basic set of critical incident method steps might typically include:
- Consider the problems or incidents you’d like to explore, whether specific situations, unknown events, or general operations and practices.
- Choose participants that conform to your ideal user personas. This helps us understand audiences through statistics or data from different sources.
- Examine the appropriate methods for gathering facts and data: interviews, surveys, observations, focus groups, discussions, and more.
- Code the data collected to establish patterns, themes, trends, and frequencies to synthesise your findings.
- Apply data analysis methods to synthesise the information and outline the range of exposed issues.
- Create and deliver your study report, revealing your findings and offering potential solutions.
- Develop strategies and solutions for your UX designers and developers.
Creating the questions for your critical incident technique interviews
The success of critical incident technique research relies on the practitioner asking the right questions. A wealth of valuable information and crucial insights are available when recalling everyday events and historical situations. Compiling and delivering the ideal questions—while allowing participants enough time to recall those relevant events—demands careful consideration of the study objectives.
- Define the study objectives, aims, and goals. This will guide your line of questioning.
- Examine the circumstances and conditions that create the background of the situation.
- Consider the implications of the study outcomes for question relevance.
- Avoid conscious and subconscious biases and leading questions.
- Refine your interviews and questionnaires through regular iteration, ensuring questions are effective and relevant and provide significant insights.
- Vary the types of questions you ask to gather the most comprehensive data.
- Ensure your questions are clear, written in everyday, easy-to-understand language, and relevant to the study objectives.
- Test the questions before conducting your critical incident technique study to ensure they provide the rich information you hope to reveal.
Summary
Critical incident technique interviews gather valuable information about specific events during an activity, task, or process. We can explore behavioural and usability problems surrounding people, systems, and processes with the details relayed from the respondents’ perspectives.
When conducting research using the critical incidents technique, we need to be clear about goals, the kind of critical incidents that may lead to your most valuable data, and how they will help form the ideal line of questioning to drive your most insightful results.
If you would like to know more about using the critical incident technique in your UX Research email us at hello@ux247.com.
