In last weeks blog we explained that diary studies are qualitative surveys that can be used to provide insights regarding user behaviour. They can be a useful and effective tool but, like all research methods they have their own strengths and weaknesses which you will want to consider before deciding that a diary study is the right thing for your particular need.
What are Diary Studies?
“A diary study is a research method collecting qualitative data about user behaviours and actions over time.”
Why Use Diary Studies?
As well as all the essential data of how each participant interacts with a process or website, diary studies uncover far more specific details of how each participant is affected.
Each study will, undoubtedly, focus on different elements and areas, from the most targeted actions to a wide-cast net over numerous activities. But, whatever the focus, the data gathered will be richer and far more contextual when monitored by this method.
- The habits of your participants: Where and when do they engage, and with who and how do they share content, ideas, and opinions?
- How they engage with the product or pages: Seeing precisely how participants engage with the product provides further valuable information for later testing. What tasks do they carry out first? Which are the most important? What are their workflows for the most time-consuming activities?
- Motivation – Understanding how participants feel and think during decision-making helps pinpoint their drivers and priorities.
- The effects of familiarity on the process. Over long-term studies, we can monitor changes in participants’ behaviours and ideas as they become more comfortable with the system. Does their brand perception alter? Do they build loyalty? Is the system intuitive, or does it take time to adjust to ease of operation?
- How does the customer journey look? A diary study includes interactions using various devices and mediums, outlining how engagement and interaction affect function over varying touchpoints. From electronic and digital opportunities to traditional means of contact—they all play a part in the customer process.
The Benefits of Diary Studies
- Frequent, real-time research records more precise data
Ensuring your participants record their experiences as and when they happen, instead of asking them to recall their feelings at a later date of time, reflects a far more natural experience.
There is always a chance of missing or forgetting the feelings, activities, and interactions that affect how participants operate, feel, or make decisions during their experience. It’s easy for them to overlook an interruption, thought, or other references that may have impacted their behaviour that they aren’t likely to remember after the event. However small that action or event may have been, it could hold vital consequences to their decision-making process.
Having the opportunity to understand how every interaction can affect a decision helps put choices in context, providing deeper analyses for the researchers and hopefully the missing pieces of the puzzle they’re hoping to find. If you can unravel motivations, objectives, and the problems encountered, you can deliver a better, more finely focused route to wherever you want to guide your users.
- Recording in a natural environment delivers more realistic results
Understanding why and where your participants carry out their actions will paint a far more authentic picture of their operation. For example, creating lab or sterile testing environments takes the participant away from an authentic experience. While this can focus the participant solely on the product, it removes the outside interactions they’re typically faced with, affecting their interactions with the product.
Sterile testing can’t always provide the information you need about your product operating in its natural environment: for example, while your users are commuting, waiting for an appointment, or standing in a queue. If those everyday experiences are where your product is most likely to be used, then your data would be better gathered while doing so.
- An extended timescale exposes changes over time
Monitoring each participant’s behaviour over longer periods delivers far more insight than studies providing a mere snapshot in time. Ongoing diary studies show changes to behaviours, actions and provide patterns according to duration, interactions, and how their opinion of the product changes or develops during extended use.
For example, will user engagement dissipate over time, or will your participants access it more regularly the more familiar they are with the system? Has it got longevity, or is it merely a novelty? This is all valuable data when introducing new features, apps, and updated versions. The information may reveal options your team hasn’t yet considered, adding more value to an upgrade. Diary studies don’t just test for the things that aren’t working well but uncover improvements that can add real value.
Another benefit of longer testing periods and deeper, diary-like records is the likelihood of spotting the precursors to actions—the cause and effect of specific situations and their outcomes. There’s a far greater chance of understanding how those interactions affect the product’s use or attractiveness and how the participant reacts, who they confer with, and about what. Where a participant’s opinion or attitude changes, it’s beneficial to understand how and why their viewpoints changed.
- Capturing external influencing factors
A further key area of diary study is understanding which external influences affect your users’ actions and how. Without long-term testing, you won’t achieve as accurate data about when and where your participants engage with your product, what causes them to pick up and turn them on, and how different situations affect how they behave.
Only via regular, ‘in the moment’ recordings of how they feel and why they do what they do will you uncover genuine information about their operations, actions, and feelings. This often reveals the social situations, people, media, and more, that stimulates the need to interact with your product. Diary studies deliver the kind of data a questionnaire can’t. Where we may think we’re in total control of what we do and are perfectly aware of our actions, a diary study confirms just how little self-awareness we actually operate on and all the data and information that disintegrates after even the shortest breaks in concentration.
- They produce far more in-depth opinions
The longer your participants engage with your product, the more consideration they’ll apply. Naturally, this changes their opinions over time—sometimes for the better, sometimes worse—but all documented so you can learn from them and enhance the experience you deliver.
Diary studies can be used as soft-launching or beta testing opportunities as well as streamlining, revamping, and strengthening weaker products. In addition, such studies reveal developing insights and expansion of your participant’s ideas and ideals, their objectives and how they’re achieved or not.
Diary studies provide each subject with opportunities to grow while using your product while providing various methods to deliver that information. In addition, expressing how they feel about each interaction builds a deeper understanding of what you offer them and how they use it. This information is often the gold that you’ve been mining for.
The Limitations of Diary Studies
But it isn’t all upside as you can imagine. Here are some of the potential pitfalls.
- Poor planning could lead to poor-quality data
Without reviewing what you really hope to achieve, it’s unlikely you’ll put together a study capable of pushing participants into exposing truly beneficial information. Therefore, when planning a diary study, you need to know what you want from each participant and how to get them to deliver; that means holding brainstorming and fact-finding missions to uncover the essential areas.
Your brief needs to leave nothing to the imagination; it needs to be thorough, persuasive and intelligent. Your participants need to know the type of information they’ll be expected to provide, how they’ll deliver it, and to stay open and honest throughout the entire process.
- Picking the wrong participants can lead to inaccurate information
Explaining precisely what you want from your ‘diarists’ is only part of the battle. A lot of people are drawn to trying to tell people what they think they want to hear or what they should say.
If you pick those types of people-pleasing participants and praising what you’ve already achieved, then your data is going to be weighted in the wrong direction. As well-meaning as those people intend to be, they aren’t who you need on your team. Instead, you need an open and honest delivery that helps highlight or uncover best and worst practices, influences, and the participant’s reactions to them.
Participants occasionally present tainted and unhelpful views based on existing biases and opinions from something they’ve been told or seen somewhere else. Even the act of writing or recording their views over a format they don’t feel overly comfortable with can change what they’re trying to say.
That’s why the team selection process can take so long. There’s a great deal of pressure to get each study underway, but there should be more pressure on hiring the right people for the job. Your participants not only need to be relevant to the product but also capable of an ideal delivery.
Additionally—and further adding to the length of the selection process—is finding those who can verbalise or present information succinctly and clearly, using methods that provide the best insight into each scenario.
- Vast pools of data leads to longer analysis periods
Without the neatly gathered data a questionnaire or other method might supply, the mass of information, in many formats, requires a great deal of processing to uncover the real gold. You may have to expand your resources to catalogue all the data you’ve accrued, and doing so could impact the speed and efficiency of your own actions and delivery.
- You may need ongoing motivation to keep participants involved and interested
With long-term analysis, participants tend to lose interest and drop out of the system. It’s often nothing to do with your product or project but more to do with human behaviour and priorities.
To address the situation, you may have to implement a reminder system to ensure your diarists complete each session as they should, instead of recalling the information at a more suitable time. Carrying out the study and recording their reactions in that way defeats the object of the real-time, accurate representation of their actions.
All in all, diary studies can prove to be a very useful tool in the researchers’ armoury, often uncovering issues, relationships and behaviours that would otherwise go unnoticed. If you would like to know more about how they operate, ring us on +44(0)800 0246247 or email us at hello@ux247.com for an exploratory, no-commitment chat.