How to Build a Strong UX Research Program

In the webinar Building Blocks of UX and Customer Journey Research, Applause experts Celeste Buckhalter and Maria Mondragon shared the crucial elements of a UX research program, and how organizations can scale their research practices to capture insights throughout the software development process. They shared a framework for measuring UX research maturity, outlined often overlooked scenarios for research, and shared some examples of how organizations are advancing their UX programs. In this post, I outline some of the key points from the webinar. 

Why UX research matters

In Applause’s 2024 State of Digital Quality report, we asked organizations how they’re measuring quality. The top metric: customer sentiment and feedback. That is the number one way that organizations are determining the success and quality of their applications, products, and digital experiences. Yet all too often, teams fail to derive value from that feedback. That’s where a strong UX research program comes in, analyzing user feedback to prioritize product improvements. 

A framework for understanding UX research maturity levels

So what does a strong research program look like? Applause has developed frameworks that outline the different building blocks, including core capabilities and practices at each stage. One framework outlining building blocks for UX programs is available in the 2024 State of Digital Quality report; in the webinar, Maria Mondragon shared a similar way that organizations can benchmark themselves in terms of maturity or sophistication in UX research efforts.

Level 1: Absent
Maria explained that in this first level, “there’s minimal awareness about UX within the organization. You don’t really have a UX research team as such. There might be one or a few individuals that have the perception of what UX is, but there’s no processes in place within the organization to integrate UX.” At this stage, internal research capabilities are limited; organizations typically rely on external providers to gather user experience feedback.

Level 2: Ad-hoc
At the next level, the product or the design teams typically drive the process, Maria said. While these teams want to attempt to do some research, it’s typically a one-time effort or tied to a specific project. “There’s really not a process in place that’s really integrated,” she said. Internal tools and capabilities are still limited. She pointed out that the individuals who understand the value of UX may not have the expertise to lead a full blown research team.

Level 3: Integrated
“After the ad hoc level, we see what we would call the integrated maturity level,” Maria said. “In this case, you have senior leadership recognize the importance of UX. You do have a UX research team that will work with other teams, to research and get customer feedback. You have internal processes and best practices in place within the organization.” Champions are trying to educate teams across the company of the importance of UX and UX research. “In this case, you do have the presence of a UX research team, and they are empowered to use different types of tools,” Maria said. 

Level 4: Comprehensive
At the comprehensive maturity level, UX research is really embedded within the whole organization and SDLC. “You have UX within all of the processes, and [teams] make decisions based on research that they have,” Maria said. At this level, companies have established education on the value of UX research; they understand that it’s a competitive differentiator. These organizations use research data to help guide product roadmaps. 

“We work with a lot of different companies. And at the end, what you want is your end customer to be happy, and the goal would be to have that gold standard: being comprehensive,” Maria said.

Integrating feedback into product development

Next, Celeste walked through how to put user feedback to use throughout different stages of the product development life cycle. She outlined goals, questions, and research techniques aligned to each stage. 

“You should be talking to users from the very beginning, even before a design is created and as the requirements are being determined,” Celeste said. At that define stage, key questions center around understanding who your users are and what those users want or need. “If we’re redesigning something, then at this phase, we’ll want to know what’s good or bad about what is currently happening.”

“As you move forward and begin creating designs, make sure that you get feedback from users early on and often to determine if they understand the concept. If you have more than one option that you’re trying to decide between, bring that to the users to find out which one works better. And then as you move to the build, ensure that you continue to iterate upon those early designs and find out if the changes that are being made are being well-received,” Celeste said. “Once you’re in production, once you’ve launched, you want to keep seeking that feedback and learn how to improve the experience.”

Pre-production: Define 
“This is when you’re going to start gathering your initial requirements and determining how and where to start your designs,” Celeste said. Questions to consider: 

• Who are your competitors? 
• What are they doing? 
• How do users perceive the experiences your competitors are currently providing? 
• Where are the gaps between the user’s expectation and the experiences your competitors currently provide?
• Who are your end users and what are their key characteristics? 
• What do they need and expect from you? 
• What are they already doing in terms of a solution and what would they like to do in the future?

Celeste recommended a number of different ways to collect feedback at this stage. “You can conduct competitive studies to review what your competitors are doing and how your potential customers would react to it. One way to do this is with a quantitative survey or a benchmarking study, where you might ask a large number of end users a series of closed ended questions,  Likert scale questions or yes/ no questions to uncover their experiences with your competitors. 

This type of study can help you see at a glance the user satisfaction across the competitive landscape via metrics.”

At this stage Celeste also recommends qualitative competitive studies. Observing individual target customers using your competitor’s site or app and asking them about their experience helps you get an in-depth understanding of what’s working and what isn’t.

“Another great way to collect this feedback is via generative research or research that generates ideas where we might conduct one-on-one interviews with people who match your target audience,” Celeste said. Questions could relate to their goals and what they need and expect from an experience. Their answers could reveal what might make them start or stop using an app.

Pre-production: Design and build
Once organizations start creating designs, it’s important to check them periodically with your target audience and iterate on them. “At this point, you want to evaluate your design and learn things like how easy or difficult it is for someone to use them. Where are those friction points? And how understandable are things like the content, the navigation, the overall structure of the page, the overall flow of the process?” Celeste explained.

At this stage, it’s crucial to observe typical end users interacting with your designs to see and hear where those challenging friction points are. You also need to understand what your users expect or need instead. “Here is where you will conduct the studies that are most commonly associated with UX research: evaluative studies,” Celeste said. “You’re going to evaluate your designs.” Celeste described several types of studies, including card sorts, A/B testing, usability and heuristic studies. 

Maria walked through an example of a usability study Applause performed for a luxury brand. On its website, the retailer offered an option to make appointments to visit a store so shoppers could see or try on items before making a purchasing decision. “Throughout the study that we conducted, the main findings were that it was great for the participants that they were able to make the appointments. However, they were worried because there was no option to see if the items they were seeing on the website were available at the store location or not. This was a pain point for them,” Maria said. Through the study, the retailer learned that to truly add value for customers, they needed to incorporate a way for users to see if each of the items they were interested in was available in the store or not.

Post-launch: Deploy
The research process after launch is fairly similar to the methods used while designing and building. Once a product has launched, you need to see and monitor how well it is performing. “You’ll want to know how easy or difficult it is to use, how understandable it is overall, and you’ll want to observe and learn from people who are using your experience so you can determine if there are places in the process that need to be improved,” Celeste said. 

While the research methods from the design stage still apply, some additional methodologies are useful as well. Customer journey studies allow teams to evaluate the entire experience, such as making an online purchase and picking up an item in-store. With this type of study, Celeste said, “You could see where the potential friction points are happening by learning from people who go through that entire process from start to finish.”

Celeste described some of Applause’s unique offerings, including user feedback analysis, which adds a brief UX survey with open and closed ended questions to a functional testing cycle. 

Applause also conducts longitudinal studies, where participants use an experience for a specific length of time and provide feedback at specific touch points along the way. 

Maria shared an example from another customer engagement, where a global company had a product that was mainly used in the U.S. and wanted to launch in other countries. The customer was testing a shopping experience that had users ordering online through a smart assistant. “We found out that, for example, in India, a lot of people would not be interested in using the smart assistant because they had different products that sometimes the smart assistant did not understand. So it slowed down the process for them,” Maria said. “Those are things that you couldn’t find out if you don’t find the right demographics and do these studies at this stage of the process.”

Some commonly overlooked UX scenarios

Customers want consistency across all the different channels and touch points that they have with a brand. It’s becoming more and more common for consumers to use multiple channels in that experience. This introduces greater margin for error, makes it more difficult to get full visibility into all the customer journeys, and increases the odds that some users will encounter friction. 

Maria provided a list of factors that organizations often fail to take into consideration when evaluating UX:

• User diversity. Research must include users from varying demographics, such as people with disabilities, or those who live in specific markets, like in the smart assistant example. 
• Unforeseen use cases. People may use products in ways they weren’t designed to be used – but users still expect them to work in a certain way. Understanding those expectations is key. 
• Real-world conditions. If your product is going to be used in different environments, understand how that changes the experience. Assess different scenarios reflecting real-world use patterns and edge cases.
• Physical channels. Organizations often focus only on digital experiences and overlook in-store experiences or hybrid customer journeys. 

“All of these different nuances in user experience are really important,” Maria said. “Those can be incorporated into different studies so that you have a holistic view of your digital properties and the user experience that you’re giving your customers.”

Where UX research can help 

“There are a number of concerns that teams sometimes forget that UX research can address,” Celeste said. In that ideation phase or in the define stage, many organizations want to create something, but they’re not sure where to start. That’s where research can help teams to understand their target audience and the ideal experience for them. 

Within the launch phase or the production stage, UX research can identify where in a process things are breaking down. “Maybe a solution’s receiving negative reviews, but we don’t know why, or there’s a drop off happening in the checkout funnel, and we’re not sure where the friction is,” Celeste said. “All of these are great candidates for UX research and for evaluative studies.”

Celeste emphasized that whenever there’s a user struggle, UX research offers a means to see what’s going on, where things are breaking down, why that is happening, and understand from the user’s point of view, what they need instead. 

Webinar