Why Retail Digital Quality Matters More Than Ever

Executive summary

For the second year, we analyzed a representative sample of our testing data to identify the most common flaws in digital experiences and map out how retailers can stop them from reaching customers. We hold unique insight into the state of digital quality based on our work with retailers who are creating a wide range of application types. Our platform data offers a wealth of intelligence on how retailers approach testing and digital quality. This year’s report also introduces digital quality frameworks — found here in the State of Digital Quality 2023 summary report — that outline core capabilities and typical processes for organizations at different stages on the journey to digital excellence. These frameworks provide concrete guidance on how organizations can improve quality and efficiency across the organization.

For general highlights from this year’s global summary report, read our blog, Improving Digital Quality in 2023: Where to Focus. In this blog, we narrow the aperture on highlights from our retail testing data, give key digital quality recommendations for retailers and offer some examples of how retailers address significant friction points in their pursuit of digital excellence. For a complete view of the retail industry’s state of digital quality, visit this year’s report data.

This blog covers the following areas:

• Retail and the current economic climate
• 5 key areas of retail insight
• Customer journey testing
• Payment testing
• Accessibility testing
• Localization testing
• Functional testing

• Digital quality recommendations
• Additional State of Digital Quality in Retail 2023 report information
  

The global economic backdrop and the retail imperative

Why is digital quality particularly salient for retail? The current global economy. Repeated U.S. interest rate hikes and those of the European Central Bank, as just two examples, slow growth, decrease retailers’ ability to service debt, and impact inventory supply and hiring practices. Add to the mix, inflation and increasing cost of goods and services, recession concerns in some circles, and the resulting drop in consumer confidence and spending.

What does all this mean for development in your retail organization? You must ensure that everything development does helps improve your ability to capture an increasingly competitive wallet share. The lifeblood of successful retail, the frictionless customer journey, becomes more critical than ever, as glitches – anything from a shopper’s inability to view comparable products to not being able to apply a promo code – can mean cart abandonment.

Of course, finding bugs early in the SDLC versus in production avoids many of these issues, but also costs your organization less. And cost optimization in today’s environment is equally key for retailers. For example, avoiding accessibility lawsuits means sidestepping legal costs as well as lost personnel time addressing the issues. Building to accessibility standards also ensures that you are not missing out on a substantial market share represented by people with disabilities (PWD). Anything less than superior digital experiences in this complex environment can mean added organizational expense and retailers leaving consumers dollars on the table.

5 Key Areas of Retail Insight

With Applause testers uncovering more than 200,000 defects last year — 71,000 of which were in the retail sector — it’s clear that retail development organizations continue to be challenged by the ever-increasing complexity in striving to provide digital excellence to customers. Along this journey, we examine five key areas with corresponding insights from Applause’s global testing data.

Customer journey testing with real customers provides key insights that move the needle for retailers

Our retail customers use many dimensions to assess digital quality, including a range of metrics directly from their customers. Customer sentiment and feedback, customer satisfaction research, retention and churn rates, the number of customer support requests… it’s impossible to evaluate the customer experience without input from real or potential customers.

You need to understand and evaluate all the interactions customers have with your brand. This complexity contains myriad sets of transactions and tasks that often cross channels, both online and in store. Consider, as one example, the ever-increasing role of AI/generative AI in customer support. How does your brand avoid inaccurate and biased results that plague many chatbots today? Not all organizations have the specialized skills and resources necessary to conduct thorough UX research covering all customer journeys, both physical and digital, from end to end. And as they expand into new markets, even those retail organizations that do have dedicated UX research teams may have difficulty quickly sourcing research participants that can give the appropriate real-world perspective and insights needed, not to mention that have the array of required devices and OS combinations to accurately represent how customers engage their brand.

Types of customer journey friction:

Customer journey friction typically falls into one of four categories (or combines some of these elements):

• Functional errors

• Payment issues

• Poor UX

• Insufficient operational readiness

Case study: A preponderance of data does not mean insight

One of the world's largest retailers found that traditional customer journey testing methods missed customer friction points, which they deemed equally important as bugs. It had plenty of data on customer sentiment, but lacked insight into the specific causes. It was also challenged with managing a myriad of customer fulfillment options, which made it difficult to create a user-friendly and intuitive customer experience.

Applause identified friction points across omnichannel journeys by having testers work online, as well as make in-field visits to stores, and provided weekly journey reports including specific causes of poor sentiment, along with actionable ways to reduce or remove friction. In addition, Applause applied a custom scoring model to each part of the customer journey to identify and track trends. This model enabled the retailer to better understand the impact of UX changes. Applause's always-on research allowed the retailer to catalog unique customer interactions that could not otherwise be found in standard, task-based, user-based research.

The buck stops here. Payment issues mean lost revenue

Without thorough payment testing processes in place, retailers can’t ensure customer transactions work successfully. With digital wallets and ever-expanding alternative payment methods on the rise, payment testing must adapt and evolve to cover these new payment instruments and paths to purchase. And there is a lot to test. Retailers need to discover bugs and technical blockers, how charges appear on receipts or customer accounts, the payment experience on various devices, OSes and browser versions, just to name a few. Testing with dummy cards just won’t cut it — real-world testing with live payment instruments is the only way to guarantee transactions work properly for every customer, every time.

Key observations

• The prevalence of each type of payment bug remained remarkably consistent with last year.

• A whopping 84% of all payment bugs were classified as functional bugs or workflow errors: without corrective action, transactions will fail, delivering a serious hit to revenue.

Accessibility innovation raises the digital quality experience for all customers

The World Health Organization’s latest estimate is that one in six people, or 16% of the global population, lives with some form of disability. In addition, many people experience temporary disabilities due to injury, illness and aging at some point in their lives. As many PWD have mobility issues, they may rely even more on the digital realm to do their shopping, increasing the importance of fully accessible online experiences. Global retailers are recognizing that digital experiences that PWD can navigate easily often create a better UX for all of their customers. Of course, doing this makes great sense for business, as improving the UX increases retailers’ total addressable market. But it's not just PWD that benefit from accessibility improvements. Some accessibility fixes, like addressing missing or unclear labels, enable automation engineers to build more stable automation and move faster.

As stated, executing accessibility improvements and innovation helps increase the total addressable market by enabling all shoppers, not just PWD, to more easily interact with your brand by making the UX more friction-free. Still, the fact remains that retailers must meet global accessibility compliance in accordance with accessibility laws around the world. 2022 accessibility data in the U.S. shows there were 2,387 web accessibility lawsuits filed, which includes an increase of 143% in companies that received multiple lawsuits year over year. Of these, the consumer goods and retail industry had the majority of all accessibility lawsuits in the U.S. – a whopping 1,378 (58%). Compliance is just the start to the ongoing accessibility journey. Brands big and small are shifting left to find accessibility issues earlier in the SDLC and involving PWD feedback in the design phase and throughout the development process.

Key observations

• Screen reader defects continue to make up the vast majority of all accessibility errors (72.3%).

• Bug type distributions remained consistent, typically fluctuating less than 2% +/- from last year.

• Retailers who progress through the accessibility framework – for example, creating a Center of Excellence with guidance on design and development best practices – show significant reduction in the most common accessibility defects, as organizational processes are documented and awareness improves.

We made sure that we incorporated persons with disabilities, and brought them into the conversation at the beginning of our SDLC. We actively solicited their feedback from the start so that as we created our user stories, designed our experiences and began testing what we created, their voice was constantly being heard.

Design Researcher, Major Global Retailer

Case study: eCommerce provider augments internal accessibility knowledge, headcount and ability

A leading global ecommerce provider that facilitates C2C and B2B sales needed to increase its focus and ability to build and maintain digital accessibility for a broad portfolio of web and mobile properties. It also recognized its internal accessibility knowledge gaps, and that it would need to initiate a cultural shift wherein the organization would prioritize accessibility and include best practices into regular operations.

It engaged Applause to conduct third-party compliance validation to provide an unbiased perspective and insights on their customer journey outside of the happy path, as well as to test flows outside of its main project team. In addition, Applause performed accessibility conformance testing in pre-production as well as provided testing expertise across web and mobile that included running ad-hoc testing and quarterly accessibility assessments for litigation risk management.

Applause’s collaborative relationship with the ecommerce provider's accessibility team significantly augmented its internal capacity. Applause was integrated into the company’s bug tracking system and development process, so now developers can quickly fix bugs, increasing development velocity, saving development time and helping launch new features more quickly to market.

Adept localization ensures more than just translation

When retailers enter new markets, they require much more than simple language translations for their digital properties. To get new customers on board, your apps need to reflect an understanding of the local language, values, preferences and processes. Seemingly small errors, like missing translations, incorrect country and region names, listing incorrect currencies or workflows that don’t align with local customs can cause customers to abandon transactions and put the business at risk.

When localization is done right by retailers, it builds credibility (and can improve search engine optimization). When done poorly, it can alienate potential customers, spark regional site blockages and boycotts, and trigger other ugly consequences. Speaking your customer’s language is a necessity for global retail brands: In a study by CSA Research, nearly 90% of non-English speaking consumers stated they are more likely to purchase a product from brands with multilingual support systems and knowledge bases and 75% said they would become repeat customers if a brand offers multilingual customer care.

Case study: Dress for success in new markets

A large global clothing brand was not seeing the conversions it had expected after launching in Germany. All functionality worked, so the company suspected there were translation issues at play. The retailer has particularly strong brand characteristics that it needed to represent and it used a language translation provider (LSP) to translate English to German. The retailer engaged Applause to verify the LSP’s work. Applause found a number of important localization issues and mistranslated words, as it can be very difficult to represent the concepts behind the words in another language. Verifying the ecommerce content in-region helped the retailer get as close as possible to the intended meaning. This was essential because the organization’s brand targeted a specific demographic of customers, and precision was of utmost importance.

Functional defects comprise just shy of 95% of all bugs

Typical retail functional tests – for example, ensuring the entire product catalog is viewable, accurate order processing, smooth checkout, timely and helpful customer support – run within very tight windows, across markets and complex device and OS configurations. While organizations have tried to integrate testing into developer workflows, this poses a challenge for many QA and development teams that are already under considerable stress. In addition, teams often prioritize test execution over developing a testing strategy or doing the routine work that makes testing more efficient and effective in the long run. Raise your hand if you’ve ever put off test suite maintenance, released while testing was in progress, pulled in help from other departments to complete testing, or released with risk because there was no time.

See the top device combinations across the retail data we analyzed:

North America data slice: most popular device configurations tested (for data on all global regions, access the report)

Key observations

We note two data interesting data changes from last year’s analysis and one that remains fairly constant:

• Crashes are the most critical flaw, but this year we saw a 1% drop from 3.2% to 2.2%. While this is a small slice of the overall pie, it is a one-third improvement year over year for this specific flaw.

• Retail comprised 23.1% of all industry test cycles (37,117). Retail had an average of nearly 2 more mobile wallets tested per cycle (4.5) vs. all other industries (2.6), noting a possible shift in retailers prioritizing the importance of new payment methods to customers.

• The majority composition of bug types – functional, visual and content – identified since last year is down just slightly, by .7%, comprising 94.9% of all bugs found.

Case study: A global cosmetic firm identifies and fills its functional testing gaps

A leading global cosmetic brand wanted to deliver a consistent experience across all 12 of its brands and address lost sales resulting from shopping cart malfunctions and missing merchandise on websites. To accomplish this, it needed to migrate all of its websites into one centralized ecommerce platform while maintaining site performance during and after this transition.

Juggling a wide variety of vendors and partners working across all the company’s websites posed a challenge. To meet the tight deadlines for substantial migration efforts, the company was pulling employees from their normal roles to take on unfamiliar QA tasks. This added friction to the day-to-day operations, as well as ramp-up time while staff learned QA skills. Finally, the company had difficulty continuously updating products and promotions on all brand sites while ensuring web pages were active and delivering optimal customer experiences.

Applause partnered with the cosmetic company and constructed a test team across 65 different countries, providing expansive test coverage to ensure the company was able to successfully execute its full test matrix. The team worked on a weekly basis to ensure site performance from all locations, and in doing so, was able to address incidents of missing merchandise and various issues with the shopping cart – for example, the shopper could not increase the item quantity in shopping cart; the order total did not update when increasing the quantity of an item in the shopping cart; and free shipping for orders over a specified amount was not given after the order exceeded the required threshold. In addition, the company leveraged Applause to meet their QA requirements, enabling internal employees to remain focused on their core jobs.

Digital quality recommendations

Digital quality is an intersectional discipline and can’t be viewed in silos. Customer experience and UX, functionality, localization, accessibility and payments, bleed into one another. Assess different components of an app or digital experience holistically, not just in isolation, to truly understand customer encounters with your brand.

With that in mind, we offer five recommendations that intersect with one another at various points:

Proactively examine how well your testing practices serve your organization and most importantly, your customers. To truly improve, you need to understand your current state, which may be siloed or inconsistent across different teams, products, groups and even individual team members. Find what’s working and replicate that — as well as what’s not working and needs to be addressed. Benchmark your team on its quality journey. Review the digital quality frameworks found here in the State of Digital Quality 2023 summary report. They outline core capabilities and typical processes for organizations at different stages on the journey to excellence, and provide concrete guidance on how organizations can improve quality and efficiency across the organization.

Once you’ve assessed yourself, set corresponding goals. If your organization has achieved excellence, think about what you need to do to maintain that position and how you can serve as an exemplar for other teams and businesses.

Know which local and regional laws govern your business and make sure you're in compliance. We meet with customers who simply don’t know that they’re violating the law. And it’s only getting more complex. In the near future, for retailers located outside of Europe but doing business in the EU, for example, the European Accessibility Act will have strict penalties if your digital properties fail to comply. If you need a starting point for accessibility efforts, start with ensuring that your digital properties serve screen readers with no issues, as the vast majority of accessibility non-compliance issues, 72.3%, involve this key assistive technology.

Recognize how localization efforts speak volumes beyond just simple translation, and double-check accuracy and nuance with double-blind validation. Even with personalization efforts in retail on the rise as a lever to enhance customer connections, the fact remains that customers and prospects continually seek reasons to trust your brand. Applause routinely sees how organizations reap the benefit of putting extra effort into ensuring that their digital properties work as intended in specific markets, reach the broadest audience possible and show that they have taken the time and care to localize all content. For retailers, any misstep along the way can have immediate implications for sales, brand reputation and customer retention and loyalty.

Translators and testers each bring their own backgrounds and linguistic preferences when localizing digital properties in new markets. Those can influence how they interpret various translations. Using two testers who speak the local dialect is key. If there’s disagreement on a particular meaning or translation, it’s best to understand and resolve it before releasing your website or app to customers. Subtle localization oversights, particularly around personal payment information or awkward language can cause shoppers to abandon carts at higher than average rates.

If you don’t have the bandwidth or resources to test end-to-end journeys, use data to pinpoint where you’re losing customers. If customers are not signing up for a loyalty program, you know to look at the application form, flow and information you’re requesting. If customers aren’t purchasing recommended products at expected levels, perhaps it’s because there’s an issue with the coding tied to the product they are purchasing in relation to the recommended products, a problem of a very different sort. Look at the data you have available to determine which elements of the customer journey demand attention, then test to understand how you can improve and optimize. If you lack data, you need to focus effort on deliberate journey mapping to understand the various paths your customers take so you can then research – and refine – their journeys.

Test with real payment instruments, plan for increasing complexity and adjust testing time accordingly. Without real payment instruments, you can’t validate information in complex flows like multifactor authentication, how a charge appears on a credit card statement or whether certain payment methods or BINs aren’t working properly. Dummy cards simply can’t test for these things. In addition, as personalized experiences become the norm, retailers’ testing matrices — and applications themselves — are becoming more complex and harder to manage.

For example, in choosing to offer new forms of payment such as buy now, pay later (BNPL), retailers must test over time, not just once, to ensure payment functions as intended. Retailers must also build in the appropriate amount of time to test certain types of transactions. BNPL, recurring transfers, subscription renewals, loyalty programs and scheduled payouts are just a few categories of transactions that can take more time to test. What happens if an auto transfer is scheduled for the 31st day of each month and a month only has 30 days? Or the credit card on file expires before an automatic renewal payment goes through? Give your teams the time needed to thoroughly evaluate these scenarios.

State of Digital Quality in Retail 2023

Use this report to better understand the gaps in your organization’s capabilities and processes and to map out a plan to improve your digital quality, regardless of your company’s current state.

For a complete view of the retail industry’s state of digital quality data, visit this year’s State of Digital Quality in Retail report. It contains details on the following key areas:

• Methodology used

• Bug type definitions

• Bug breakdown charts

• Charts illustrating browser and OS combinations (desktop) and device and OS combination (mobile & tablets) and more

Readers can also access our digital quality frameworks in the State of Digital Quality 2023 summary report.