Put Digital Quality Center Stage in Media and Telco

For the second year, Applause analyzed a representative sample of our testing data to identify and analyze the most common flaws in digital experiences for media and telecommunications companies and map out how they can stop them from reaching subscribers. We hold unique insight into the state of digital quality based on our work with companies in diverse industries and regions, from varying technological maturity levels, who are creating a wide range of application types.

We summarized key points from our State of Digital Quality 2023 report in our recent blog, Improving Digital Quality in 2023: Where to Focus. While that report marks broader digital quality trends, this blog post explores what our findings mean for media and telecommunications providers. High churn rates and intense competition put the impetus on media and telco brands to provide digital experiences that delight subscribers. Learn from and adapt to defects in these industries to earn loyalty from customers.

In this media and telco blog, we will dig into how:

• Successful brands entice and retain subscribers

• Functional testing reduces the chance of technical difficulties

• Localization testing establishes meaningful connections with subscribers

• Accessibility testing validates the subscriber experience for all

• Payment testing keeps revenue streams healthy

• Customer journey testing supports the subscription model

• Digital quality recommendations

• Index and data references

Successful brands entice and retain subscribers

Streaming media is now a mainstay in households, domestically and internationally. Over the next few years, the industry could reach a compound annual growth rate of 21.5%, pushing the market size to nearly a half-trillion dollars by 2030. With present-day streaming media subscriptions numbering in the hundreds of millions, brands stand to win — or lose — billions in potential revenue over the next decade.

Success means winning over subscribers with friction-free experiences that keep them from hitting that Unsubscribe button. And this is a continuous challenge, as brands must integrate new technologies like AI and blockchain that advance streaming functionality, performance and experiences. Strong proprietary content helps retain subscribers, but clunky experiences — not to mention non-exclusive content that reverts to a competitor’s platform — will chase them away.

The same goes for telecommunications companies, where revenue among the “big four” mobile providers alone reaches roughly a half-trillion dollars annually. Many customers rely on their telecommunications services more than ever — not just for voice services, but also remote work, video communication, cloud backup and so much more. Reliability and innovation are key for customers in their merged physical and digital worlds, with everything accessible — ideally, easily — via the phone at their fingertips.

Unlike in other industries, where a conversion might mean completing a point-in-time transaction, these companies must convince customers to subscribe over a longer period. While a streaming media subscriber and a telco subscriber differ, the overall goal is the same: maximize the average revenue per user, reduce churn and expand the user base. It’s a tough juggling act, made even more difficult when digital quality fails those subscribers.

Successful brands must meet a lengthy list of subscriber requirements — any deviation from their expectations on usability needs, promotional offers, partnerships and bundles, licensing restrictions, ease of payments, cultural sensitives and other requests threaten revenue streams. In times of economic uncertainty, the need to deliver on these requirements only goes up. Conversely, brands that manage to connect with their subscribers can become part of their daily routines in a meaningful way.

5 key areas of insight

Applause evaluated media and telecommunications experiences across more than 9,000 individual mobile devices, 700 unique desktops and 500 OS versions to assess endless combinations of networks, browsers, payment instruments and integrations worldwide.

The result? Applause testers uncovered nearly 100,000 bugs across thousands of device/OS/browser combinations. Testing included websites, apps, connected devices, mobile web, OTT devices and mobile apps in real-world scenarios.

Our assessment produced insights in five key areas of digital quality. Read along to learn how media and telecommunications companies can deliver exceptional subscriber experiences.

Functional testing reduces the chance of technical difficulties

Different devices, different experiences, same high expectations. When delivering streaming media services to subscribers, the bar is high. Unreliable or cumbersome streaming platforms are prone to cancellations, especially with more competitors entering the space every year.

Functional testing helps ensure that subscribers receive the services they expect on real devices and in real scenarios, whether it be a smart TV with a lightning-fast private connection or mobile phone on a busy public network. Starting with the basics — from making sure users can download and install the platform, through validating complex workflows and integrations, including APIs and connected devices — organizations must ensure broad functionality before releasing to customers. The focus on quality must never cease — every launch and service update must deploy successfully.

See the top device combinations across various regions in the media and telco data we analyzed (for data on all global regions, access the report).

North America:

Europe:

Asia:

Key observations

• The composition of bug types identified remains the same as last year. Functional, visual and content defects continue to comprise over 90% of all bugs found — each bug type was within 1% of last year’s report data. These findings indicate media and telco companies are largely struggling with the same complex digital quality challenges as they did one year ago. With new content launches and updates happening all the time, vigilance is key to reducing these defects.

• While crashes are the most critical flaw, they continue to plague media and telco apps and platforms with no sign of year-over-year improvement. Notable streaming media crashes have occurred during times of high traffic volumes, which exemplifies the point: with these platforms serving content to millions of people and relying on multiple third-party services and systems, there are many potential points of failure in the subscriber experience.

How Vodafone shifted quality left

Vodafone leveled up its functional testing practice by shifting testing left to bring in valuable feedback earlier in the development process. The telco company’s user acceptance testing (UAT) department, led by an Agile evangelist, carries out in-sprint testing to ensure no single feature is released without thorough testing. Rather than getting slowed down by complex requirements and traceability matrices, the team leverages user stories, acceptance criteria and screenshots to release products and features in weeks, rather than months.

With Applause, Vodafone places users at the heart of its in-sprint testing efforts. Crowdtesting complements Agile QA because it ensures that the customer voice is heard at key stages of the SDLC. This customer focus enables Vodafone to deliver products that work and fit the market demand. Crowdtesting’s scalable, flexible model also meets the pace of Agile development. Rather than having full-time, monolithic internal QA teams, Vodafone drafts in customized, scalable Applause tester teams when it needs them, day or night.

Localization testing establishes meaningful connections with subscribers

With streaming media pushing consumers to cut the cord, it’s an imperative for brands to deliver access to relevant local programming around the world. Can a subscriber access the live local programming they desire, in the language they speak? Does that content delivery adhere to regional digital rights management standards and global content partnerships? Are you ready to provide regional support for eSIMS and iSIMS? How do you take into account the local and regional sensitivities of the subscriber? How well do your services work when roaming over international borders, and does that adhere to both legal requirements and customer expectations of how well it should work?

Entering new markets requires much more than simple language translations — to get new subscribers on board, digital products and platforms must reflect an understanding of the local language, values, preferences and processes. Seemingly small errors, like missing translations, incorrect country and region names, or workflows that don’t align with local customs can cause customers to cancel subscriptions and put the business at risk. Herein lies the nuance of localization testing, as media and telco brands must successfully navigate a tangled and constant web of local customs, legislation, expectations, devices, currencies and functional challenges to thrive.

Key observations

• The good news: Missing translations dropped 16% from one year ago. Hopefully this means media companies are investing more resources and attention into translations.

• The bad news: poor translation (3.5% increase), truncation and overlap (3%), and corrupted characters (2.6%) all went up, showing the myriad of ways brands can let down their subscribers with a poorly localized experience.

Case study: Navigating local sensitivities

A global streaming media company turned to Applause for help with its localization strategy. Applause conducted test cycles on hundreds of device/OS combinations across more than 80 countries, including a variety of Arabic-speaking countries in Africa, Asia and the Middle East. Through in-market localization testing, Applause helped this company navigate the various cultural and technical challenges of this launch, such as ensuring the accuracy of right-to-left Arabic text and validating in-market streaming rights.

This market launch presented a particular challenge along Israeli-Palestinian borders. The historical conflict in the Middle East presents extreme sensitivities on both sides. Thus, some Palestinians would bristle at even seeing Hebrew as a language option, and likewise for some Israelis seeing Arabic as a language option. Applause helped the streaming media provider validate its platform to navigate this exceptionally sensitive matter and ensure both sets of subscribers could access programming that matches their expectations.

Accessibility testing validates the subscriber experience for all

The World Health Organization’s latest estimate is that one in six people, or 16% of the global population, lives with some form of disability. Further estimates suggest that friends and family of persons with disabilities (PWD) account for an approximate 3.3 billion potential consumers who act based on their connection to PWD.

Leading global media companies have realized that digital experiences that people with disabilities can navigate easily often create a better UX for all of their customers. For example, accurate closed captioning appeals to PWD as well as subscribers who are watching content in a loud environment like a coffee shop.

It’s clear that a focus on accessibility helps everyone, and as organizations commit to more inclusive design and better digital accessibility, the bar gets higher. In a recent survey, 15.7% of respondents report canceling a subscription because of accessibility issues. Unsurprisingly, accessibility is taking on a higher priority for media and telco brands. One Applause client, for example, went from facing fines a few years ago, to building an internal team to test accessibility issues in-sprint. And that’s just the tip of the iceberg, as a shift left to inclusive design can identify and remediate accessibility challenges even earlier in the SDLC. Winning media and telco brands with accessible products tap into the total addressable market and avoid legal exposure, doing right by both their brand and PWD in offering high-quality experiences for every customer.

Key observations

• Screen reader defects continue to make up the vast majority of all accessibility errors. While many streaming media providers offer native or third-party screen reader support, the technology — or the implementation of it — doesn’t go far enough to support subscribers with vision impairment.

• Bug type distributions remained consistent, typically fluctuating less than 2% +/- from last year. One area of increase: color and color contrasts, which accounted for 16% of bugs this year compared to 12.5% last year.

Case study: Creating a culture of inclusivity

A leading network provider made a significant commitment to make its video conferencing application collaborative, accessible and inclusive of all people with varying abilities and backgrounds. It embraced a shift-left approach to bring inclusive design and innovation earlier in the SDLC. Its focus went beyond simply meeting WCAG standards: It wanted to stimulate innovation in its organization that would derive from involving the most diverse population it could, emblematic of its users around the world.

As the company developed its empathy-based approach to inclusive design, it rallied team members to become ambassadors and help spread the word throughout the organization. The company got management buy-in, and as momentum continued, the company partnered with Applause to provide education sessions including PWD demonstrating their interactions with the conferencing platform to broaden awareness of the overall empathy-based inclusive design program. To keep up momentum, it built a repository, an accessiwiki, that serves as a resource employees can use to get a wide range of information around accessibility — from WCAG standards explained to internal initiatives, inclusive design basics and more.

Payment testing keeps revenue streams healthy

Without thorough payment testing processes in place, organizations can’t confidently ensure transactions work as intended. As digital wallets, alternative payment methods, and options like buy now, pay later (BNPL) become more popular — yes, even for subscription services, not just the latest phone — payment testing must adapt and evolve to cover these new payment instruments and paths to purchase.

Think about the myriad ways customers subscribe and make purchases with your services. To meet customer expectations and entice subscribers, streaming media companies offer promotional rates, free trials, bundled discounts and different tiers of service. At the same time, these platforms have evolved to meet revenue needs, such as limiting access to a certain number of devices and launching first-run movie releases. Telco companies offer similar promotions and bundles, plus charges and payments that apply to extra service or data usage. All of these features or promotions tie back to payment flows that must be validated.

Real-world testing with live, relevant in-market payment instruments is the only way to guarantee transactions work properly for every customer, every time. These transactions typically must occur in a production environment with real transactions, making for additional challenges as telco and media brands look to expand globally.

Key observations

• The prevalence of each type of payment bug remained remarkably consistent with last year, varying less than 1% from year to year. A whopping 80% of all payment bugs were classified as functional bugs or workflow errors. Without corrective action, transactions will fail, delivering a serious hit to revenue.

Customer journey testing supports the subscription model

Our customers use many dimensions to assess digital quality, including a range of metrics directly from customers. You need to understand and evaluate all the interactions subscribers have with your brand and products: myriad complex sets of transactions and tasks that often meander across varying channels. Not all organizations have the specialized skills and resources necessary to conduct thorough UX research covering all customer journeys, from end to end. And, as they expand into new markets, even those organizations that do have dedicated UX research teams may have difficulty quickly sourcing research participants.

While telco companies might offer blended physical and digital journeys, such as buy online, pick up in-store (BOPIS), streaming media subscribers follow multi-step digital journeys. These subscriber paths might be nuanced, with multiple in-market devices affecting the customer experience. These subscriber journeys must be tested quickly and effectively post-launch to grasp the challenges faced by real in-market users, rather than simulated experiences that fail to account for device sprawl, connectivity issues, bundled subscriptions and other nuances.

In the United States, 37% of survey respondents canceled a streaming media subscription over the past six months — and those rates were higher for millennial and Generation Z respondents. While mobile telecom churn rates tend to be in the single digits, the goal remains the same for both highly competitive industries: provide friction-free customer journeys that satisfy subscribers.

Statista report

Using AI, voice and chatbots to enhance the customer experience

AI and voice are taking on expanding roles in personalization and customer support for streaming media and telecommunications companies. Brands can enhance the subscriber experience through AI-aided localization efforts, intelligent recommendation engines and better customer support. Consequently, the quest to hire staff with more specialized and in-demand skill sets becomes even more intense.

Applause’s 2023 AI and Voice survey found that of more than 5,000 respondents worldwide, 92% typically expect companies to have chatbots on their apps or websites so they do not need to call for support. If customers do have to call, 86% always or sometimes expect to be greeted by an IVR system when they call.

In addition, consumers are concerned about bias in AI due to poor or insufficient training data: 86% of survey respondents indicated some level of concern. While only 19% stated they were very concerned about bias, 41% indicated that their level of concern depended on the situation and use case. Respondents expressed similar sentiment around whether AI technology should be regulated, with the majority (53%) saying regulation should depend on its use.

With customer concerns and expectations in mind, organizations must have a strategy in place to incorporate chatbots, IVR, and AI into customer touchpoints in ways that streamline and enhance the overall experience. For example, an AI-powered recommendation engine can evaluate past viewing choices and suggest new content. Thorough testing and adequate ML training are crucial parts of that equation.

Digital quality recommendations

Media and telco companies are at the forefront of a massive shift in how customers interact with a brand. The subscriber experience for these companies presents nuance: roaming travelers, mobile-first services, set-top boxes, high expectations for streaming content and lengthy content consumption sessions, sometimes for work and sometimes for leisure. There’s no room for error — and there’s no buffer zone for buffering.

Mapping to the five primary areas of digital quality mentioned above, here are five recommendations that can help media and telco organizations chart a path to subscriber bliss:

Serve your subscribers. Start by understanding your current state of product development, which may be siloed or inconsistent across different teams, products, groups and even individual team members. Find what’s working and replicate that — as well as what’s not working and needs to be addressed. Build in time for developers and testers to document processes, review with teams, and regularly update to ensure they still work and that everyone’s following those processes. Digital quality excellence is a constantly moving target as new technologies and innovations come to market. While you may be highly automated and mature with some apps, to maintain competitive advantage, you’ll likely have to develop new features that your customers demand. For example, if your primary competition offers an ad-supported tier and you don’t, that’s a powerful differentiator for them. It’s crucial to maintain your investment in quality. The ability to quickly adapt and scale depends on maintaining the capabilities you’ve built. Guard them fiercely to reduce risk and maintain your competitive advantage.

Go hyper-local, globally. Double-check accuracy of localization efforts and nuance with double-blind validation. Translators and testers each bring their own background and linguistic preferences to the tasks at hand. Those can influence how they interpret various translations. If there’s disagreement on a particular meaning or translation, it’s best to understand and resolve it before releasing your website or app to customers. While you’re at it, maintain a translation glossary for frequently used terms and company-specific language. Save time with a resource that developers can refer to as they localize apps; update and expand as you enter new markets or localize additional features, products and websites. Be consistent. Also, include customer support and post-purchase experiences in your localization efforts. Commit to supporting your subscribers in their local languages and cultures throughout the full experience with your brand.

Maintain compliance and empathy. We meet with customers all the time who are unaware that they’re violating regional accessibility laws. Even if you’re located outside of Europe, for example, the European Accessibility Act (going live in 2025) will have strict penalties for companies doing business in the EU that fail to comply. In addition, certain governments may impose additional regulatory requirements, like the Twenty-First Century Communications and Video Accessibility Act for media and telecommunications providers. But accessibility isn’t just a litigious issue; it’s a moral one. Find allies in the organization who are passionate about accessibility to serve as ambassadors and drive awareness. Seek out like-minded colleagues, start sharing ideas, then start with some small changes. If you’re stuck, get an executive sponsor on board to help facilitate awareness, training and input from PWD into the design process. The result is a better UX for all subscribers.

Keep it real with payments: real cards, real transactions, real insights. Test with real payment instruments to capture actual intelligence about your payment flows. Without real payment instruments, you can’t validate information in more complex flows like multifactor authentication, how a charge appears on a credit card statement or whether certain payment methods or BINs aren’t working properly. As the product matures in the market, use data to drive decisions about which payment methods and instruments to offer or accept when expanding into different markets. As new digital wallets, cryptocurrencies, BNPL partners and alternate payment providers emerge, deepen your understanding of your subscribers’ preferred payment methods in each market where you operate to determine whether you should add them into the mix.

Focus on points of friction. If you can pinpoint what features or issues cause subscribers to churn, you can focus efforts on that pain point rather than a costly end-to-end customer journey analysis. Look at the data to determine which elements of the customer journey demand attention, then test to understand how you can improve and optimize. UX and customer journey testing aren’t one-time assessments – they’re ongoing evaluations to understand subscriber perceptions, inform the product development process and guide market strategy as consumer preferences shift and evolve. Revisit as you add new features, make substantial changes to workflows and enter new markets. Find the cadence that makes the most sense for your platforms and services.

Index and data references

Use this report to understand the gaps in your organization’s capabilities and processes and to map out a plan to improve your digital quality, regardless of your company’s current state. Assess your digital quality maturity by comparing yourself against our digital quality frameworks, located in our primary State of Digital Quality 2023 report here.

For a complete view of media and telco’s state of digital quality data, visit this year’s State of Digital Quality in Media and Telco report. It contains details on the following key areas:

• Methodology used

• Bug type definitions

• Bug breakdown charts

• Charts illustrating browser and OS combinations (desktop) and device and OS combination (mobile & tablets) and more