Anatomy of a Streaming Media Company Pilot

When a company decides to do a pilot with Applause, there is a lot they don’t know – and that’s the point. Of course, before the pilot begins, the scope is clearly defined and all parties agree on what the engagement will encompass, but we’ve found that most companies we work with on pilots find the results and insights stunning.

In this blog, I highlight a pilot engagement with a global media company to assess their digital accessibility maturity.

Prioritizing accessibility

There were a few key reasons the media company prioritized accessibility for its pilot:

  • Broadening its audience - Accessibility would allow the media company to serve a wider audience and be better positioned to achieve the long-term subscriber goal, as limited accessibility could mean missing out on 15% of users with disabilities, who wield significant buying power.

  • Fine and lawsuit risk mitigation - Pursuing accessibility excellence would help avoid fines and legal action. Non-compliance with CVAA (Twenty-First Century Communications and Video Accessibility Act) can result in fines up to $100,000 for each offense, with some fines continuing daily until resolution. In addition, ADA (Americans with Disabilities Act) web-related lawsuits continue at a significant pace.

  • Improved user experience for all subscribers - Optimizing the UI/UX for accessibility benefits all users, regardless of abilities.

  • Better brand image - Creating frictionless digital experiences for all customers helps improve customer loyalty.

Pilot components and key findings

For this pilot, lasting about 8 weeks, Applause executed one in-market accessibility assessment which included up to 15 pages/screens for agreed-upon platforms. The media company prioritized the OTT (over-the-top) environment. Testing considered the traditional TV practice of assuming the viewer is watching on a large screen at approximately 10 feet of “leanback,” the average distance from the viewer to the screen. Applause did one inclusive design study focused on persons with low vision. We enlisted 5 people who did 60 minutes of testing each. Lastly, Applause completed one inclusive design review with an expert which included up to 15 hours of consulting on accessibility and inclusive design.

Accessibility assessment

The audit revealed 48 issues, two of which were critical severity and 21 of which were high severity. The majority of issues had to do with incorrect or missing screen reader feedback. In addition, there were issues with incorrect heading levels and screen readers skipping over important UI elements.

Inclusive design study

Five low-vision users gave feedback on the media company’s streaming experience using Roku devices. All participants mentioned that some text was hard to read because of low color contrast ratios and small font size. Searching for a particular title was a challenge via Roku, and keyboard layout was difficult for some. When testers used the “save” function, no confirmation message was given to show that media was actually saved. Sign-in was particularly difficult for one tester, taking 7 minutes.

Inclusive design review

The inclusive design review found a solid foundational visual design and the need for only a few basic tweaks such as better consideration of contrast for foreground text and background imagery. The primary feedback to the media company was that designs did not include any non-visual requirements. For example, when a customer uses a Roku remote device, it has basic navigation of up, down, right, left and enter for input. When designs don’t account for these non-visual aspects of a UI, developers have to fill in the blanks with their own interpretations with varied degrees of success.

Applause suggested design annotation for things such as focus order, button labels and semantics. In addition, we helped create a customized checklist that can be embedded in Figma, their chosen design tool, allowing their designers to ensure nothing is missed in future layouts. Applause also found that the media company's designs did not adhere to system accessibility settings for font size, style or magnification.

How Applause assesses maturity in accessibility

Applause’s maturity model for accessibility looks across five key qualities of an organization: vision, champions, operational processes, capabilities and KPIs. We then classify organizations as one of four maturity levels: informal, active, defined and optimized. As a result of this engagement, Applause found the following maturity level:

Vision = defined. This means an accessibility vision has been discussed but not endorsed by leadership.

Champions = informal. One or more individuals in the organization (product, design, engineering) was trying to change internal perception around the importance of accessibility.

Operational processes = informal. There were no processes defined for reviewing products or processes against any standards.

Capabilities = informal. Lacking regular training or formal assessment of skill set. No visible declarations of accessibility on websites or apps.

KPIs: informal = None defined for accessibility.

In addition, the media provider is operating in an elevated risk environment vis-à-vis CVAA requirements.

Recommendations resulting from pilot

When we complete pilots with customers, we create and prioritize a list of recommendations in order of highest priority to least. As part of this process, we also consider staffing, priorities of the organization and potential stress on the customer that may result from specific remediation recommendations. In the case of the media company, here’s what Applause suggested:

In the high priority grouping:

  • Schedule assessments integrated into the SDLC: specifically, accessibility bug fix verifications

  • Basic training for key stakeholders including design enablement and engineering enablement training

  • Author an accessibility statement to guide all related efforts

  • Set KPIs and executive stakeholder training

  • Test OTT experiences with UX research

In the medium priority grouping:

  • Perform accessibility expert wireframe and design review

  • Execute “Intro to Accessibility” training sessions

  • Perform exploratory UX research for other platforms

Rationale for addressing accessibility issues

Addressing pilot issues has obvious and less obvious objectives. As mentioned, by proactively assessing its accessibility status, the media company mitigates legal risk, but also sets a baseline for product improvements. There are many other motivators for addressing accessibility hurdles:

  • Build a culture of empathy and an understanding of the impacts of inaccessible content

  • Shift left in the SDLC to design inclusive and compliant products for all users, increasing development velocity and minimizing developer stress

  • Lower total cost of ownership and increase developer efficiency by minimizing accessibility and inclusivity issues and related poor user experiences

  • Receive education and training on accessibility program design and implementation for maximum program success

  • Improve cross-functional team collaboration and communication

  • Understand a product’s usability for all customers including those with disabilities

  • Identify design components that cause accessibility issues prior to building in order to reduce overall costs

  • Design and develop with people with disabilities instead of for people with disabilities

Learn more about how a pilot may be the right next step for your organization, please contact us.

Want to see more like this?
William Reuschel
Inclusive Design Practice Lead
Reading time: 5 min

Some Key Considerations for Testing Blockchain

Blockchain technology is changing the ways people think about — and process — secure transactions across different sectors. Transparency and testing are crucial in helping people trust the results. Explore some use cases and quality considerations.

Generative AI Use Is Growing – Along With Concerns About Bias

See the results of Applause’s survey on generative AI use and user sentiment

How to Land Your Dream Software Testing Job

Here’s how to stand out in a competitive yet thriving QA job market

What Is Continuous Testing in DevOps?

Learn about continuous testing benefits, tools and frameworks

Digital Quality as a Service: A Necessary Evolution of Software Testing

Read how Digital Quality as a Service (DQaas) can help organisations to deliver the very best digital experiences.

Creating Seamless B2B Software Experiences for All Users

B2B software companies must ensure they cater to multiple audiences, often with varied needs, to stay ahead in a competitive market.