Accessibility Testing: Everything You Need To Know

Accessibility and testing for accessibility is critical to organizations big and small around the world. Yet, according to the Baymard Institute, 94% of the largest e-commerce sites are not accessibility compliant. Not being compliant on your digital properties creates issues for potential customers and affects current customers who may experience temporary disabilities. It also can impact revenue and lead to legal issues. Inclusive design, a close sibling to accessibility, is another key component for organizations to address. It goes beyond compliance to create digital experiences that consider a broad spectrum of abilities and needs from project inception. Inclusive design should also involve people with disabilities throughout the entire software development lifecycle (SDLC) – particularly in design and testing.

How should your organization begin, or continue on, this complex, iterative and multi-faceted accessibility journey? In this blog, we address 17 questions that we believe should be answered for organizations working along the accessibility continuum.

What is accessibility testing and why is it important in software development?

Accessibility testing is an important part of software development that focuses on ensuring that software, websites, applications, and digital products can be easily used by people with disabilities (PWD). The goal of accessibility testing is to find and address barriers or friction that individuals with disabilities might face while using software.

The process of accessibility testing evaluates software for compliance with various accessibility standards and guidelines, such as the Web Content Accessibility Guidelines (WCAG). Testers use assistive technologies and various additional testing types, such as manual functional testing or automated functional testing, to gauge the accessibility features and identify areas for improvement.

Reasons why accessibility testing is important for software developers:

• Inclusivity and equal access: Accessible products and services make it possible for people with disabilities to have equal access to the digital world. Accessible designs also improve the overall experience for everyone, not just PWD.

• Legal compliance and protection from lawsuits: Many countries have laws and regulations that mandate software accessibility, such as the Americans with Disabilities Act in the U.S. and the Equality Act in the U.K. Non-compliance can lead to legal issues and result in major financial and reputational consequences.

• Expanded user base and market reach: Improving accessibility broadens the potential user base of a software product. People with disabilities constitute a substantial portion of the global population with significant buying power, often augmented by friends and family of PWD who gravitate to brands that prioritize inclusivity.

• Corporate social responsibility and brand reputation: Demonstrating a commitment to accessibility aligns with corporate social responsibility values. This also boosts brand reputation and customer loyalty.

• Early detection of issues/cost savings: Addressing accessibility concerns during the development phase is more cost-effective than making extensive modifications later in the development cycle or after the product is released.

• Compliance with industry standards: Many industries have specific accessibility requirements that ensure that software is compatible with specific platforms and assistive technologies.

What are the key principles and guidelines for accessibility testing?

Accessibility testing is guided by various guidelines and principles to help ensure that software applications and websites are accessible to people with disabilities. Two primary guidelines:

Web Content Accessibility Guidelines (WCAG):

WCAG, developed by the World Wide Web Consortium (W3C), offers guidance for making web content more accessible. The latest version – WCAG 2.2 – was released on October 5, 2023. Like its predecessor, WCAG 2.1, 2.2 are organized around four fundamental principles often referred to as POUR:

• Perceivable: This includes things such as providing text alternatives for non-text content, providing captions and more.

• Operable: This involves keyboard accessibility, providing sufficient time for users to read and use content, and avoiding content that could potentially cause seizures or physical discomfort.

• Understandable: Users should be able to understand and navigate the interface, as well as predict how the system will behave. This includes clear and consistent navigation, readable text, and input assistance.

• Robust: Content must be robust enough that it can be reliably interpreted by a wide variety of user agents, including assistive technologies.

Section 508:

Section 508 is a part of the U.S. Rehabilitation Act of 1973. It requires federal agencies to make their electronic and information technology accessible to PWD. The guidelines cover broad requirements: software applications, websites, telecommunications, and multimedia. Some of the key areas of focus include providing text alternatives, keyboard accessibility, and compatibility with assistive technologies.

Compliance with Section 508 is key to ensuring that digital products are accessible to PWD. These guidelines also help organizations avoid legal issues related to accessibility compliance.

How can accessibility testing be used to comply with accessibility regulations and standards?

Accessibility testing is a pivotal tool for organizations to align with accessibility regulations and standards effectively. Here's how it can be employed:

Understanding regulations and standards

• In-depth awareness - helps organizations gain a thorough understanding of the specific accessibility regulations and standards applicable to the target audience or region, such as ADA, WCAG, or Section 508.

• Interpretation and analysis of guidelines - helps decipher the precise requirements and criteria that the software must meet for compliance.

Tailoring testing approaches

• Aligning with standards - enables development organizations to structure the accessibility testing strategy to align with the defined standards.

• Addressing compliance criteria - enables the creation of a comprehensive test plan that encompasses all relevant compliance criteria outlined in the regulations and standards.

Verification and compliance

• Automated assessment - helps teams use automated accessibility testing tools to perform an initial assessment. This helps identify common issues and areas that need improvement to comply with the standards.

• Manual validation - helps validate automated results and address nuanced aspects that may not be adequately captured by automated tools.

• Iterative improvements - focus on testing feedback to ensure compliance with the specified accessibility standards and regulations, demonstrating a commitment to inclusivity and accessibility.

What is the difference between accessibility and inclusive design, and how do you test both?

Accessibility and inclusive design are related. They focus on providing equal access and usability to individuals of varying abilities. Accessibility addresses the removal of barriers so people with disabilities can effectively interact with digital products and/or physical environments. It aims to comply with WCAG, ADA and Section 508 and focuses on technical and functional requirements such as proper labeling, keyboard navigation, alternative text for images and color contrast. Inclusive design goes beyond compliance to create products that consider a broad spectrum of abilities and needs from inception. It involves designing in a way that accommodates diverse users – often involving these targeted users in product concept and design phases – regardless of their abilities, preferences or circumstances, by considering various aspects like different disabilities, language, culture, age and literacy levels. It strives to provide a seamless and meaningful user experience for everyone.

Accessibility testing and inclusive design testing requires a comprehensive approach. For accessibility testing, organizations should use automated and manual testing methods to evaluate compliance against established accessibility standards (e.g., WCAG). Automated tools can quickly identify common accessibility issues, but manual testing is required for nuanced evaluations and a deeper understanding of the user experience. Conduct tests using assistive technologies like screen readers, magnifiers and voice recognition software to simulate real-world usage by individuals with disabilities. Inclusive design testing involves gathering feedback from a diverse user base representing various abilities, cultures, languages and demographics. User research, usability testing and feedback from user personas with different needs and abilities are crucial components of testing inclusive design.

Are there different ways to test accessibility for different types of applications such as web applications, mobile applications and desktop applications?

Accessibility testing for different application types involves custom approaches for the various web, mobile and desktop platforms. Successful accessibility testing requires an understanding of unique features and interactions of each type of application and adjusting the testing approach accordingly. Combining automated tools, manual testing, adherence to platform-specific guidelines and involving users with disabilities for real-world feedback are essential strategies for building a high level of accessibility in web, mobile, and desktop applications.

Specifically, accessibility testing for web applications includes evaluating page structure, keyboard navigation, color contrast, form labels, and multimedia alternatives. Automated tools and manual testing using screen readers and keyboard-only navigation are also used to identify and fix accessibility errors.

For mobile applications, testing typically focuses on aspects such as touch gestures, screen reader compatibility, text scalability, and adherence to platform-specific accessibility guidelines such as iOS Human Interface Guidelines and Android Accessibility. Testing on actual devices using mobile accessibility testing tools and involving users with disabilities in testing are great ways to establish comprehensive accessibility.

Finally, for desktop applications, testing should encompass keyboard support, UI navigation, focus management, text resizing and compatibility with assistive technologies. Using automation for repetitive tests along with manual testing using screen readers and keyboard input are key to assessing and improving the accessibility of desktop applications.

Can artificial intelligence be used for accessibility testing?

While AI offers significant potential in automating and enhancing accessibility testing, it's important to note that human expertise and manual testing remain crucial components of the process. Manual testing is essential for contextual understanding, inclusive evaluation of user experience, identification of non-standard usage patterns and dynamic adaptation to evolving accessibility challenges.

Human testers can interpret complex use cases, ensuring genuine inclusivity across diverse scenarios. They can also provide insights into user perceptions and unconventional usage patterns that automated tools – even those powered by AI – might miss. Relying solely on automation often leads to oversights in understanding nuanced contexts and unique user experiences, potentially missing critical accessibility issues that require human insight and judgment. With human testers, organizations can create truly accessible software.

As such, AI tools should be used in conjunction with human validation to ensure accurate results and a thorough evaluation of accessibility compliance.

What are the best ways to test for accessibility issues related to visual, auditory, motor, and cognitive impairments?

Testing for accessibility issues around visual, auditory, motor and cognitive impairments is critical to ensure that digital products and services are usable by the widest range of individuals. Incorporating the following methodologies will help identify and address accessibility issues, ensuring a more inclusive and user-friendly digital experience for all individuals.

Visual impairments:

• Use screen reader software to navigate through the interface and evaluate the comprehension and flow of content.

• Adjust screen contrast and brightness to simulate different visual conditions and assess readability.

• Test keyboard navigation – without relying on a mouse – to ensure all elements are accessible.

• Analyze text alternatives for images and other non-text elements to check for most meaningful descriptions

Auditory impairments:

• Evaluate the content and interactions for alternatives to audio cues, such as visual or text-based alerts.

• Assess if critical information is adequately presented through visual means when audio is disabled.

• Test with closed captions and transcripts to check the clarity and accuracy of the information provided.

• Ensure the application provides the a to adjust volume and mute audio without losing functionality or critical information.

Motor impairments:

• Use different input devices (e.g., keyboard, mouse, touch screen) to verify ease of use and accessibility.

• Test with limited dexterity/fine motor control to ensure all interactive elements are reachable and usable.

• Check for adequate time limits and the ability to extend or pause time-sensitive interactions.

• Assess the efficiency and ease of completing tasks using various assistive technologies like speech recognition or adaptive keyboards.

How does accessibility testing integrate into the development and QA processes?

Integrating accessibility testing seamlessly into the development and QA processes is key for ensuring inclusive and accessible digital products from the start. Here's one approach to do this:

Development process:

• Incorporate accessibility requirements by including accessibility considerations in the initial design and requirements phase of a project, making it an integral part of the development scope.

• Educate development teams by conducting regular training sessions on accessibility guidelines and best practices.

• Automate accessibility testing by implementing automated accessibility testing tools in the development pipeline to catch issues early in the SDLC and provide rapid feedback to developers.

• Include accessibility checkpoints in code reviews to ensure adherence to accessibility guidelines and identify any potential issues early in the development process.

QA process:

• Build a comprehensive set of accessibility-focused test cases and integrate them into the QA testing suite. Cover a range of impairments and assistive technologies.

• Conduct user testing with diverse users with varying abilities to collect valuable feedback and insights that can drive improvements.

• Integrate assistive technologies such as screen readers and voice recognition software into the testing environment to simulate the experiences of users with disabilities.

• Conduct regular accessibility audits to check compliance against industry standards and guidelines, identifying areas for enhancement and on-going improvement.

By embedding accessibility considerations into both the development and QA processes, dev organizations can proactively identify and address accessibility issues and deliver more inclusive and user-friendly products.

What are the best practices for accessibility testing?

Incorporating the following best practices into your accessibility testing approach will enhance the quality of digital products and make them more accessible and inclusive for all users.

Focus on diverse testing scenarios:

• Cover a range of disabilities: Include individuals with varying disabilities such as visual, auditory, motor and cognitive impairments to validate the accessibility features across a broad spectrum.

• Test various assistive technologies: Use a variety of assistive technologies like screen readers, voice recognition software and magnification tools to replicate the experiences of users with disabilities.

• Assess realistic user conditions: Test under different circumstances, such as low bandwidth, varying lighting conditions, or noisy environments, to evaluate the product's usability and performance in real-world situations.

Develop a continuous iterative process:

• Conduct early and frequent testing, beginning as early as possible in the SDLC and conduct iterative tests throughout the project to catch issues, reducing the cost of remediation.

• Develop comprehensive test plans that cover specific accessibility guidelines, standards, and success criteria relevant to the project, ensuring a thorough and structured evaluation.

• Engage accessibility experts to guide the testing process, offer insights and ensure that the product complies with relevant accessibility standards and best practices.

How do companies measure the impact of accessibility testing on user experience and accessibility compliance?

Measuring the impact of accessibility testing on user experience and accessibility compliance involves assessing the changes and improvements made to the product as a result of accessibility testing. It also means evaluating how these changes have affected the overall user experience and adherence to accessibility standards. Here's a structured approach to measure this impact:

Evaluate user feedback and testing metrics

• Gather feedback from users – especially those with disabilities – to assess their experience and satisfaction with the product after you have implemented accessibility improvements.

• Monitor key metrics such as task completion rates, time taken to complete tasks and error rates in post-accessibility testing. This helps quantify tangible impact on usability and efficiency.

Conduct compliance assessments and audits

• Conduct regular accessibility audits and compliance assessments using relevant standards (WCAG, for example) to quantify the level of compliance after accessibility testing and improvements.

• Compare the pre-testing and post-testing compliance scores to demonstrate the direct impact of accessibility testing on achieving and maintaining compliance with industry standards.

By systematically comparing pre- and post-accessibility testing assessments, analyzing feedback from users, and employing a mix of quantitative and qualitative evaluation methods, you can effectively measure the impact of accessibility testing on both user experience and accessibility compliance. This data is valuable for making informed decisions and improvements to enhance accessibility in the long run.

What are some ways to address common challenges and obstacles encountered during accessibility testing?

Addressing challenges during accessibility testing is crucial to ensure an effective testing process and achieve a more inclusive product. To start, maintain a comprehensive understanding of accessibility guidelines and standards, such as WCAG. Ensure that the testing teams receive adequate training on these guidelines and stay updated with the latest developments in accessibility practices. Additionally, collaborate with individuals who have expertise in accessibility. Incorporate their insights and feedback to tackle complex testing scenarios. Establish a clear communication channel with stakeholders to address concerns and manage expectations throughout the testing process.

Automate accessibility testing wherever possible to expedite the process and increase efficiency. Leverage accessibility testing tools and frameworks that can identify common issues, saving manual effort and enabling testers to focus on more complex, nuanced aspects of accessibility. It’s important to note that automated tools don’t always catch all accessibility concerns, so complement them with manual testing to close this potential gap. Lastly, build a culture of continuous improvement by conducting post-testing reviews, learning from challenges encountered and implementing strategies to reduce obstacles in future projects.

How can WCAG or Section 508 standards be followed and integrated into the testing process?

Accessibility standards are crucial guidelines that provide a framework for creating inclusive and accessible digital experiences for individuals with disabilities. Two primary accessibility standards widely recognized and adopted are WCAG (Web Content Accessibility Guidelines) and Section 508 in the United States. These standards lend themselves to test-driven development.

To integrate WCAG and Section 508 standards into the testing process, start by thoroughly understanding the specific guidelines and requirements outlined by both. WCAG provides a comprehensive framework for web accessibility. Section 508 focuses on accessibility for electronic and information technology in the U.S. federal sector. Develop detailed test cases that align with these standards, covering aspects such as keyboard accessibility, alternative text for images, color contrast and more. These test cases should encompass a range of disabilities and assistive technologies to ensure comprehensive compliance.

Next, incorporate WCAG and Section 508 compliance checks into the regular testing process. Leverage automated accessibility testing tools to scan the application for adherence to these standards and identify non-compliant areas. Conduct manual testing to validate the automated results and delve into more complex interactions that require human judgment. Collaboration with accessibility experts or consultants can provide valuable insights and guidance throughout the testing process, ensuring that the product is aligned with the WCAG and Section 508 standards and is accessible to all users, including those with disabilities. Regularly update and refine test cases based on the evolving guidelines to maintain continuous compliance and improve the overall accessibility of the product.

What are the common tools and techniques used for accessibility testing?

There are many tools and techniques used to evaluate the accessibility of software applications, websites, or digital content. A comprehensive approach involving a mix of automated tools, manual testing, user involvement, and expert reviews is often the most effective way to ensure that software applications and websites are accessible to all users.

Screen readers - JAWS and NVDA, for example, convert text on a screen into speech or Braille, enabling individuals with visual impairments to access digital content.

Color contrast analyzers - tools such as WebAIM help evaluate the color contrast between foreground and background colors and ensure readability for users with visual impairments or color vision deficiencies.

Accessibility testing browser extensions - for example, WAVE or axe tools assist in evaluating web content for accessibility directly within the browser and provide real-time feedback.

Keyboard testing - checks the apps functionality using the keyboard alone (no mouse) to ensure that all interactive elements are accessible and work via keyboard commands.

Automated accessibility testing tools - tools like axe, Pa11y, Tota11y, and Google Lighthouse scan web pages or applications to identify potential accessibility issues based on predefined rules, helping testers find and prioritize problems.

HTML validators - HTML validators such as W3C Markup Validation Service check for markup errors and issues, helping to ensure that the structure of the content is compliant with web standards.

Screen capture and video analysis - Camtasia, Snagit and similar tools assist in capturing and analyzing video footage to evaluate the accessibility of multimedia content, including subtitles, captions, and visual elements.

Techniques:

Manual testing - Testers manually navigate through the application or website, using assistive technologies like screen readers and keyboard-only navigation, to identify accessibility issues and ensure the user interface is operable and perceivable. Involve individuals with diverse disabilities in the testing process to gather first-hand feedback and insights into the usability and accessibility of the application or website.

Focus on use cases - Prioritize testing based on the most common user interactions and workflows to ensure that critical functionalities and features are accessible.

Expert reviews and audits - Conduct reviews with accessibility experts who can analyze the application or website against established guidelines, identifying potential issues and suggesting improvements.

Regression testing - Incorporate accessibility testing as part of regular regression testing to ensure that new updates, features, or changes do not introduce new accessibility issues.

Documentation review - Evaluate documentation, including user guides and help content, to ensure the materials are accessible and provide clear information on using accessible features.

How do the tools available for accessibility testing differ in terms of features and performance?

There are many tools and technologies available for accessibility testing, each with its own set of features, strengths, and performance levels. Below is a short list of a few of them along with a brief overview of their features and performance:

axe Accessibility Checker - provides automated testing for WCAG compliance, identifies accessibility issues and provides actionable insights. It can be integrated into various development workflows, including browsers and development environments. Axe is known for its ease of use and quick scans, making it suitable for rapid and continuous accessibility testing during development cycles.

WAVE (Web Accessibility Evaluation Tool) - offers in-browser evaluation, providing detailed accessibility feedback by highlighting potential issues and suggesting improvements. It can analyze both static and dynamic content. WAVE is known for its user-friendly interface and ability to scan complex web applications, making it a preferred choice for evaluating web accessibility.

JAWS (Job Access With Speech) - provides auditory feedback to users by reading the screen. It interprets web content and applications, allowing users to navigate and interact with digital interfaces. It’s widely recognized for its accuracy and robustness, making it a staple tool for evaluating the auditory accessibility of websites and applications.

NVDA (NonVisual Desktop Access) - offers screen reading functionality similar to commercial options via a free open source program. It reads the content of web pages and applications aloud, aiding users with visual impairments in navigating digital interfaces. It is known for its cost-effectiveness and strong community support, often preferred for comprehensive testing and development of accessible digital products.

How can accessibility testing ensure that software is usable by people with disabilities?

By focusing on various aspects of accessibility such as visual, auditory, motor, and cognitive impairments, testers evaluate the software from the perspective of individuals with diverse abilities. For example, testing for screen reader compatibility and keyboard navigation ensures that individuals with visual impairments or motor disabilities can effectively interact with the software using assistive technologies. Assessing color contrast and text legibility caters to individuals with visual impairments and enhances their ability to comprehend and navigate the user interface.

Incorporating accessibility testing into the software development lifecycle allows for early detection and remediation of potential barriers, ensuring that the final product is accessible to a broader user base. Testers play a critical role in identifying accessibility issues and advocating for necessary adjustments, making certain that the software complies with relevant standards such as WCAG or Section 508. It’s important to involve individuals with disabilities in testing to incorporate their feedback. Developers can gain invaluable insights into the user experience, leading to a more inclusive and user-friendly software that meets the needs of all users, regardless of their abilities.

How can accessibility testing results be analyzed and used to improve the accessibility of software products?

Analyzing accessibility testing results is a crucial step to improve the accessibility of software products. Start by categorizing the identified issues based on severity and impact. Prioritize critical issues that significantly hinder usability for individuals with disabilities, such as screen reader compatibility or keyboard navigation failures. Establish a clear grading system or priority scale to facilitate efficient focus on high-priority issues for immediate resolution.

Next, take the feedback and insights from accessibility testing to guide design and development decisions. Collaborate with developers to implement necessary changes addressing the identified issues. Leverage the identified problem areas to enhance the user interface, adjust color contrasts, improve navigation, and optimize content readability. In addition, conduct user testing with individuals with disabilities to validate the improvements and ensure that the changes effectively address their specific needs. This iterative process, guided by the accessibility testing results, fosters continual enhancement and refinement of the software product to provide an inclusive and accessible user experience for all.

What are some common challenges faced when conducting accessibility testing and how can they be overcome?

Accessibility testing presents several challenges that can impede an effective evaluation process. Varying interpretations and implementations of accessibility standards can create inconsistencies in testing approaches. It's essential to establish a clear understanding of the relevant standards (e.g., WCAG) and provide training to testing teams to ensure a clear interpretation. A lack of accessibility expertise within the testing team may hinder the identification of nuanced issues. Collaborate with accessibility experts, involve individuals with disabilities in the testing process and seek external consultations to provide valuable insights and guidance in addressing complex accessibility challenges.

The ever-evolving nature of web technologies and frequent updates to web content can present challenges in maintaining accessibility compliance. Automated testing tools may not always keep pace with emerging technologies, necessitating a balance between automated and manual testing approaches. Regularly update testing tools, stay informed about the latest trends in web development and conduct a mix of automated and manual tests to help overcome these challenges and ensure ongoing accessibility compliance in a rapidly evolving digital landscape.

Ensuring test coverage across a wide spectrum of disabilities and assistive technologies remains a challenge. To address this, build a diverse testing team that includes individuals with diverse abilities. Utilize a range of assistive technologies during testing to comprehensively evaluate the software's accessibility.

Learn more about how to ensure accessibility and deliver inclusive digital experiences for everyone.