Screenshot of the Aria-AT Test Runner Prototype home page in an illustrated browser window.

Screen Reader Interoperability Test Suite Design


Developing a manual test suite for screen reader interoperability on the web.

View project

We worked with the ARIA and Assistive Technology (ARIA-AT) Community Group to develop a assertion model and testing workflow for testing AT interop and implement a prototype runner.


The web has an Assistive Technology (AT) interoperability gap. ATs are not interoperable enough for web applications to reliably deliver user interfaces to AT users.

Without AT interoperability, there is no way to determine if the patterns utilized in our web apps result in good experiences for users. Consistent, repeatable, and testable user interfaces are the backbone of software engineering. Without consistent user interface rendering for ATs, Facebook can’t build community for or bring AT users closer together.

With the release of ARIA/APG 1.2 in 2019, we finally have sufficiently stable standards for authoring accessible experiences and delivering them consistently to ATs. What we're missing is the coordination and consensus between AT vendors on how to measure the AT interoperability gap, and the tools to track progress towards closing it.


We worked with the ARIA-AT Community Group to design an assertion model and test planning workflow. We also designed a working mode for authoring tests and agreeing to results. The assertion model allows test authors to build out a set of assumptions about the expected behavior of a screen reader interacting with an accessible design pattern from the ARIA Practices Examples.

We're using this prototype as the basis for a new testing system that we're developing in 2020.


Through the course of implementing this test model, we were able to build consensus between web accessibility community members on how to comprehensively test AT interpretation of ARIA. With the working mode, we also have a playbook for how to build consensus between AT users, AT vendors, browser vendors and tech companies on what correct AT behavior should look like.

I love working with Bocoup. I cant imagine a better partner for standards strategy!

Matt King ARIA AT Community Group Chair and Accessibility Technical Program Manager at Facebook
Two layered screenshots of the Aria-AT prototype, each in its own illustrated browser window. The first is of a test plan for checkbox. The second is of test run results for the checkbox pattern.

Contact Us

We'd love to hear from you. Get in touch!


P.O. Box 961436
Boston, MA 02196