Facebook funded Bocoup to write comprehensive regression tests for the examples in the Accessible Rich Internet Applications (ARIA) Authoring Practice Guidelines (APG).
Ensuring Correctness of W3C Accessibility Examples
Writing a regression test suite for the accessible interaction design pattern examples in the ARIA Authoring Practice Guidelines (APG).
Challenge
The Authoring Practices Guidelines (APG) help web developers gain a deeper intuition for the purpose and use of ARIA, the accessibility API for the web. This is done via the definition of design patterns that use features in web browsers specified by the ARIA specification. These design patterns show how modern day, interactive web applications can be accessible to screen readers with the use of ARIA roles, attributes and expected key handlers for keyboard interaction.
The APG includes HTML and JavaScript examples for the design patterns to help web developers get started. The example implementations are frequently complex, with a multitude of keyboard interactions and necessary attributes. Without a any automated regression testing, the APG did not have a way to comprehensively check for bugs in future contributions to the guidebook and example code.
Solution
We worked closely with the W3C ARIA Authoring Practices Task Force to produce a testing framework that consumes the examples on each example page in the APG. We wrote tests in Node.js using the open source AVA framework and Selenium WebdriverJS to hook into the browser, allowing for interactive tests of the examples. Over the course of three months, we wrote 583 tests for 40 example widgets, and produced extensive documentation of the framework, test norms, and use of AVA and SeleniumJS.
All tests run on TravisCI to provide continuous integration testing for future work on the Authoring Practice Guide. In addition to the tests, we created an automated report, which runs on every pull request and details how comprehensive the regression tests for a given example page are.
Impact
The tests written by Bocoup caught and documented 33 bugs in draft of the Authoring Practice Guide examples when we first implemented it. The practice guideline examples for the ARIA specification promote accessibility in keeping with our core values of an inclusive web platform. This regression test suite continues to help the ARIA community maintain the practice guidebook examples, encourages new contributions, and helps them land smoothly.
More work like this at Bocoup
Contact Us
P.O. Box 961436
Boston, MA 02196