Ensuring Correctness of W3C Accessibility Examples

The Authoring Practices Guidelines (APG) help web developers gain a deeper intuition for the purpose and use of ARIA, the accessibility API for the web. This is done via the definition of design patterns that use features in web browsers specified by the ARIA specification. These design patterns show how modern day, interactive web applications can be accessible to screen readers with the use of ARIA roles, attributes and expected key handlers for keyboard interaction.

The APG includes HTML and JavaScript examples for the design patterns to help web developers get started. The example implementations are frequently complex, with a multitude of keyboard interactions and necessary attributes. Without a any automated regression testing, the APG did not have a way to comprehensively check for bugs in future contributions to the guidebook and example code.

We worked closely with the W3C ARIA Authoring Practices Task Force to produce a testing framework that consumes the examples on each example page in the APG. We wrote tests in Node.js using the open source AVA framework and Selenium WebdriverJS to hook into the browser, allowing for interactive tests of the examples. Over the course of three months, we wrote 583 tests for 40 example widgets, and produced extensive documentation of the framework, test norms, and use of AVA and SeleniumJS.

All tests run on TravisCI to provide continuous integration testing for future work on the Authoring Practice Guide. In addition to the tests, we created an automated report, which runs on every pull request and details how comprehensive the regression tests for a given example page are.