Ringmark Tests Open Source

I am really excited that the tests powering Ringmark have been pushed up to Facebook’s fork of coremob/coremob-tests repo on GitHub. Read more about it on Facebook’s dev blog.

We think that unit test authoring will be a great way for web developers to engage in the standards process, and so we are looking forward to seeing new tests from the community.

When we kicked off this project with Facebook, we were presented with the challenge of authoring modular, well-formed and practical tests against the existing W3C JavaScript Testharness.

In order to overcome the challenges presented by testharness.js, Rick and Adam wrote a shim that encourages all tests to be written through a single assert() function, which maps to the basic assert function in a given test runner.

Using a single assertion interface enables us to develop tests using our preferred JavaScript tool chain and best practices; while allowing for the flexibility to build the tests into static files running testharness.js.

This process exposed us to a number of issues with the W3C testing system, and the way that testing is managed at the W3C. Testharness.js is undocumented, has inconsistent API signatures, inconsistent function naming conventions, and broken implementations of deep equals and async test. What’s more, working groups currently write and approve tests in silos, taking disparate approaches to code style, organization and testing helpers. This makes it so we can’t run cross working group test suites.

At Bocoup, we strive for a high standard of code quality. At the same time, we realize the importance of working within the testing paradigm that has formed at the W3C to expedite the adoption of our tests. That’s why the tests we pushed up to GitHub run on testharness.js.

Keep in mind, however, that all the tests are still using a single assert() function, making these tests easily portable to other systems, both now and in the future. So if you write your tests using the assert() shim, your tests will be more portable, and more futureproof.

We built some great tools that simplified test authoring and iteration. We’ll be working to open source them, and make them part of the standard W3C Open Web platform testing workflow.

In the meantime, we encourage the community to join us in authoring new tests for the Coremob Community Group, the tests that power Ringmark. Go ahead and fork coremob/coremob-tests, write new tests using the assert() API and file pull requests. Take a look at the tests we wrote on facebook/coremob-tests for reference.

We believe that the tools and workflow we’ve developed, and the testing principles that we employed as a part of building Ringmark are the way testing should be done at the W3C. We’ll be working closely with Facebook and the W3C over the coming months and years to improve testharness.js, and to ratify new approaches to managing testing at the W3C.

We are deeply committed to moving the Open Web forward. Here’s to the future of a well-tested Open Web platform!