Migrating Service Worker Tests to WPT

Google

Unifying and improving divergent test forks for the benefit of all.

The Google Chrome team wrote a large set of valuable tests for a new web standard. The team asked us to move them to a place where more people could use and improve them.

Challenge

The Google Chrome team had accumulated close to 100 tests for the developing Service Workers standard. They were authored to run within the Chromium infrastructure (Chromium is the open source project behind Google Chrome) and relied on non-standard features to varying degrees. Some included erroneous assertions based on outdated versions of the specification. Still others only offered partial coverage of the functionality under test.

Solution

We worked to transfer these tests to the Web Platform Tests project, where they would be consumed and maintained by many browser makers. This involved, verifying specification compliance, removing non-standard code, re-formatting code, and improving general coverage and thoroughness. Throughout this process, we took special care to preserve test coverage within the Chromium project.

Impact

By extending the Web Platform Tests project with Chromium’s tests, we helped other browser makers discover incompatibilities in their own implementations. We reported bugs to both Chromium and Firefox, and expect that more were identified in the closed-source Safari and Edge projects. WebKit developers reported that access to these tests while implementing Service Workers was extremely useful, giving them confidence that their implementation was interoperable with other browsers, and helping them find critical bugs.

We also reported issues with the Service Workers specification itself and submitted patches for various editorial problems.

It's been awesome having Bocoup driving both the tooling infrastructure and key strategic test suite investments here! Your team has shown the same productivity, independent drive for excellence, and good judgement under ambiguity that I expect from Googlers!
—Rick Byers, Google