Web Platform Tests Infrastructure Maintenance

Google

Burning Down Open Bugs in WPT

View project

Learn about our work to stabilize Web Platform Tests infrastructure in order to make this shared test suite more reliable for browser teams.

Challenge

The Web Platform Tests project was growing both in terms of contributors and functionality. While the project is always excited to welcome new contributors, this expansion was exposing weak spots in infrastructure code and increasing pressure on third-party services. This kind of volatility was particularly challenging because it discouraged browser makers from using the shared suite of tests as a part of their in-house development process.

Solution

We took ownership of infrastructure issues and feature requests by publicly committing to investigating and resolving them. The form of our solutions varied widely, including modification of infrastructure code, additions to project documentation, and bug reports to external dependencies. The problems and their solutions were tracked with a dedicated label on GitHub.com, and they occasionally warranted further communication via the project mailing list (e.g. "Recent Instability in WPT").

Impact

About 60 of our infrastructure patches were merged over the course of 3 months, and we triaged many more infrastructure bug reports. (A partial listing of our efforts in this area is available online.) We encouraged confidence in the infrastructure despite occasional instability by publicly documenting our findings and demonstrating our (e.g. "[update] Recent instability in WPT"). Due to the irregular frequency/severity of problems, we were able to complete unrelated tasks concurrently.

Thanks [...] for continuing to invest heavily in the tooling and making using WPT easier and more enjoyable! Keep up the great work!

Rick Byers, Google

Contact Us

We'd love to hear from you. Get in touch!

Boston

201 South Street, Boston, MA 02111

New York

315 Church St, New York, NY 10013

Phone & Email

(617)379-2752 hello@bocoup.com