Learn about our work to stabilize Web Platform Tests infrastructure in order to make this shared test suite more reliable for browser teams.
Web Platform Tests Infrastructure Maintenance
Burning Down Open Bugs in WPT
View project
Challenge
The Web Platform Tests project was growing both in terms of contributors and functionality. While the project is always excited to welcome new contributors, this expansion was exposing weak spots in infrastructure code and increasing pressure on third-party services. This kind of volatility was particularly challenging because it discouraged browser makers from using the shared suite of tests as a part of their in-house development process.
Solution
We took ownership of infrastructure issues and feature requests by publicly committing to investigating and resolving them. The form of our solutions varied widely, including modification of infrastructure code, additions to project documentation, and bug reports to external dependencies. The problems and their solutions were tracked with a dedicated label on GitHub.com, and they occasionally warranted further communication via the project mailing list (e.g. "Recent Instability in WPT").
Impact
About 60 of our infrastructure patches were merged over the course of 3 months, and we triaged many more infrastructure bug reports. (A partial listing of our efforts in this area is available online.) We encouraged confidence in the infrastructure despite occasional instability by publicly documenting our findings and demonstrating our (e.g. "[update] Recent instability in WPT"). Due to the irregular frequency/severity of problems, we were able to complete unrelated tasks concurrently.
Thanks [...] for continuing to invest heavily in the tooling and making using WPT easier and more enjoyable! Keep up the great work!
Rick Byers, Google
More work like this at Bocoup
Contact Us
We'd love to hear from you. Get in touch!
P.O. Box 961436
Boston, MA 02196