Recently, I had the opportunity to contribute to a massive, meaningful effort: the open-source Web Platform Tests (WPT) project. My task was to improve WPT test coverage for areas of the HTML specification dealing with navigation —things like the details of loading new web pages, browsing around the web, and opening new windows.
I didn’t anticipate that I’d stumble onto a bug affecting several browsers that dates back nearly 20 years, in one of the most established areas of the HTML specification.
How Web Platform Tests make the web better
Standards organizations like the W3C and WHATWG produce a set of specifications that, taken together, define how the web platform as a whole functions. These specs include greatest hits like HTML and the collection of specifications that define CSS , but also dozens of other standards, some lesser-known (e.g. Subresource Integrity , Pointerlock ).
The web platform is complex, but these specifications give it definition. A definition isn’t quite enough, though, to make the open web platform stable and predictable. For that we also need tests. Tests that not only detect whether an individual browser implements a given feature correctly, but also validate—collectively, programmatically—whether the specification itself makes sense.
WPT brings together tests for dozens of web standards into a test suite that can be run in any browser. By serving as connective glue between the specifications and the way that browsers work in the real world, these tests build a shared consensus of how the web platform should operate. They help to find where browser makers fall short in spec compliance, but they also help to identify places where elements of the platform itself need better or different definitions. That is, by stabilizing and growing WPT, we’re expanding the definition of a “web standard” from specifications and implementations to include tests.
Taking a deep breath and diving into WPT
I’m a web developer. While I have familiarity with many web standards — a deep familiarity with certain specifications — and a good handle on testing, combining the two is relatively new to me. I admit: the first time I forked the WPT repository and started pawing through tests, I was intimidated. WPT can feel like sitting in the cockpit of an Airbus A380.
The language and structure of specs is formal and precise, and the facility and speed with which test contributors and spec authors get work done is awe-inspiring. As with any community, any project, domain-specific vocabulary and processes have to be felt out. Diving in was challenging but my web developer perspective turned out to be valuable to browser engineers and spec authors, and I was made to feel welcome by community leaders and reviewers.
Testing navigation features, obsessing over browsing contexts
The bulk of navigation-related features in the HTML specification—the features I needed to improve test coverage for—are contained in section 7, “Loading Web Pages” . I rolled up my proverbial sleeves and started at the top of section 7, looking for areas that could use more complete test coverage.
In the course of this work, I spent a lot of time thinking about how browsing contexts work. A browsing context “is an environment in which Document objects are presented to the user”. Each open window or tab in your browser represents the most evident examples of individual browsing contexts; some embedded elements (e.g. iframes) also contain their own browsing contexts.
If you’re a web dev, it’s tempting to equate a browsing context with a window object, but that’s not quite right—a given window object is not reused when you navigate a tab (or window) in your browser to a new document, but the containing browsing context is. For the purpose of this story, however, maintaining a rough mental equivalency between a tab (or window) and a browsing context is a reasonable approximation.
OK. Navigation features…browsing contexts…how do these high-level, abstract concepts come together in an applied form? For that, let’s take a look at the decidedly ho-hum window.open method, and its connection to the way that specific browsing contexts get identified by name or keyword.
The ins and outs of creating and navigating browsing contexts with JavaScript
The open method on the global window object (window.open) has been around for-evvvvv-er and is the reason that popup blockers exist.
In its stock form, window.open('http://www.example.com') will open example.com in a new tab or window (well, it will if you don’t have a popup blocker enabled—which you probably do. The remaining examples in this article assume popups are being allowed by the browser).
The second argument to window.open allows the script’s author to specify which browsing context the page should be opened into by name or keyword (a string in either case). If no browsing context is indicated, or if the one indicated doesn’t yet exist, the browser will create a new one.
window.open('http://www.example.com', 'hellothere');
window.open('http://www.example.org', 'hellothere');
The first call to window.open above will open the document at http://www.example.com into a browsing context named hellothere. If no browsing context with that name exists yet—it very likely would not—the browser will create a new browsing context, opening a new window or tab and naming the context hellothere.
The second call will reuse the hellothere browsing context and navigate it to example.org. The net effect after both invocations is that, at most, one tab or window is opened. http://www.example.org will replace http://www.example.com in the hellothere browsing context. That’s an example of using a browsing context name to reference a specific browsing context.
Browsing context keywords
There is also a set of browsing context keywords, each providing a specific behavior. For example, _self will open a document into the current browsing context.
_blank , another keyword, should always create a new browsing context, no matter how many times it’s used. The following code should always open two new windows or tabs (i.e. create two new browsing contexts):
window.open('http://www.example.com', '_blank');
window.open('http://www.example.org', '_blank');
This stuff isn’t exactly cutting edge—the amount of time _blank has been supported in browsers can be measured in decades.
Connecting specs to tests
Having studied the spec and gained insight into some of the details of browsing contexts, names, and keywords, it was time to figure out what, exactly, in this section still lacked tests—and get to work writing those tests. My enduring tactic with WPT is being very thorough. This is work in which focused pedantry isn’t just allowed, it’s often necessary. I read and re-read spec passages slowly and carefully, often writing them out longhand and marking them up with my own system of colors and symbols.
Determining whether test coverage is lacking for a section of a spec is in itself a complex cognitive task, a large part of the mental puzzle involved in this kind of work.
One challenge is that many things set out in specifications are not directly testable via JavaScript in the browser. Several of the steps defining the creation of a new browsing context cannot be observed directly, for example:
Set up a browsing context environment settings object with realm execution context, and let settingsObject be the result.
We can’t see the browser actually taking those steps—those are internal browser objects not exposed to JavaScript. On the other hand, steps like:
Ensure that document has a single child html node, which itself has two empty child nodes: a head element, and a body element.
are things we can check for by examining the active document in the newly-created browsing context.
Contributing to WPT is a constant cognitive workout: specs are not simply a list of steps that translate seamlessly into test assertions. Instead, it’s a complex puzzle. How can we make sure the browser is doing what it’s supposed to if we can’t always tell what it’s doing directly? What happens when different scenarios are combined; what kinds of situations arise from the interplay of different situations and environments? How do web authors use web platform features in real life, and what happens when they make mistakes in the use of a feature or use it in a different manner than spec authors originally foresaw?
Zeroing in: case-sensitivity in browsing context keywords
As an example of this process of analysis, let’s return to the browsing context keywords ( _blank, _self , e.g.). Those are defined in the Browsing Context Names section of the HTML specification. I was combing this section line by line, looking for gaps in test coverage. I paused at this line:
A valid browsing context name or keyword is any string that is either a valid browsing context name or that is an ASCII case-insensitive match for one of: _blank, _self, _parent, or _top. (emphasis mine)
After reading that, what would you infer the spec-compliant behavior to be for the following?
window.open('http://www.example.com', '_bLAnk');
window.open('http://www.example.org', '_bLAnk');
It’s a bug!
No tests yet existed for checking case-sensitivity, so I got to work.
Given that keywords should be case-insensitive, the code above should result in two new browsing contexts (i.e. two new tabs or windows will be opened, one containing example.com and one containing example.org).
That was indeed the case in Firefox. But when I ran the tests I wrote for this in Chrome and Safari, only one window (tab) was being created. Those browsers were not performing a case-insensitive match. Instead of creating a new, unnamed browsing context in the first invocation, the browsers created a new browsing context named _bLAnk, which then got reused by the second window.open — _bLAnk is treated as a browsing context name, not a browsing context keyword.
And that’s how I found an ancient bug pertaining to a mundane part of the HTML spec, affecting multiple browsers.
Whose fault is it? What’s actually broken?
I opened bugs on Chromium (702178) and Webkit (169747) for the _blank case-sensitivity issue. Later tests I wrote demonstrated that the case-sensitivity issue also affected the other keywords (_self, _parent, _top) in these browsers (the Edge browser is also affected; a corresponding bug (11403319) was opened).
What may seem cut-and-dried—these browsers are not spec-compliant for case-sensitivity on browsing context keywords, ipso facto the browsers are broken and should be fixed—is in fact more nuanced.
This behavior, erroneous when analyzed against the current spec language, nonetheless has been the way those affected browsers work for years and years. Despite the exhortation in the spec itself that web devs like us should “read [the HTML specification] cover-to-cover, multiple times…[and] it should be read backwards at least once”—no mean feat considering that it clocks in at over a half million words (barely edged out by War and Peace)—most sane devs design applications to work in real-life browsers, not for aligning with specifications.
So there are many accumulated years during which web developers may have made scripts that opened windows in _BLANK or _Blank and relied on what actually happens, not what should. Would making browsing context keywords suddenly case-sensitive cause widespread breakage?
That is, should these keywords be case-insensitive? Or should the spec itself change? And thus an issue was opened on the HTML specification to examine just that. After a discussion involving the analysis of case-mixed keywords , it was decided that case-insensitivity—that is, the way the spec currently reads—is indeed the most web-compatible choice and the spec was not altered.
Onward marches the web
With such alacrity things progress! Already, browser code patches have been made, resolving both the Webkit and Chromium bugs.
And the cycle of analysis, testing and assessment marches on, as the WPT contributors make the web platform more predictable and stable, day by day.
