Writing more tests, earlier
Stage 3 is an important milestone, in particular, because it is the last stage before acceptance into the standard. To advance, features must be implemented in two or more engines and pass Test262 acceptance tests. In practice, however, coordinating and testing multiple implementations presents a difficult chicken-and-egg situation. Individual browser engines, for example, are often hesitant to take the first step in implementing a new feature without tacit understanding that other browser engines will follow suit. Meanwhile, the implementation process itself often uncovers issues which will necessarily change the design of the feature, making successful implementation a moving target.
This coordination complexity can and does impact the time it takes for features to reach Stage 4 acceptance (although there are plenty of other factors that impact standardization time as well). To help smooth the process, we worked to add Test262 tests for all Stage 3 features in 2018, regardless of whether or not vendors had begun to implement those features.
In 12,278 new tests, we authored and reviewed coverage for features including BigInt, class fields & methods (public and private), dynamic import, a variety of Internationalization APIs, and expanded coverage for SharedArrayBuffer and Atomics used with BigInt TypedArrays. These Stage 3 Test262 tests gave implementers a head start by turning spec text into conformance test code, allowing runtimes (and in turn web browsers) to ship the features more quickly and with better interoperability.
This nearly unprecedented move provided a unique opportunity to revisit the design of the Atomics API and rectify a naming usability issue. At the May 2018 TC39 meeting, attendees agreed that Atomics.wake should be renamed to Atomics.notify to prevent confusion with Atomics.wait. Although such a change would normally be out of the question given the risk of breaking existing websites, the forced removal of SharedArrayBuffers in the Spectre aftermath gave the committee a narrow window to rectify the API name.
Bocoup led the charge to move all four browser engines to the new API name before the feature was shipped again by implementing the name change in Test262. In this case, the test suite provided the extra impetus to coordinate a speedy patch in all existing implementations. Failing tests can be an efficient way to convey a high priority fix to multiple vendors and ensure coordination within a narrow window of time.
Test262 played another role in ensuring that SharedArrayBuffers evolved functionality efficiently. When the BigInt proposal reached Stage 3, TC39 agreed that the Atomics and SharedArrayBuffers APIs would be extended to cover the semantics of sharing memory with BigInt64Arrays. Normally, adding this kind of functionality to an existing API would require vendors to update or write new unit tests to provide coverage for the new feature. However, because we updated Test262 to support preemptible BigInt64Arrays at the beginning of Stage 3, by the time vendors began to update their implementations they were able to do so without writing local test material. The Test262 tests could be imported and used directly to ensure that the semantics of a relatively complex API were correctly updated.
Improving Overall Interoperability
The future of Test262 in the TC39 process