Why it matters: Apple currently maintains Speedometer 2.1, a tool for benchmarking web browsers across various workloads. The software’s next major iteration will receive input from three browser vendors to offset user concerns about possible bias. The new project is in its early stages but the collaboration marks a turning point.
Apple, Google, and Mozilla have announced they are jointly developing the next version of the Speedometer web browser benchmark to better represent modern browser tasks. The new version is still unstable, but users can start testing it now.
Speedometer 3 is designed to test the full range of how normal users operate browsers by running on normal web pages instead of benchmarking in a tight environment. The developers plan to continually tweak the software based on up-to-date data on what features are most important to everyday users. Google confirmed the benchmark will represent JavaScript frameworks and other modern workloads.
It will be a while before Speedometer 3 is fully up and running, but more information about it will emerge in the coming months. If the collaboration works out, it should give web designers a better picture of how their work will run across multiple browsers.
We’re glad to be collaborating with @WebKit and @GoogleChrome on Speedometer 3. 🌪️
Here’s why this work is important for the Web 🧵
— Mozilla Developer ‘🏾’ (@mozhacks) December 15, 2022
Currently, most users who want to see how fast their browser responds to web applications should probably use Speedometer 2.1. However, because Apple operates 2.1, it’s easy to assume its results could be biased against Safari’s competitors like Google Chrome, Mozilla Firefox, or Microsoft Edge. A benchmark based on WebKit might not fully understand the performance challenges of Blink, V8, or SpiderMonkey, for example.
Although Apple’s WebKit GitHub page hosts Speedometer 3, Google and Mozilla will develop it alongside the Cupertino giant. This should ensure it treats all three browsers fairly and that it accurately understands how users engage with them. Mozilla’s announcement noted that a good browser benchmark requires cooperation between browser vendors, website builders, framework builders, and web standards groups.
A governance policy based on consent between the three browser giants will guide Speedometer 3’s development. Trivial changes require just one of the companies’ approval, non-trivial changes need two to agree on them, and significant changes necessitate consensus between all three.