Ben Schwarz
December 11, 2018
We have been working closely with the Lighthouse team to ensure that Calibre provides the best experience of the most popular performance audit suite.
As of right now, all tests are being conducted using the latest version of Lighthouse.
Here’s a summary of changes:
For some of you, it might mean that some Lighthouse grade score metrics are marginally different than before—that’s because the score algorithms have been tweaked to more accurately portray user experience.
Prior to this update, JS Parse & Compile was calculated using an older version of Chrome Devtools that was built into Lighthouse 2.9.x. This older version of devtools did not calculate all aspects of JavaScript runtime performance, so the update to Lighthouse 4 will more than likely show significant increases to the JS runtime metrics. These increases more accurately describe JavaScript main thread performance.
More importantly, Lighthouse results you see in Calibre are far closer to release parity with Chrome Developer Tools add-on and the Chrome extension.
We know that test reliability is crucial to performance monitoring and making confident decisions. Test data is now more consistent thanks a number of developments focused on smoother instrumentation.
We’ve added a test verification process in which Calibre watches the results of each test and compares it to historic data.
If an anomaly occurs (for example, a significant increase for a page’s Speed Index on a mobile device), we will automatically re-test the given page a number of times to confirm the results. Test verification has already marked a reduction on metric variation and will prevent unexplained spikes or falls from happening.
You can now download the Lighthouse JSON, GIF and video renders of each test against your pages. Head to a Snapshots → choose a relevant test → click the dotted menu in the upper right corner and choose what data you’d like to get!
We have improved the display of request timings and headers so you find what you’re looking for more easily.
Last week while releasing the Lighthouse 4 upgrade, we suffered from a much longer than expected data migration that caused a few episodes of downtime. No data has been lost. If you experienced a less-than-smooth experience with Calibre during this time, we are truly sorry.
We've written a detailed account of what went wrong, how we will be handling significant releases in the future and providing transparency into the service status.
If you have any questions or concerns we’d be delighted to speak with you directly.
In the New Year, expect to see detailed email reports, clearer metrics and new tools that will help you make your pages faster.
References:
We will send you our latest web performance advice and notifications about new Calibre features.