Our Test Agents are now using Chrome 119 and Lighthouse 11.3.0 to test your Sites. You can find details surrounding the 11.3.0 release in the official Lighthouse changelog.
Lighthouse 11.2.0 removed the “PWA” (Progressive Web App) category for its tests. Inline with this change, Calibre no longer records PWA score metrics. You can access historic PWA scores and audits by downloading pre-11.3.0 reports via the Calibre API, or by downloading lighthouse.json from the Calibre UI.
Single performance tests are a great way to find the reason for observed metric changes that might be caused by requests or asset change. Because we see Snapshots as a critical discovery tool, today we’ve released numerous improvements to Snapshot Overview that make investigating the performance of a page much easier.
With the new Snapshot Overview, you can:
View the new Snapshot Overview by choosing a test in Site → Snapshots. You can also learn more in the release blog post.
We released a new patch version of our React Live Chat Loader package. In this release, we bumped outdated dependencies and addressed a bug that had caused the distribution bundle to drop support for ES5.
Over the last few months, we’ve made significant additions to Pull Request Reviews (you can go through all of the changes here).
Today, we have overhauled the performance report posted by Calibre in Pull Requests. Now, the report clearly features:
With this information in place, your team can quickly assess the impact of proposed on work:
Find more information in our Pull Request Reviews documentation.
Calibre Test Profiles allow you to specify an emulated device and network throttling for your tests. Until today, all device and network throttling was applied during runtime, meaning that the throttling was applied to the browser as it was running the test.
Google’s PageSpeed Insights uses a different approach. It runs page tests at full, unthrottled speed, then performs a simulation of the throttling that would have been applied to the page load.
Today, we have added a new option to Calibre Test Profiles that allows you to run your tests with the same simulated throttling as PageSpeed Insights. Now, there are two new Test Profile options:
These two device options will run your tests with the same throttling settings as PageSpeed Insights. While we don't recommend comparing monitoring from different tools, this option will allow you to compare your Calibre tests with PageSpeed Insights tests more closely as the test conditions will be more similar.
To start testing with PageSpeed simulated throttling, create or update an existing Test Profile in Site → Settings → Test Profiles.
Our Test Agents are now using Chrome 117 and Lighthouse 11.1.0 to test your Sites. You can find details surrounding the 11.1.0 release in the official Lighthouse changelog.
The 5.1.0 release of Calibre CLI introduces new commands to help you automate your performance testing with more ease than previously possible.
Designed for use with CI/CD systems, CLI Pull Request Reviews are a way for Calibre to compare production (or another environment) to a preview deployment (e.g.: your work-in-progress branch). Each Pull Request review generates a comparison report outlining any key changes in site speed, which can be used to make decisions about whether to merge the branch or not.
You can automatically fail builds when performance budgets are no longer met using the --failOnUnmetBudget flag, too!
1calibre site create-pull-request-review \2 --site "my-shopfront-production" \3 --title "(feat): add lightbox to PDP" \4 --url "https://pr-123.example.com" \5 --branch "feat/add-lightbox-to-pdp" \6 --sha="1234567890abcdef1234567890abcdef12345678" \7 --waitForResult \8 --failOnUnmetBudget \9 --markdown > comparison_report.md10
Read the Pull Request Review CLI documentation for more information.
We’ve overhauled the Single Page Test summary terminal output to include a test summary including Web Vitals assessment and key request sizes:
Read the Single Page Test CLI documentation for more information.
Single Page Tests now generate a markdown-powered performance report. You can use this output with other tools that support markdown rendering like GitHub, GitLab and Bitbucket to ensure that all the code you release is reviewed for performance and best practices.
Read the updated documentation for calibre test create and calibre test show commands.
When creating a Single Page Test, you can now specify a Webhook URL to post test results to. Using this new technique, you can run async tests and have the results delivered to a custom endpoint of your choosing. You can use --webhookSecret to verify the incoming webhook using HMAC.
1npx calibre test create https://calibreapp.com \2 --location Sydney \3 --webhookUrl="https://webhooks.example.com" \4 --webhookSecret="my verification secret"5
Read the updated documentation for calibre test create command.
As part of our regular release cycle, we have applied dependency updates that include important security patches to those projects.
If you are using Pull Request Reviews to test the performance of your changes in GitHub, you can now be even more proactive with enforcing budget limits.
We added an option to fail Calibre’s budget checks if the pull request exceeds the budget that’s currently passing on the production site. Turn pull request fail checks in Site → Settings → Integrations → GitHub.
Having automations in place, monitoring a non-critical site or a preference for viewing data in-app rather than receiving emails are just a handful of reasons why you might like to create Performance Budgets without email notifications.
All new performance budgets by default have no notifications turned on. If you’d like to receive them, subscribe to relevant Sites’ alerts in your Email Notifications Settings.
Our Test Agents are now using Chrome 116 and Lighthouse 11 to test your Sites. You can find details surrounding the 11.0.0 release in the official Lighthouse changelog.
Chrome 116 introduced updates that affect how Largest Contentful Paint (LCP) is evaluated and calculated. You can read details of these changes on the official Largest Contentful Paint changelog.
There are dozens of ways to keep up-to-date with new releases from products your team is using. To make it easy, and accessible without leaving Calibre, we added a notification to the main navigation that signifies unread Changelog posts.
We have several major releases coming in the near future, so it’s a good time to be checking in on what’s new!
Our Test Agents are now using Lighthouse 10.3.0 to test your Sites.
You can find details surrounding the 10.3.0 release in the official Lighthouse 10 changelog.
From July 4, 2023, test artifacts (lighthouse.json, har.json and other test JSON reports that Calibre stores) will be compressed with the Brotli algorithm. This change brings 15-30% smaller downloads compared to previously used compression with Gzip, which also results in faster downloads.
Customers using the Node.js API or Command Line Interface to access test artifacts should update to version 5.0.4, which has been released as a result of this change. You only need to update if you are using the following functionality, or commands:
If you have a custom API integration, you will need to ensure that artifacts with a response header Content-Encoding with value br are decompressed using Brotli. Tests recorded before this change will remain compressed with Gzip.
From July 6, 2023, Calibre will require all webhook Integrations to use the HTTPS scheme. Existing Webhook Integrations can continue using plain HTTP until that date, but after July 6, we’ll automatically turn off webhook integrations which have not been updated to use HTTPS.
You can review your webhook integration settings by visiting Site → Settings → Integrations.
We published a patch CLI release to download Test Artifacts stored with Brotli compression.
Our Test Agents are now using Lighthouse 10.2.0 to test your Sites.
The upgrade from Lighthouse 9.x to 10.x introduces several changes to the way Lighthouse calculates performance metrics. You can find details on the official Lighthouse 10 changelog.
As a consequence of first two points, your Performance Score might change.
Our Test Agents are now using Chrome 113 to test your Sites.
Chrome 111 and 112 introduced updates that effect how First Contentful Paint (FCP), and Largest Contentful Paint (LCP) are evaluated and calculated. You can read details of these changes on the official Largest Contentful Paint changelog.
Managing service subscriptions can be unnecessarily difficult. Every so often, you might want to pause payments while your team changes focus, re-evaluates tools or needs a breather. That’s why we introduced the ability to temporarily pause your Calibre subscription.
Organisation administrators can pause subscriptions on the Billing → Overview page.
We resolved an issue where the team member count was not appearing in the Team membership section visible on the People → Edit page.
We have added the ability to sort tables within the Lighthouse audits (Performance, Accessibility, SEO, Best Practices and PWA):
Sortable tables allow you to focus on the metrics most relevant to your requirements and quickly identify areas that need improvement, along with those performing well.
Head to a Snapshot and browse the Lighthouse audits to see the new sortable tables in action.
We have made several improvements to reliability of Pull Request Reviews in GitHub:
We refreshed the Third Party page, so you can quickly assess the impact of external tools and scripts on performance. The report visualises how first and third-party resources contribute to page transfer size and main thread execution time. You can also quickly pinpoint which category of tools is the most costly, whether it’s bandwidth or time.
Soon, we’ll be adding more information about third-party requests, including details about file types. You can view the Third Party report by navigating to a Snapshot and clicking the Third Party tab.
It’s now easier to prioritise which improvement possibilities to focus on based on Lighthouse audits. You can view the Opportunities and Diagnostics for all Lighthouse audit types (Performance, Accessibility, SEO, Best Practices and Progressive Web Apps) for each test.
For Performance audits, you can filter by Web Vitals metrics impacted by the audit and check the potential time and byte size savings you could observe by addressing specific advice given by Lighthouse.
Check out the refreshed audits by heading any selected Snapshot!