There are three test modes built into Calibre that ensure each Snapshot returns key performance metrics, even in cases when Lighthouse raises an error.
Generally these errors occur when a page has a set of characteristics, some of which are:
By default, all Calibre tests run in Standard Mode, however in the event that a test fails due to a Lighthouse Timeout Error, Calibre will automatically re-run the test in Compatibility Mode.
This is the default mode for all Calibre tests. All audits, metrics and artifacts are recorded.
Compatibility Mode omits known problematic Lighthouse audits in an attempt to ensure the test completes successfully and performance metrics are recorded.
There are two stages of compatibility mode: Reduced Audits and Minimum Audits.
| Test audits or artifacts | Present in Reduced Audits? | Present in Minimum Audits? |
|---|---|---|
| Performance Audits | Yes | Yes |
| Accessibility Audits | No | No |
| Best Practices Audits | No | No |
| SEO Audits | No | No |
| Third Party Audits | Yes | No |
| Main Thread Execution Timeline | Yes | No |
Reduced Audits is the first compatibility mode. This mode will omit all non-performance related metrics, which means your test result return all paint based, runtime, byte size and request based metrics but will not return Accessibility, SEO or Best Practices scores and audits.
Minimum Audits is an alternate compatibility mode. In addition to omitting all non-performance-related metrics like in Reduced Audits, this mode will not generate a Main Thread Execution Timeline or the Third Party Tracking scores and audits.