How does Calibre differ from Google PageSpeed Insights?


When testing performance using several tools, we often encounter vastly different results. The discrepancy between measurements is not only confusing, but also doesn’t inspire confidence. It becomes difficult to establish which tool to trust.

This issue manifests itself when comparing Calibre to other services, in this case, PageSpeed Insights. There are significant differences that make comparison between the two tools an impossibility. In this guide, we explain those disparities, so you can better understand your performance results.

Test conditions differ

It’s important to note that each monitoring service operates under notably contrasting conditions:

Geographical location

The location of where tests originate from is a significant factor in performance testing. The latency for every request will differ from location to location. Your site may be closer, or further based on where the test is ran from.

CalibrePageSpeed Insights
Tests originate from a pre-selected test location. Run on Amazon Web Services (AWS).Tests originate from 1 of 4 global locations, based on round-robin selection and latency to your browser. Run on Google.

PageSpeed test locations

These test locations have not been officially published by Google, which makes it more difficult to understand the internals of PageSpeed Insights.

PageSpeed is hosted in 4 locations:

  • Oregon, USA
  • South Carolina, USA
  • Netherlands
  • Taiwan

Network and emulation speed

Network conditions change based on locale, but another important factor is the applied network speed of the test.

CalibrePageSpeed Insights
Test Profiles allow you to choose from a list of pre-configured network throttling settings. They’re chosen based on their relevance to observed global averages.PageSpeed tests under two different network speeds: Desktop (described as “Dense 4G 25th percentile”) and mobile (described as “Mobile Slow 4G”).

CPU and GPU performance of the test agent

The underlaying hardware running performance tests has an effect on the accuracy of simulated hardware devices. Both Calibre and PageSpeed use different types of machines.

Simulation

In order to return results fast, PageSpeed runs at full network speed, then applies a simulation to calculate performance metrics.

Calibre applies bandwidth and device throttling as it conducts the testing—metrics are recored in realtime.

Should we be comparing metrics obtained from different tools?

No. Based on the factors described above, it’s impossible to make any form of informed comparison. When observing performance data, it’s important to establish a baseline and compare it in relativity to historic results. Only when recording in a stable environment, under the same conditions, we’ll be able to draw confident conclusions about our site’s performance.


References

Related Articles