Beyond the Bubble: Real-World Web Performance

My work is dedicated to helping people build better, faster, more accessible apps and websites. Running Calibre has given me a lot of insight into the challenges that teams face while building and maintaining their little slices of the web. This post is a transcript of a talk I’ve given and touches on real-world web performance and connectivity looks.

Something that has struck me this past year is how little we as a web industry know about how people access the internet and, tangentially, how antiquated our methods of delivering content to users are.

Sure, we’ve had some significant advances in the last couple of years that dramatically improve how quickly we can push content down to devices. Still, as an industry, not much has changed from the core premise of “load the HTML, find the other resources, then load them too”.

Today, global internet access is around 46.1%, which is only half of the population on this beautiful blue marble that we call home will have a rough idea of what ‘pull to refresh’ means. The rest? Well, they’re not connected, so they probably don’t.

If you investigate the growth of the web over the last three years, it won’t take very long to find that internet access in India has been growing at a rate that we’ve never seen before. In 2016 alone, India introduced 106 million people to the internet for the first time. To add some perspective, that’s around 290,000 people every day.

That’s a growth of about 30% (from 2015); if we matched those numbers again in 2017, that’s another 140 million people. Indeed, only 35% of Indians are online today, and the population is 1.2 billion. We’re set for tremendous, unprecedented growth in the next few years.

Mobile usage surpassed desktop usage sometime during 2014—51.3% of devices with an Internet connection are hand-held. Hand-held devices are generally far cheaper (and often just as capable for everyday tasks as their more expensive and less portable counterparts, desktop computers).

Data and connectivity

Last year I researched how much mobile data cost in a number of locations around the globe. I took the local minimum wage, found the carrier with the largest market share, and found the best value-for-money prepaid plan with at least 500MB of data.

It goes a bit further than that, too, because even if you can afford a device with a decent monthly data allowance, do not assume it will be fast.

Regardless of the average LTE speed, I have more, perhaps shocking news: 60% of the world’s average mobile connections are 2G.

That isn’t just “2G speeds”. That’s a 2G connection. You might’ve seen the network advertised as ‘Edge’ on your iPhone, and everything… well, it stopped working, right? You may be thinking, “Well, almost everyone I know has at least a DSL connection… that should be better than the speeds I’m reading here”, and unfortunately, that isn’t quite the case. The global average Internet connection speed? 7 Mbps.

Enabling connectivity in India

Late last year, a new 4G-only wireless network provider appeared after a year of private beta made up of their friends, family and employees.

In a few short months, Jio has, in one fell swoop, changed the face of connectivity in India and what I believe to be the world’s fastest customer growth curve that would bring Silicon Valley to its knees. Since its arrival on the market, Jio has reduced overall tariffs by up to 80%.

The growth curve to end all growth curves

The company commercially launched its services on 5 September 2016.[4] Within the first month of commercial operations, Jio announced that it had acquired 16 million subscribers. This is the fastest ramp-up by any mobile network operator anywhere in the world.[17] Jio crossed 50 million subscriber mark in 83 days since its launch.[18] Jio crossed 100 million subscribers on 22 February 2017.

Wikipedia

Not everyone has an iPhone

Right now, you might be sitting at a nice desk, using Wi-Fi on a computer that costs somewhere in the region of $1600–3000, or maybe a reasonably current specification Android or iPhone. Given you’re most likely a part of the web development community (hey, that’s my audience), that isn’t surprising.

Aside from your parents, who have the same broken computer that was there when you left home, other people have current model machines. Over the last few years, I’ve seen and heard a lot about Android fragmentation, but I’m not confident I ever understood it until I researched global smartphone sales. Turns out I’d never heard of ⅔ of the phones being sold.

What I find striking about these statistics is the tremendous number of devices and the fact that I’d never heard of a handful of these vendors—who sell more than 200M devices in just three months.

Such a wide range of devices in the market leaves plenty of web developers with a sickly feeling in their stomachs. How on earth are we meant to test all these and understand their quirks intimately?

A strategy that I’m fond of is:

• Test using real hardware (and automate your testing)
• Test in all major browsers for a given market
• Test using realistic average network conditions for a given market

There’s some subtlety in testing on the suitable devices, but thankfully Alex Russell was kind enough to share his 2016 research, in which he did some detective work to find a single mobile device that best-represented devices in the wild could be purchased today and would make for a trustworthy test phone in the years ahead.

The criteria for such a device are as follows:

1. This device needed to be globally available, reasonably affordable, have minimal bloatware, and thus, a low barrier to entry.
2. The device would need to be slightly ahead of the 2016 average, making it a strong candidate to purchase as a test device. (an investment).
3. The Nexus 5X wasn’t an option due to thermal instability. Sometimes they’re fast and occasionally slow (A57 vs A53 Chipsets).
4. There are billions of Android devices in the wild that are using the Qualcomm 28nm A53 Chipset. These Quad-core CPUs are everywhere and also come with hardly any cache. It’s best to choose a device on that chipset.

Most people receive phones via their carrier, and the devices vary greatly over demographics and geography.

After conducting this research, Alex arrived at the Motorola G4 — which is statistically quite above average. There are a few caveats:

• The actual median is a slower, older, cheaper phone.
• The Moto G4 is a great candidate for a device that will be the median in late 2017 or early 2018.
• There’s quite a lot of RAM (2GB)—memory just hasn’t gotten cheap. The cheap phones are still shipping with 512mb-1GB of RAM.

If you develop and test with a Motorola G4 starting today, and it performs ‘kinda well’, it’ll be excellent for everyone who uses newer, more powerful devices.

Device specifications are huge factors for a smooth, delightful user experience, mainly if you’re relying on JS for interactivity and rendering. The more script, the more your app depends on local single-core performance. Lower-powered devices will feel sluggish, whereas newer devices can run several pages simultaneously without breaking a sweat.

Automation

Testing on a real device can become pretty limiting too, thankfully both Web Page Test and Calibre have got you covered.

Real devices are essential to developing empathy and understanding, but making testing trivial is crucial for teams. Just like a continuous integration process, it should be automated and run regularly to create a complete timeline of changes in performance.

In Addy Osmani’s excellent article “JavaScript startup performance”, he studied the time it took to parse (that’s parse, not execute) 1MB of JavaScript on a wide array of devices.

Today, 620Kb of JavaScript is the global median, which sounds like a lot, but remember, this is GZipped. Forget 620; it’s more like 2–3MB decompressed. We’re talking total lock up on a slower device—almost unresponsive for several seconds.

That’s why testing with real devices is crucial because your $1,000 iPhone performs almost as well as a reasonably new Macbook Pro. That isn’t reality.

It’s your job to build a deep understanding of what users are experiencing

I hope that the information I’ve given you has given you something that you can point at, stamp your feet and say, “We need to do better” because that’s the truth. We do.

The nature of the web is change. There are no constants. Experiences vary greatly by context.

For the web development community, there are two versions of the web, the one we use daily and the one we don’t. We know that connections vary by where you are: maybe you’re at the wrong end of the building, at a conference, on a plane, in a tunnel or for whatever reason, the modem needs restarting again.

We must stop optimising for $3,000 computers with fast connections. If you take a peek outside your bubble, you’ll find that we’re just not trying hard enough to build beautiful, fast, efficient and delightful user experiences for our beloved ‘users’.

I’m sure no one signed up to build the wealthy western web. We all create for the World Wide Web, and it’s time we started acting this way as a community. Make the web fast for everyone.

Huge, bellowing applause for my friends and heroes who helped me prepare this information: Karolina Szczur, Addy Osmani, Alex Russell and Bruce Lawson.

Start a free 15-day Calibre trial today.

We have so much more that we’re excited to share with you as we continue our mission to make the Web fast for everyone.

Ben Schwarz

Ben is the Founder and CEO of Calibre. He uses his experience in far-reaching Open Source projects and web standards to build tools for a better, more accessible web. Find him on Mastodon or LinkedIn.