Lighthouse is a nice tool, but it’s heavily oriented towards large, complex JavaScript apps and other Google-centric problems. One example of this is its use of the “SpeedIndex” metric. Maciej Cegłowski described it nicely:
If you’ve ever struggled to lose weight, you know there are tricks people use to fool themselves into thinking they’re thinner. You suck in your gut, wear a tight shirt, stand on a certain part of the scale.
The same situation obtains with performance testing. People have invented creative metrics to persuade themselves that their molasses-like websites load fast.
Google has a popular one called SpeedIndex. (You know it’s from Google because they casually throw an integral sign into the definition.)
SpeedIndex is based on the idea that what counts is how fast the visible part of the website renders. It doesn’t matter what’s happening elsewhere on the page. It doesn’t matter if the network is saturated and your phone is hot to the touch. It doesn’t matter if the battery is visibly draining. Everything is OK as long as the part of the site in the viewport appears to pop into view right away.
Of course, it doesn’t matter how fast the site appears to load if the first thing the completed page does is serve an interstitial ad. Or, if like many mobile users, you start scrolling immediately and catch the ‘unoptimized’ part of the page with its pants down.
There is only one honest measure of web performance: the time from when you click a link to when you’ve finished skipping the last ad.
Everything else is bullshit.
Tons of tools for measuring web app performance have come and gone, with Lighthouse being the current favorits. As always, overengineered tools aren’t the best solution. The best way I’ve found to measure web app performance:
Acquire a slightly-worse-than-normal device. “Normal” means “something a person making 1/5 of a typical swdev salary could afford, three years before you bought your current device”.
Calibrate yourself: use a similar lightweight app written in a fast and native toolkit (I’d pick GTK) for at least 15 minutes. There’s a decent chance one exists since hardly any apps are particularly novel.
Start a video recording of your face.
Switch to the web app and use it for a few minutes.
Go back and play the recording.
Count the number of wrinkles on your forehead, or the number of times you wince. Swearing gives 5x points. That’s your slowness metric.
Lighthouse is a nice tool, but it’s heavily oriented towards large, complex JavaScript apps and other Google-centric problems. One example of this is its use of the “SpeedIndex” metric. Maciej Cegłowski described it nicely:
Tons of tools for measuring web app performance have come and gone, with Lighthouse being the current favorits. As always, overengineered tools aren’t the best solution. The best way I’ve found to measure web app performance: