AMP

Measuring AMP Performance

Uncategorized

Editor’s note: The following was posted on Medium by Martin Schierle, Mobile Solutions Consultant, Google.

TL;DR: Whenever performance testing AMP, keep in mind that a test from origin will include potentially suboptimal server settings like bad cache headers or missing image optimizations. Also the biggest speed gain (near instant load through prerendering) will not be reflected in commonly used performance tools and metrics.

Developers implementing AMP are looking to get blazingly fast loading speeds, and are therefore often curious to see how much their site improved with AMP. However, a naive run of one of the many available performance tools (e.g. PageSpeed Insights or Lighthouse) does sometimes yield surprising and seemingly suboptimal results.

To understand why this is happening, it is important to be aware that AMP speeds up a website on three different levels:

  1. AMP itself is already very fast, as custom JS is forbidden, critical path is unblocked, CSS is inlined and many other optimizations. However, there may still be bottlenecks from the server-side, e.g. unoptimized images or insufficient cache headers, which can’t easily be fixed by the client side AMP library.
  2. The second level of speed-up is then achieved through the caching by the AMP Caches (e.g. the Google AMP Cache), which will reoptimize images, add prefetch hints, minify html, serve via HTTP/2, along with many other optimizations. Keep in mind that the bulk of those optimizations can also be done on origin.
  3. The third (and potentially most impactful) level of speed improvement is based on the fact that AMP can be prerendered in a safe and secure way, by prerendering only assets in the first viewport, and not executing third-party scripts. This is described in far more detail here.

So if a performance check is done on AMP on origin, the speed scores (while normally much faster than the canonical) are not yet representative. A better way to test is to run the performance test on the same site served from one of the AMP Caches (you can use this tool to get a cache URL for the Google AMP cache). This will include optimizations through the cache into the measurement. This score will already be much better, and basically shows what you could achieve on your own host as well by applying optimizations like the ones described previously.

Let’s see what this might look like for an actual page, here one of our example pages from ampbyexample.com:

measure1

Detailed results can be found here. The chart nicely shows how the performance improves if measured from an AMP Cache instead of origin across most metrics. First meaningful paint is, for example, 0.9s faster from cache.

Unfortunately the third and most important case (near instant load through prerendering) can’t be measured easily with the regular tools, as it would need to be measured in the flow coming from an earlier visited site or app. It should however be obvious that a page can be shown near instantly, as long as all visible content was already prerendered beforehand. Webpagetest.org allows to test flows like this via the scripting option, but it is cumbersome and error-prone to setup and maintain. A script could look like this:

// don’t log data for first navigation step

logData 0 

// navigate to the first page (e.g. Google SRP) which prerenders AMP

navigate INSERT_URL_CALLED_BEFORE_AMP

// sleep a bit to give prerendering time,
// a user normally also doesn’t click through immediately

sleep 10

// start logging now for clickthrough to AMP

logData 1

// click through, insert correct query expression
// to find the right link to click for your doc

execAndWait document.querySelector(‘[…]’).click();

When adding this in, the complete comparison of all three modes looks like this (with detailed results being here):

This shows the advantage of prerendering, which makes it possible to start render pretty much instantaneously (66ms) and to be visually complete and interactive after approximately 1s.

So, whenever performance testing AMP, keep in mind that not all speed advantages are directly obvious, as some will come in through caching and especially prerendering.

Posted on Medium by Martin Schierle, Mobile Solutions Consultant, Google