AMP Conf 2019. April 17/18. Tokyo.
AMP

Contributing to WebKit for a more predictable web platform

Uncategorized

Over the past two years, the AMP Project has been working with Igalia to identify bugs and missing features within iOS WebKit and then fix them. We create repro cases, write web platform tests, perform debugging and analysis, and, of course, write patches to actually fix things. We think this is particular rewarding work, because it helps ourselves achieve our goals faster, but also makes the web more predictable for developers overall.

In this blog post, we provide an overview of the work done in 2018 with hints about when improvements will be available in iOS releases or when they will have to be handled by Apple. Some of this work is still in progress and we keep proposing new ideas and reporting bugs.

iOS 12.1

We submitted patches for the following bugs which are now fixed in the latest iOS 12.1 releases:

Additionally, Igalia assisted Apple with improvements to custom elements. This one is fixed in the latest iOS 12.1.1 beta:

iOS 11

We have been watching some improvements and verified that they are included in the latest iOS 11 releases. These two items were handled by Igalia in 2017 thanks to support from the AMP project:

  • Improving frame sandboxing, including implementations of new sandbox attribute values such as allow-popup-to-escape-sandbox, allow-top-navigation-without-user-activation and allow-modals flags.
    This allows apps to more securely sandbox ads from doing bad things.
  • Fixing flickering for fixed positioned elements in iframes when touch scrolling is on. See bug 175135

These bugs triaged by us were fixed by other WebKit developers:

WebKit trunk

We have collaborated with Igalia’s and Google’s Web Platform engineers to make the following features available in WebKit trunk. Apple does not comment on future releases but we keep watching them to verify when these improvements arrive in iOS releases.

To be handled by Apple

Apple has been working on two big features that are interesting for AMP:

After analysis and discussions we concluded that it will be up to Apple to fix the following issues:

Work in progress

We are working with Igalia on the following improvements and some patches are already available:

Conclusion

We are thrilled to continue collaboration with Igalia and other browser developers in order to keep improving WebKit’s web platform implementations. Next year, we plan to continue the effort on the current tasks but we also have various other ideas related to networking, UI and more. Stay tuned!

Posted by Frédéric Wang, browser engineer at Igalia

Use AMPHTML ads for better ad performance, page usability and user safety

Uncategorized

This is part of a larger AMP monetization series on Medium that includes the below topics:

  1. Ensure Ad Density is equal on AMP & non-AMP pages
  2. Optimize your AMP pages for high ad viewability or views
  3. Take advantage of more ad competition with multi-size ads & fluid
  4. Better than header bidding → AMP RTC
  5. Take advantage of video ads in AMP
  6. Leverage rich media ad support in AMP
  7. Use AMPHTML ads for better ad performance, page usability & user safety

Originally posted on Medium by Vamsee Jasti, Product Manager for AMPHTML ads at Google.

Up until now, this monetization series was about ensuring you made the most revenue from existing JavaScript(JS) based ads served to AMP pages — which have a number of challenges independent of the web page. In this post, let’s talk about how we may approach improving the display ads ecosystem.

We’ve made a lot of progress in delivering a user-first advertising experience on AMP pages, but along the way we’ve learned that the principles of AMP pages can be transferred to display ads to make a step function improvement.

We set out to solve the issues of security & performance using AMPHTML ads (FKA A4A/ AMP ads) and have the benefits available not only to AMP pages, but also to any environment where display ads are served — regular web pages & mobile apps.

AMP’s tech lead, Malte, wrote about AMPHTML ads’ humble beginnings in this post (I’ll summarize most of it below but you should also consider reading it to know where we started and how we’ve evolved over time).

We’ve come a long way since then, and today I’d like to talk about:

  • What AMPHTML ads are
  • What AMPHTML ads aren’t
  • Why publishers should want AMPHTML ads
  • Why advertisers should want AMPHTML ads
  • Support status of AMPHTML across the display ads ecosystem

What AMPHTML ads are

AMPHTML ads is a framework that gives ad developers, advertisers, ad servers and publishers, the building blocks to create and deliver ads that are performant and secure.

Similar to AMP, an AMPHTML ad is only valid if it’s made of HTML + CSS + the JS from the open source AMP Project repo.

AMPHTML ads are a strict subset of the AMP page spec and ships with many good-by-default ads UI components, an analytics measurement framework, a spam detection system, viewability measurement and other goodies.

A simple AMPHTML ad

<!doctype html>
<html ⚡4ads>
<head>
<meta charset=”utf-8">
<title>My amphtml ad</title>
<meta name=”viewport” content=”width=device-width,minimum-scale=1,initial-scale=1">

<style amp4ads-boilerplate>body{visibility:hidden}</style>
</head>
<body>
<a target=”_blank” href=”https://www.ampproject.org">
<amp-img width=”300" height=”250"
src=”https://www.ampproject.org/static/img/docs/ads/amp-300x250.png"></amp-img>

</a>
</body>
</html>

Notice, no arbitrary JavaScript and all the code is declarative.

An AMPHTML ad on the left and a regular ad on the right

What AMPHTML ads are not (at least, not yet)

AMPHTML ads don’t ensure that the visual content of the ad is high quality.

Why publishers should want AMPHTML ads

When an ad is developed in AMP, the ad will have minimal performance impact to the web page and it gives back publishers control over the user experience of the page.

Ad creators vary a lot. Even a well-intentioned advertiser could create an ad that could lock up the CPU, janking the webpage and resulting in your visitors quitting your site. Then, there are a small minority of malicious advertisers set out to serve themselves ahead of your experience and your users. These issues can give knowledgable visitors another reason to install an ad blocker. One of the main reasons for this behavior lies in early design decision for legacy ads on the web allowed embedding ad iframes with arbitrary (a lot of times with terrible performance) JS.

Wouldn’t it be great if we could guarantee that the JS used within an ad is always performant and doesn’t deteriorate the page performance by too much? We expect AMP to meet that bar and since the code is all open source, anyone can review it and suggest changes to improve further. In addition, the following properties of AMP solve a number of issues of the display ads ecosystem:

Statically analyzable

All AMPHTML ads are statically analyzable, which means that ads can’t run:

  1. any code that isn’t part of the public AMP GitHub repo
  2. additional JS that isn’t part of the rendered ad code
El Pais saw a reduction of 90% in ad latency using AMPHTML ads driving the CTR up by 32%

Trustworthy

All of the code in the AMP repo is open source which is carefully reviewed. Therefore, it can’t have things like JS that takes advantage of a chipset level vulnerability which could steal passwords entered to your websites. If a malicious actor tried to add such code, the AMP maintainers have a process in place to ensure that such code isn’t merged into the repo.

Rapid & universal upgradeability

If we discover a vulnerability in the AMP code or at lower level stack that’s taking advantage or stealing user information from your site, the AMPHTML ads runtime can be updated and most users will have the latest version of the secure ads runtime within a day.

Self-aware of page performance

Animations within an ad can have a lot of impact to page performance. AMPHTML ads automatically pause animations when they are off-screen saving precious battery & CPU.

Caching a single library

Another performance benefit of using a single ads runtime standard is that different ad functionality can reference the runtime that is in the cache across multiple AMPHTML ads served to the same browser. As the volume of AMPHTML ads increases on the web, more users will have the ads runtime available in the browser cache. Contrast this to loading two different libraries for two separate functionalities, both of which are less likely to be available in the browser cache.

Efficient rendering in same-orgin iframes

For AMPHTML ads, the ads script can serve them into iframes on the same origin as the script tag which is more efficient compared to loading the ad into a cross domain iframe. This is only possible because AMPHTML can’t have custom JS and therefore reduce risk from a security perspective.

Time Inc saw an increase of 13% higher revenue with AMPHTML ads that were 3X lighter and 6X faster.

Why advertisers should want AMPHTML ads

Built-in ad UI components

<amp-video> component used with scroll bound AMP animations

AMPHTML ads ship with a number of UI components that allow ad creators to build great looking ads, with full access to the web animations API, a first-class analytics framework and viewability support. These built-in modules are developed with performance in mind, so advertisers can be assured they are using the best in class browser/app APIs to build this functionality.

Viewability measurement without discrepancies

Viewability is probably one of the most important metrics advertisers care about, and publishers, advertisers and ad tech, each want to verify viewability on their own resulting in each inserting their own JS to collect viewability data.

Problem is, each one of them uses their own technique to collect viewabilty. This is not only a terrible waste of CPU, it leads to viewability mismatch between the three collectors. AMPHTML ads rely on browser native APIs like Intersection Observer to collect the most accurate viewability measurements and send them to anyone that requests it. Since all of this code is open source, an advertiser can inspect the collection methodology themselves if they choose to.

Development-time validator

A lighter / more performant ad leads to more clicks and higher viewability. This should be pretty straightforward to reason, because one can’t click or view an ad if it doesn’t render on screen fast enough. This means that an advertiser should be incentivized to build a performant ad so they get more ROI on the same spend.

However, given the typical separation between advertisers, media agencies & creative agencies, the advertiser has little control over the performance of the ad created by the creative agency. AMPHTML ads ship with a development time validator which gives a boolean answer to developers as they are building an ad if it’s valid AMPHTML. If it is valid, the creator can be assured that the ad would be performant.

Create once, deliver everywhere

In the near future, an advertiser would be able to create a single AMPHTML ad and deliver across web pages, AMP pages & mobile apps. AMPHTML ads will also natively support the SafeFrame API and the MRAID API, so advertisers can take advantage of advanced host (web or app) level functionality in a uniform way.

Ecosystem Status

It takes a village to make transformational changes like these, but some industry thought leaders have already spent significant amount of time & resources in helping bring more AMPHTML ads to the web. You can note current status of each of these using this link.

What’s the end goal?

With AMP’s new governance model and industry participation, we think we can help advertisers and publishers use AMPHTML ads to deliver every single ad served on the web and mobile apps. If we accomplish this, we will be in a world where:

  1. Ads will be respectful to users’ devices and publishers’ web pages
  2. Ads won’t impact the performance of a publisher page, earning them better revenue
  3. Ads will earn advertisers higher ROI including better viewability
  4. Users will be more secure on the web and therefore find fewer reasons to install ad blockers

I hope you’ll join the AMP team in helping solve one of the most important and interesting challenges on the open web — advertising.

Thanks to Malte UblKatharina Familia Almonte, and Rudy Galfi.

Originally posted on Medium by Vamsee Jasti, Product Manager for AMPHTML ads at Google.

AMP Project’s new governance model now in effect

Uncategorized

In September I announced a proposal for a new governance model for AMP that more explicitly gives a voice to all constituents of the community.  Since that announcement we have worked with the community to improve the proposal through a wide variety of channels including comments on the proposal pull request on GitHub, issues in the ampproject/meta repository, and discussions at the AMP Contributor Summit.  We are happy to announce that AMP’s new governance model goes into effect today.

Two key features of AMP’s new governance model are the Technical Steering Committee (TSC) and the Advisory Committee (AC).  We have endeavored to ensure that these committees consist of people who bring a wide variety of perspectives, with representatives from different AMP constituencies.  The initial membership of these committees is:

Advisory Committee

  • Charles Vazac, Akamai
  • Dane Knecht, Cloudflare
  • Dave Merrell, The Washington Post
  • Elisa Budelli, Automattic
  • Guilherme Souza, Terra
  • Joe Alicata, Vox Media
  • Léonie Watson, The Paciello Group
  • Levi Durfee, Bulldog Creative Services
  • Nicole Sullivan, Google
  • Pablo Delgado, El País
  • Senthil Padmanabhan, eBay
  • Sumantro Das, 1-800-Flowers.com
  • Terence Eden
  • Tim Jones, The New York Times
  • Tobie Langel, CodeSpeaks
  • Yinhuang Lu, AliExpress 

Technical Steering Committee

  • Chris Papazian, Pinterest
  • David Strauss, Pantheon
  • Dima Voytenko, Google
  • Malte Ubl, Google
  • Paul Armstrong, Twitter
  • Rudy Galfi, Google
  • Saulo Santos, Microsoft

 

Another key feature of the new governance model is the Working Groups, where most of the day-to-day work in AMP is done.  A proposal for the initial set of Working Groups has been made. The Technical Steering Committee (TSC), who is responsible for creating Working Groups, had their founding meeting November 29, 2018 and discussed the proposal as well as made recommendations for additional groups. The initial list will be confirmed in the next TSC meeting in 2 weeks.

I am excited to see AMP entering this next phase, and I am looking forward to working with the entire AMP community towards achieving our vision of a strong, user-first open web forever.

Posted by Malte Ubl, Member of the AMP Project Technical Steering Committee

Progressively AMPlify EC-CUBE

Uncategorized

EC-CUBE, one of the largest open source ecommerce CMSes in Japan, has released a beta plugin to use AMP for their ecommerce stores. The plugin was built by EC-CUBE’s well known partner SUNDAY SYSTEMS, Inc. in collaboration with Mobile Solutions Consultants from Google, in just one month 😲. For websites who use EC-CUBE, you can try the plugin here.

A PHP based CMS with more than 1.8M downloads and 30K+ live merchants, EC-CUBE was seeking further opportunity to satisfy the UX of their end users. They just released their major update v4.0.0 in October 2018 which focused more on the backend performance and architecture. Now they have decided to put effort in optimizing the frontend as well, leveraging AMP and key PWA technologies. The newly released AMP plugin is still in an experimental phase but already has some remarkable features which provide faster and optimized UX/DX.

  • To reuse existing assets and work with their PHP developer community, the AMP plugin will convert current PHP Twig templates into valid AMP templates. No need to struggle with AMP specific syntax!
  • The plugin is fully integrated into the CMS admin interface. Developers will be able to turn on/off the plugin, customize layouts, build components and edit the converted AMP templates directly from a dedicated UI.
  • Developers will have the option of “paired AMP mode” and “AMP first mode”. In paired AMP mode the main site is unaffected, but the first landing page (such as from a search engine) is optimized using AMP. In AMP first mode, the whole site is changed to always return AMP for all pages on the site.
  • It also works as a PWA. Having Service Worker and Web App Manifest bundled in the plugin, developers can now add an EC-CUBE store to the homescreen of a mobile device and even think about offline use cases.

EC-CUBE has performed a trial with their default template (an ice cream shop🍦) and have seen a significant improvement in speed, as well as their canonical features being converted correctly to AMP.  For example, using `amp-sidebar` for the slide menu, fetching product options with `amp-list`, changing the price once the user selects the option with `amp-bind` etc.

Visual comparison of non-AMP page and a pre rendered paired AMP page in Fast 3G / Moto G4 simulation in WebPageTest

 

Fully responsive AMP first mode. Store top page, listing page and product detail page are all AMP. The GIF below shows smooth AMP to AMP navigation with interactive UI.

In their trial, they also experimented PWA features such as add to home screen and offline browsing. Merchants can now start thinking about leveraging these App-like features in their business strategy.

EC-CUBE’s solution was featured during the “Progressive Content Management Systems” session at the Chrome Dev Summit earlier this week. Also, EC-CUBE and SUNDAY SYSTEMS have published the details of their AMP plugin and the overall project in an article linked here (Japanese). You will be able to see what they have done by looking at the code snippets and their architecture. Also please visit the demo page, try the plugin, check out the code on GitHub, and let us know what you think!  Comments and feedback are welcome via the plugin’s GitHub issues or the Slack channel.

Special thanks to Alan Kent, Developer Advocate and Chris Sater, Global Product Partnerships at Google for advising us throughout the project.

Posted by Yusuke Utsunomiya, Mobile Solutions Consultant and Monisha Varadan, Global Product Partnerships at Google

Launching Ad Monetization for AMP stories

Uncategorized

We’ve seen steady growth over the past year in AMP Stories and we are delighted to see the various ways content creators have taken advantage of the rich, immersive canvas for storytelling. We’ve been testing this with a handful for pilot partners and today, we are excited to give all content creators an opportunity to monetize their stories.

A CNN AMP Story showing a Google Pixel Story Ad

Introducing Story Ads

Story Ads are fullscreen ad placements that appear in AMP stories. When we set out to create the advertising experience for story ads, we built it on top of three principles:  

1. Immersive
We wanted to ensure that story ads were immersive and engaging. Like AMP story content, story ads use the entire screen to convey a brand or message using a combination of video, image, or animations. A user continues the tap gesture to skip over the ad if uninterested but also has a consistent way to explore the ad using the call to action button. In addition, every ad has a ‘consistent’ ad attribution label, so users are easily able to distinguish between an ad vs organic content inside a story.

Ad placement & insertion are orchestrated by the runtime and therefore story ads are shown to the user only once they have fully loaded. As a result, story ads by definition are 100% viewable.

2. Performant & Secure
Story ads leverage the open source AMPHTML ad framework, which ensures that the ads are always performant and secure. This is achieved by replacing custom JavaScript with powerful AMP components that provide ad functionality. You can learn more about AMPHTML ads here or check out the resources below to create AMPHTML ads specific for AMP Stories.

3. Open
Continuing in the AMP project’s footprints, we wanted to create an ad ecosystem where any ad provider could participate in delivering monetization solutions for story creators. AMP now has close to 100+ ad network integrations, and we hope to achieve a similar level of diversity for publishers. If you are an ad provider that wants to integrate with AMP stories, please reach out to us.

A Washington Post AMP Story showing a ‘Retropod’ podcast story ad

Today, we are launching support with Google Ad Manager to deliver direct sold ads.

Here is some helpful information to get started:

  1. Learn how to create an AMP Story ad
  2. Learn how to tag your AMP Story to serve ads from Google Ad Manager
  3. Learn how to traffic an AMP story ad in Google Ad Manager
  4. Play with a live sample on the AMP By Example page (the first ad will appear after the first 7 story pages)

Looking ahead, we are working with a number of publishers, advertisers, and ad providers to bring programmatic story ads capabilities and make it easy to create these ads.

As always, we look forward to your feedback.

Posted by Vamsee Jasti, Product Manager, AMP Project

Developer Preview of better AMP URLs in Google Search

Cache

AMP users and publishers have told us that they prefer that the original domain names be used anywhere their AMP pages are displayed.


Earlier this year, we demonstrated a technology named Signed HTTP Exchanges that supports transforming cached AMP URLs on any AMP Cache. Google Chrome has since started an origin trial for Signed Exchanges in Chrome 71. Today Google Search is opening up a developer preview of this technology that any publisher can try out for themselves.

Signed Exchanges, used by an AMP Cache, have benefits for users and publishers, beyond the visual experience of the URL bar. Signed Exchanges also:

  • Provide a guarantee, using cryptographic signatures, that the content is exactly what the publisher intended to show the user.
  • Allow the browser to treat a document as belonging to the publisher’s Origin. This allows a publisher to use first party cookies to customize content, execute service workers, and measure analytics.

Signed Exchanges are part of a wider web proposal named Web Packaging. The Web Packaging specifications, first proposed in 2015 as a W3C draft are evolving over time with feedback from members of standards bodies, other browser vendors, security experts, and publishers and web developers.

Steps to try a Google Search demonstration

  1. Signed Exchanges are currently only enabled in Chrome version 71 or greater. At the time of writing, this requires installing from the Chrome Beta channel but it will soon be released to all Chrome channels.
  2. If you are not using a mobile device such as a smartphone or a tablet, enable mobile emulation on your browser. AMP pages are only displayed to mobile devices. Next, visit https://g.co/webpackagedemo.
  3. This will display a search box. From there, enter a query such as [learn amp by example] and click on “Learn AMP by Example” for the ampbyexample.com home page. There are only a small number of publishers that have implemented this feature so far, so you may want to try this specific query. If you’ve completed these steps correctly, you will see “https://ampbyexample.com” displayed in the browser’s URL bar.

That’s it! The AMP Cache has preloaded the AMP document and Chrome has cryptographically verified that the AMP document was never modified from what the publisher intended, thus enabling the publisher’s URL to be the one that populates the browser address bar.

Under the hood

Prefetching

The moment a Google Search returns an AMP result, the Search results page instructs the browser to fetch the AMP document, “prefetching” it by the device in the background.

When a user clicks on that result, the AMP document can be displayed instantly. This works in part because the document clicked was already fetched and partially loaded.

Heavy resources like images and videos haven’t all been fetched in advance, but the AMP JavaScript libraries make sure to reserve space on the document for those and optimizes the order of loading them. This saves bandwidth if the user doesn’t end up clicking. It also protects the user’s privacy.

An important privacy principle is that until the user clicks on a document, third parties should not be able to determine that the user is interested in it. If preload caused a browser to make network requests to servers other than Google’s, then some of the intent of user queries would leak to sites the user never even clicked on.

To achieve this, neither the document nor any resources fetched before loading the document, can be fetched from third parties. Only once the user has signaled intent to load the document by tapping it do those resources get fetched. We call this mechanism “Privacy Preserving Prefetching” and it provides a buffer between the user and the publisher until the user has chosen which document to visit.

In order to preserve privacy while still prefetching documents, the referrer, Google Search in this case, must load the document from a Google server, the Google AMP Cache.

Signed Exchanges

We would like to achieve the Privacy Preserving Prefetching while simultaneously maintaining the URL and Origin model of AMP documents. We are achieving this using a new collection of browser technologies being drafted under standards bodies such as the W3C and IETF, called “Signed Exchanges”.

A Signed Exchange is an HTTP Request / Response pair, cryptographically signed using a publisher’s own Certificate Private Key. In other words, Signed Exchanges provide digital proof to a browser that the document delivered by an AMP Cache has not been modified from what the publisher intended.

When a browser sees a Signed Exchange and can validate the signature, the browser can display the publisher’s URL, regardless of where the file was delivered from.

AMP Packager

To help publishers build Signed Exchanges, the AMP team has built a tool that can be run by publishers to package and serve Signed Exchanges. The tool is called the AMP Packager.

The AMP Packager runs on publisher’s own infrastructure as a web server backend. It acts like a proxy, accepting an HTTP request for an AMP Web Package, forwarding that request to the publisher’s own backend, and then transforming it into a Signed Exchange.

In addition to packaging the document, the AMP Packager also applies transformations to the document that optimize the page for serving on an AMP Cache. These transformations modify the document itself, without changing the way it looks to users. However, publishers should be able to inspect the code that makes these transformations, so the AMP Packager code, including the transformations applied, is open source.

Developer Preview

Developers who want to opt-in to this developer preview should be first aware that the specification is still changing, and this is an implementation of a snapshot of it.

Once deployed, allow a few days for the Google Search crawler to revisit your site, after which you should be able to try the experience by using the Google Search Demo instructions above.

Option A: Run the AMP Packager tool

The AMP Project has provided the AMP Packager, an open source tool, with detailed instructions on how to run it on your own infrastructure.

During these steps, you will need to generate a new certificate with the CanSignHttpExchanges extension from your Certificate Authority. At the time of writing, the only Certificate Authority that has built a Signed Exchange certificate request flow is DigiCert, with detailed instructions that can be found at Digicert CanSignHTTPExchanges.

Option B: Use a Signed Exchange–Enabled CDN

If you use a CDN provider, ask them if they can provide AMP Signed Exchanges.

At the time of writing, Cloudflare has announced an experimental Cloudflare Worker application implementation of Signed Exchanges as a service to its customers. See this Cloudflare blog post about Web Packaging for more information on how to enable Signed Exchanges on Cloudflare. Once deployed, allow a few days for the Google Search crawler to revisit your site, after which you should be able to try the experience by using the Google Search Demo instructions above.

We’d love to hear from you

We would love to hear your feedback on Signed Exchanges in AMP. You can join the spec discussion, report a chrome bug, report an AMP Packager bug, or general AMP feedback. Your feedback will greatly help us shape the future of AMP and the web.

Posted by Greg Grothaus, Software Engineer, AMP at Google

So your AMP test doesn’t perform — now what?

Uncategorized

Editor’s note: The following was posted on Medium by Martin Schierle, Mobile Solutions Consultant at Google. 

Accelerated Mobile Pages (AMP) are a great and easy way to build very fast pages, and as we know speed is key — for every second delay in mobile page load, conversions can fall by up to 20%. So obviously the first thing people do after they built their first AMP is to A/B test it, sometimes just to find out it may not perform well…

In fact it’s not as trivial to test AMP as it may seem at first. There are a few things to be aware of when you go down the AMP path of speed, and by keeping these in mind you should be just fine!

Target Metrics

When testing AMP, different audiences often look for different metrics. Conversions are often a good target, but it’s important to keep in mind that AMP will have less impact the farther the conversion is from the AMP page. If the conversion is at the end of a 10 screen purchase funnel, it may not help much to just AMP the initial landing page, when the user might still have to navigate through nine slow pages afterwards to finish the conversion. For publishers, ad revenue is a compelling metric, but it’s often overlooked that the incremental uplift through AMP will not necessarily be seen in CPM, but rather in traffic and user engagement. For publishers it therefore makes more sense to look at revenue per session, rather than just revenue per page.

One of the most interesting and misleading metrics is the bounce rate. As page speed is directly correlated to bounce rates, and bounce rates very often directly correlate to conversions and ad revenues, this is what most people look out for when testing AMP.

And — surprise! — it’s more than likely that you’ll see higher bounce rates when testing AMP!

But why?

Survivorship Bias

It is said that when the British army introduced metal helmets in WW1, the rate of head injuries increased afterwards — which seems more than puzzling. However, giving more thought to it the reason becomes clear — a lot of the soldiers who would have died without a helmet now survived (and thus, the number of absolute injuries went up). This effect is called the survivorship bias — you only measure the ones which survived, thereby introducing bias.

The same frequently happens for slow websites — your analytics package of choice might kick in really late during the loading process, and will only measure the users which made it that far. The users bouncing before analytics fires are not seen and not measured.

Consider this graph showing the cumulative bounces over the total load time of an example page.

This page loads in 15 seconds.

We add the moment in time when the Analytics provider fires the first ping: 10 seconds.

This means that we are only measuring 20% of the visitors that are bouncing before the page is fully loaded.

If we would fire the analytics ping after 3 seconds, we’d measure 45% of the visitors that bounce before the page is fully loaded.

When putting the load times and analytics pings next to one another, we can see that the AMP page measures more of the bounces that are happening.

Even if AMP outperforms the normal page, the measured bounce rate might still be higher, since we capture more of the bouncing traffic. You can solve this by improving analytics load time on canonical, or by measuring real load abandonment as described here.

The Low Bar

In a time where most websites are too slow, it often takes really dedicated users to complete a purchase funnel on mobile — whoever is willing to wait ten or twenty seconds for a website to load, is probably very determined to finish the task she came for. So if AMP gets introduced, relative conversions might seem to go down — as the bar is now so low, that even less dedicated, loyal or interested users make it to your website who might convert less frequently. Be aware that this is still incremental traffic and incremental conversions, just that the relative conversion probability might seem lower. In these cases it might be worthwhile to also look at absolute conversions and general traffic.

App Users

Many websites offer an alternative native app which shows the same content as the website, and the mobile phone might open up the native app for URLs to this domain (but not for the same content on AMP cache). So if an app user visits a page on the AMP cache and then clicks through to regular website, the native app will open instead. As most analytics packages define a bounce as a session with only one hit, such a user flow would count as bounce — even though the user didn’t bounce but potentially finished her journey successfully in the native app.

Luckily, this scenario only applies for companies with a high app penetration in the market, but is still important to keep in mind.

Visual and Functional Parity

Historically many websites implemented AMP as a parallel version of their website, as a fast entry point from search (‘paired AMP’). In some of these cases AMP was intentionally or unintentionally not equivalent to the regular page — maybe due to oversight, maybe to save resources, or maybe the two versions got out of sync over time. However — given most companies are going through massive efforts and A/B testing to get their landing pages as optimized as possible, it seems obvious that the AMP version of the same page might not perform well, if not being functionally and visually identical (at least with respect to all critical content and user actions). For a fair A/B test, the pages need to be comparable. From a visual perspective you can use the Chrome screenshot tool to create screenshots of AMP and canonical, and compare them manually or through an image diff tool (there are many available, e.g. this one). From a functional perspective it’s worth clicking through the main user actions on both versions, and make sure they feel and behave the same way — look out especially for autofills, autocompletes, search auto correction etc.

Interaction Events

There are certain user actions on a website, which you might want to count as a successful page visit (e.g. watching a video, filling a lead form etc.). Normally a page visit with such actions is not counted as a bounce, even when the user directly leaves the page afterwards. Within Google Analytics these events are called ‘interaction events’ in contrast to ‘non-interaction events’, where a bounce is still counted. So if the regular website has many interaction events defined, but the AMP site not, this will directly influence bounce rates of both versions, and will make them impossible to compare. For Google Analytics this can be verified easily for both versions via the Google Tag Assistant Extension.

User IDs

Two of the core value proposition points of AMP are AMP Caches and Prerendering. However, the delivery from the AMP cache means the initial landing page is delivered from a different host than the regular site. Given that most analytics vendors are using first party cookies (due to third party blocks as e.g. introduced through Safari’s ITP), it means a user will be assigned two different cookies with two different user ids on her journey from AMP to canonical website. This is known to (artificially) inflate user ids, sessions and bounce rates, and is explained in more detail here.

The solution is to have a unique user ID across both hosts — AMP offers this with its AMP Linker, which can be integrated by 3rd party analytics vendors, and is also integrated in Google Analytics. However — not all analytics vendors support this, and if not implemented correctly it may fail. Then the AMP visit would be erroneously counted as a bounce, as the same user wasn’t seen again on canonical. So it’s important to verify that your analytics package is sending and using a consistent client id across AMP cache and canonical — this can be verified manually through Chrome DevTools or potentially dedicated troubleshooting tools from your analytics vendors. For Google Analytics, this can be easily checked with the Tag Assistant recording feature. Here you can navigate from search result page to AMP from cache to canonical, and then doublecheck in the recording that there is only one user session counted, which started on the AMP page, and uses the same user ids across all page hits.

Checklist

So what are the things to keep in mind when testing and evaluating AMP?

  • Make sure to verify visual and functional parity (e.g. through Chrome screenshot tool and by trying all CTAs).
  • Make sure that analytics on regular mweb doesn’t fire much later than on AMP, and stay aware of the blind spot and the survivorship bias.
  • Make sure to not measure against a goal which happens much later in the funnel after the AMP page. If you measure conversions, try to focus on those happening on AMP itself (e.g. clickout).
  • Be sure to keep in mind the bias through your native app, in case it has significant market penetration.
  • Click through the whole journey from search to regular mweb, and verify that your analytics records only one user session, which starts on AMP, and uses a consistent user ID. You can do this via dedicated troubleshooting tools from your analytics provider (if available), via Chrome DevTools (look out for user identifiers in analytics pings) or via the Tag Assistant recording feature for Google Analytics.

Posted on Medium by Martin Schierle, Mobile Solutions Consultant at Google

Streaming in the Shadow Reader

Uncategorized

What did we do?

We made The Shadow Reader faster again!

We created The Shadow Reader (https://amp.cards) to demonstrate how AMP pages can be used within a Progressive Web App (PWA) (read our announcement post for more context). The site folds real articles from The Guardian into an immersive news reader experience. It’s a demo, but it’s also a fully functional site, containing everything you need to embed AMPs inside a beautiful PWA.

Earlier this year, we enhanced the Shadow Reader to follow an AMP=>AMP/PWA pattern, which made initial article loads faster and made the entire app more SEO-friendly. Now, we’ve made the site even faster by adding DOM streaming. That means the browser can render content as it loads – it doesn’t have to wait until the full load!

The video below shows the new streaming Shadow Reader on the left and the old version on the right. To clearly see the difference streaming makes, I used Charles proxy to simulate a 56kbps connection. Though this is obviously worse than the average connection, the Shadow Reader was already quite fast!  In AMPs, all JavaScript is asynchronous. And in many AMPs, including The Guardian’s, most of the HTML consists of CSS in the <head>. There isn’t that much <body> to stream! So it’s tough to spot the difference on a good connection, or even in a 3G connection simulated in DevTools.

It takes a little while to load the HTML in the <head>, so this video starts a few seconds in. Once we get to the <body>, the streaming works its magic, and in my unlablike test environment, the main article text was entirely visible 11 seconds earlier with streaming than without. Interestingly, the article’s YouTube thumbnail showed up right after the text in the streaming version, whereas in non-streaming, it showed up 67 seconds later than it did in the streaming version.

 

How does this work?

AMP comes in a flavor called Shadow AMP, which allows an AMP to exist entirely within a shadow root. Shadow AMP lets you embed an AMP in another webpage. You create a shadow AMP like this:

const shadowDoc = AMP.attachShadowDoc(container, doc, url);

If you’ve read Jake Archibald’s post on the subject, you know that it’s possible to stream content into the DOM via a trick involving an <iframe>. What you may not know is that this technique has been used to stream AMP content into a shadow root!  You simply create the shadow AMP with a different method:

const shadowDoc = AMP.attachShadowDocAsStream(container, url); 

Here’s how you stream AMP content into a shadow root, in 4 steps:

  1. Create the streaming shadow AMP
  2. Access your content with fetch()
  3. Stream that content into the shadow AMP.
  4. When fetch() tells you it’s done, close the writer.


The four steps

1. Create the streaming shadow AMP. See above.

2. Access your content with fetch(). Open a fetch() to the AMP URL. fetch() returns a promise that resolves to a response object. This object is a stream. Usually you’d use a method that read the stream to completion, then processed it in some way. For example, you might use response.text() or response.blob(), like this:

fetch('example.com/amp.html')
.then(response => response.text())
.then(text => console.log(text));

In our case, though, we’re going to use the response object as a stream. That’s easy to access via its body property, which exposes a ReadableStream.

fetch('example.com/amp.html')
.then(response => response.body())
.then( // read from the ReadableStream );

That ReadableStream contains a getReader() method which locks the stream and returns a ReadableStreamDefaultReader. The ReadableStreamDefaultReader exposes a read() method that provides access to chunks of HTML as they stream in, like this:

// read from the ReadableStream
let reader = response.body.getReader();
let chunk = await reader.read();


3. Stream that content into the shadow AMP.
Meanwhile, the shadowDoc we’ve created contains a writer object, and that writer contains write and close methods that can be used for streaming. So you can stream content into the DOM like this:

shadowDoc.writer.write(html);

and close it like this:

shadowDoc.writer.close();

So once we have the stream reader and the shadowDoc streamer, we repeat the following steps:

  • get chunk from stream
  • decode chunk and write it into the DOM

until the stream reader tells us it’s done, which it does through a boolean.

One way to do this is through recursion. In the Shadow Reader, we used await. Here’s the gist of our streaming code:

   const shadowDoc = AMP.attachShadowDocAsStream(container, url);
    fetch(url).then(async response => {
      let reader = response.body.getReader();
      let decoder = new TextDecoder();
      while (true) {
        let chunk = await reader.read();
        if (chunk.done) {
          shadowDoc.writer.close();
          break;
        }
        let html = decoder.decode(
          chunk.value || new Uint8Array(),
          {stream: !chunk.done}
        );
        if (html) {
          shadowDoc.writer.write(html);
        }
      }
    });


4. When
fetch() tells you it’s done, close the writer. Looks like we covered that in step 3 above. And Bob’s your uncle!

Wait, you say – what about browsers that don’t yet support streaming?  Fortunately, in such cases, a call to AMP.attachShadowDocAsStream() gracefully degrades to the same functionality as AMP.attachShadowDoc().

The Shadow Reader loads and prerenders the top three articles when the app loads. We didn’t use streaming for these, and just to be on the safe side, we also don’t use streaming for browsers that don’t support it. You can conveniently compare the streaming and non-streaming approaches in the code by looking at load() and stream() in Article.js.

 

Reality is complicated.

What would life be without a complexity or two?

Testing. If you’re like me, you’d like to see the streaming happen, chunk by chunk, by inserting a breakpoint into the streaming code. But this may not give the desired result. At the time I’m writing this, with breakpoints set in Chrome, the shadow root remains invisible until it’s closed. You can see streaming more easily if you use DevTools to throttle your connection to 2G. It works even better to use a proxy server that sends content in smaller chunks. Or to test with an exceptionally large AMP, since many AMPs contain little HTML, and the entire content may arrive in only 2 chunks or even 1.

Order of operations. Before streaming, the Shadow Reader showed a loading spinner, loaded the article, rendered it in a hidden div, then removed the spinner and showed the article. With streaming, we want to show the article immediately!  This meant we had to move around some of the animations. It also meant that any modification of the AMP had to occur immediately, not after it fully loaded. In particular:

Hiding unwanted bits of the AMP. When the Shadow Reader imports an AMP, it needs to remove the original article’s menu, header, or footer, since those are already part of the PWA, and it would be silly to have, say, two menus. The Shadow Reader used to create the whole article in the DOM, remove the unwanted bits, then display the sanitized article. With streaming, we have to instead never show the unwanted elements in the first place. We could filter out unwanted elements as HTML came in, but this is difficult. Instead, we simply hide those elements with CSS.

Normally this is simple. AMP automatically applies an amp-shadow class to the shadow root, which allows you to easily style an AMP differently when it’s in a shadow root. Unfortunately, in our case, we don’t control the Guardian articles. We solve this by looking for a chunk that contains <style amp-custom>, and then injecting our CSS right into that.

This is useful anyway, because the Shadow Reader included a custom stylesheet in the AMP. Unfortunately, during streaming, a browser can’t be counted on to load and apply CSS as soon as it sees a <style> tag – it may wait until streaming is done. We solve this problem by injecting the custom stylesheet as well. Done!  Here’s the actual final streaming code:

    fetch(this.proxyUrl).then(async response => {
      let reader = response.body.getReader();
      let decoder = new TextDecoder();

      while (true) {
        let chunk = await reader.read();

        if (chunk.done) {
          shadowDoc.writer.close();
          break;
        }

        let html = decoder.decode(
          chunk.value || new Uint8Array(),
          {stream: !chunk.done}
        );

        // check each chunk of HTML to see if it contains <style amp-custom>. If so, add in some extra CSS.
        if (html) {
          html = shadowReader.backend.injectCSS(html);

          // when we've got the body, start the process of animating the card and showing the article,
          // placing the card before the article
          if (html.includes('<body')) {
            html = article.prependCardHtml(html);
            shadowDoc.writer.write(html);
            article.card.animate();
            article.show();

          } else {
            shadowDoc.writer.write(html);
          }
        }
      }
    });


Left as an exercise for the reader.
At press time, I have not dealt with an edge case: this fails if <body is split between two chunks. Please submit a PR!

Posted by Ben Morss, Developer Advocate for AMP, at Google

The latest from AMP Analytics providers

Uncategorized

As AMP turns three, we can’t help but look back with nostalgia on how far along we’ve come along in our mission – Build the best open web technology that lets publishers create fast and secure pages without compromising on user-experience. That is why we built the analytics infrastructure in AMP that allows for “collect once, notify many” architecture for metrics on the page. This allows for AMP Analytics to be fast and accurate while using fewer resources on mobile devices.

AMP Analytics’ interface allows third-party analytics partners to easily integrate with the service. Websites building with AMP can choose from over 55 analytics providers today to get accurate metrics of the page.

We’re continuing to invest in the ecosystem, and wanted to give a shoutout to some of the amazing providers who also continue to invest in AMP.   Over the coming few months, you’ll see deeper integrations from the following partners with AMP.


Adobe Analytics
AMP continues to be adopted by Adobe customers, who love the speed that AMP provides and the mobile first approach. Adobe has supported AMP from the beginning with two different analytics templates that customers can use based on their needs. In 2019, Adobe would like to build out more robust ID linking capabilities as well as enhance the current templates to support more events out of the box to make the implementation even easier. For customers interested in implementing AMP they can reference implementation guide.

Bizible
Bizible is currently in the process of building out their AMP integration.  By next year, native event and pageview tracking will be available on AMP-enabled content.  Additionally, AMP pageviews will be considered as touchpoints for attribution across the entire customer journey.

Chartbeat
Chartbeat was one of the inaugural analytics partners for AMP and continues to provide active support and insight to publishers around AMP traffic. Their AMP integration, as well as their trends and best practices reports, illuminates a major source of traffic for publishers and enables them to understand how engagement compares across platforms.  Chartbeat will continue to monitor AMP trends as well as broader platform shifts so they can shed light on the implications that major changes hold for publishers. For more questions about AMP capabilities or initiatives, please reach out at feedback@chartbeat.com.

Google
Many websites use Google Analytics to better understand how well their site is meeting the needs of their users, and some sites also benefit from being able to measure campaign performance in Google Ads and Search Ads 360.  Google Tag Manager offers a solution that allows websites to instrument AMP pages with analytics tags once and make subsequent updates without touching code. For those who are not using Google Tag Manager, the recently released global site tag for AMP provides the benefit of having a single implementation for Analytics, Google Ads, and Search Ads 360.

New Relic
The New Relic Browser allows users to capture and visualize the performance of their AMP pages against their standard webpages. AMP data is captured and visualized through New Relic Insights, New Relic’s dashboard solution, to create detailed and customizable views into AMP page performance.

New Relic has plans to include AMP performance metrics into the New Relic Browser’s product in the first half of 2019. This will provide Browser customers with a clear way to compare AMP to non AMP page performance using New Relic Browser to measure if their investment in AMP is paying through page performance.  For more information, please get in touch here.

Nielsen
Nielsen supports measurement of static/text content on AMP as part of Digital Content Ratings, which measures digital content consumption across platforms, in every market globally where the product is available. This enables publishers to get a holistic view of their audience across the variety of distribution channels they may use, as well as the unique reach they achieve on AMP. Nielsen currently supports DCR measurement on various online video platforms and plans to extend support for native amp-video in 2019. This would allow publishers to see audience composition on AMP by content type. For further information on Nielsen’s measurement capabilities, please click here.

Optimizely
Many publishers today use Optimizely to A/B test article headlines, ad viewability, and subscription flows.  Optimizely is working with the AMP Project on a solution to support these same powerful experiments in AMP. This solution will allow customers to use Optimizely’s visual editor to build A/B tests and then see them take effect on AMP pages, along with measuring real-time experiment metrics through Optimizely’s Stats Engine.

Parsely
Parse.ly fully supports measuring audiences on AMP, including the ability to support engaged time. With Parse.ly, content creators are able to see a single, unified view of their audience and content across multiple channels like AMP, mobile apps and website.  Soon, Parse.ly will support better AMP breakdowns in their API, including newer standards like AMP Stories. They are committed to support audience measurement on AMP as the standard evolves.  Please contact support@parsely.com if you have any questions.

Tealium
Tealium supports AMP in multiple ways.  One is that Tealium’s utag.js running JavaScript in an AMP iframe.  A second implementation is to host JSON on Tealium’s CDN for dynamically creating tracking beacons or defining custom event listeners.  The third is the “Tealium Collect” AMP Analytics component (current waiting merge in Github) that sends data to Tealium’s EventStream for server-side delivery of your events.

Tealium is currently working on Tealium Collect AMP Analytics component. Tealium’s Hosted Data Layer (HDL) service recently added JSON support which means that you can leverage this for definition and delivery of AMP JSON.  For questions, please go to Tealium’s Learning Community here.


As you can see from these updates, the AMP Analytics format and the number of providers supporting it continue to grow.  As updates to the format become available and new providers come online, we will update the community.

Posted by Jeff Jose, Product Manager, AMP, at Google

AMP story learnings and best practices

Stories

Since we announced the developer preview of AMP Stories, we’ve been focused on ensuring the format is an easy and compelling way for publishers to create visual content on the web. It’s been exciting to see the experimentation and direction that some early testers have taken with the product.

Naturally we’ve also had a lot of people asking us to share early learnings about AMP stories – what works, what doesn’t –  so we’ve tried to compile a quick run-down of best practices here:

Story Quality

1) AMP stories are just that – stories, intended to communicate a message, explain a situation, an event, a person, a concept. Publishers should be telling important stories worth reading that are both visually-rich and share relevant information in their text accompaniment. We’ve seen visual stories created in a variety of content categories, from sports to recipes to hard news to entertainment. Some things to avoid:

  • AMP stories which are just image slideshows or auto-generated are often lacking the core content that users are expecting and looking for when choosing what to read.
  • AMP stories should be complete, self-contained articles; they should make sense and be useful to the user without requiring the user to link out.

See AMP story examples from the Washington Post, Mic.com and FIFA (shown above).

Example from FIFA

2) Asset Quality Matters 💯: We see dramatically better user engagement when the right assets are selected and properly formatted for an AMP story so they feel native to the experience. Beyond relevance, that means the images and videos are high-resolution, portrait-oriented, and full-bleed as it makes the experience more immersive, beautiful, and engaging to users. While we know leveraging existing assets can expedite story creation, it’s critical that the assets selected feel native to the experience and are best-suited to the dimensions of the device they are being viewed on.

Example from The Washington Post

3) Go Beyond Still Images: Some of our most engaging AMP stories incorporate short videos or Ken Burns effects to deeply engage the user. When using video, check out our video documentation on AMPproject.org, including these tips:

  • Best practice is to use 720p, which at a 3:4 aspect ratio is 720px width by 960px height: Over-indexing on high-resolution video unnecessarily slows down the story load times for diminishing returns on quality.
  • If audio is used, it’s important that the AMP story can stand on its own with the audio turned off, given that a fair amount of consumption happens when users are on-the-go and may not want to turn audio on. For this reason the AMP framework supports captions, as well as the ability to specify whether a video contains audio via the `noaudio` attribute.

Example from WIRED 

Building an Audience & Monetizing It

4) “If a great AMP story gets created but no one sees it, did it happen?” Because AMP stories are self-canonical, the quality and relevance (across the web) is derived from the AMP Story itself. Like any other webpage, these signals are important in order to be served by platforms like Google Search. Publishers must promote their AMP Stories to users on their own homepages and sites, via social channels, and any other standard mechanisms of promotion. News publishers should consider linking from a Google News sitemap.

5) Discoverability & Indexing: There are a number of things that publishers will want to do to ensure that AMP Stories – while they may be visual-heavy – have the necessary metadata and text structure to ensure Google can discover and understand AMP stories:

  • Make all text crawlable: Use text overlays, rather than baking titles or accompanying text into images or videos themselves.
  • Apply structured data markup: Like AMP, publishers should include the appropriate structured data in addition to the AMP Story specific metadata. Note that there are unique sizing requirements for the primary thumbnail and publisher logo images specific to AMP Story documents.

6) Experiment with monetization: While it’s just been a few months since we shared public documentation on github for Single Page Ads on AMP Stories, we’ve begun to see early adoption of the format leveraging both image and video assets. We’ve also seen experimentation with ad real estate to run experiments for their own products and for their own podcasts, newsletters, etc.

AMP story ad

Tooling & Development

7) Test early and often. There are self-service tools available (Google’s AMP Test site and AMP Validation Testing Site) so publishers can test out AMP stories prior to pushing them live and identify and solve for errors. Additionally, publishers can test structured data markup here. The AMP Stories community will look to make these tools even more robust over time and would love feedback on how to make them more useful.

8) Make It Yours!: Like AMP, AMP Stories is an open-source project and its success will depend on the contributions of publishers and partners around the globe contributing feedback, requests for functionality, and even code themselves. Captions and subtitles support was added to the project by an organization that wanted support for the feature, but needed it quickly.  With a bit of guidance from the AMP team, they implemented the feature, making it available to all here.  If you have questions or thoughts, don’t hesitate to reach out to us via the right channel here.

Example of captions in an AMP story

We’re excited to continue to work with partners to evolve the format and look forward to hearing – and seeing! – from you.

Looking for more resources?

Posted by Lisa Lehman, AMP story Partnerships, Google