AMP

Make sure your Signed Exchanges serving is up and healthy

Cache, Signed Exchanges

Have you already set up an amppackager server to serve your AMP pages packaged as Signed Exchanges? If so, that’s great! This means the browser displays your domain when landed from a Google Search.

Any service that runs in production can run into issues, and amppackager is no exception. Below are a few examples of such issues:

  • The instance running the amppackager may go down.
  • The underlying AMP page server may go down. 
  • The unsigned AMP page may not be a valid AMP page, or be larger than 4MB, or violate some of the other amppackager‘s requirements.

You can notice these issues early and even prevent them by monitoring amppackager’s health and performance. The server’s /healthz endpoint lets you do a one-off manual health check. As of August 2020 the server also provides the /metrics endpoint, which produces elaborate metrics. The latter endpoint supports the Prometheus framework, letting you set up automatic checks and alerts.

Quick health check: the /healthz endpoint

Let’s start by checking the /healthz endpoint to ensure that amppackager is healthy, meaning it’s running and has a fresh and valid certificate. 

Run the command below on amppackager’s instance (adjust the port if necessary):

$ curl https://localhost:8080/healthz
okCode language: Bash (bash)

If unhealthy, the server will provide an error message:

$ curl https://localhost:8080/healthz
not healthy: Cannot find issuer certificate in CertFile.Code language: Bash (bash)

You can use the /healthz endpoint in a few ways:

  • Check it when you’ve just installed or reinstalled amppackager. 
  • Check it as a first step in any troubleshooting scenario. 
  • Integrate the check with control plane tooling, for instance using Docker HEALTHCHECK or Kubernetes readiness probes.

Elaborate stats: the /metrics endpoint

The /metrics endpoint provides a large array of metrics, specifically:

  • the processed requests count,
  • the error rates,
  • the latencies distribution,
  • the document size distribution (lets you notice when you approach the 4MB limit),
  • and other stats.

The endpoint provides stats for the amppackager server, and also stats for the underlying AMP page server. 

The example command below checks how many requests amppackager has processed since it’s been up.

$ curl  https://127.0.0.1:8080/metrics | grep total_requests_by_code_and_url

# HELP total_requests_by_code_and_url Total number of requests by HTTP code and URL.
# TYPE total_requests_by_code_and_url counter
total_requests_by_code_and_url{code="200",handler="healthz"} 3
total_requests_by_code_and_url{code="200",handler="validityMap"} 5
total_requests_by_code_and_url{code="200",handler="signer"} 6
total_requests_by_code_and_url{code="502",handler="signer"} 4
total_requests_by_code_and_url{code="404",handler="handler_not_assigned"} 1Code language: Bash (bash)

The command fetches all the metrics available via the /metrics endpoint. It then greps the report for total_requests_by_code_and_url, the metric in question. 

The report starts with the # HELP section explaining the metric, followed by the # TYPE section with the metric’s type, followed by the actual stats.

The stats are broken down by the response HTTP code, and by the internal amppackager’s module (handler) that has handled the request. The stats above report 3 requests to the healthz handler that got a 200 response (OK), 4 requests to the signer handler that got a 502 response (Bad Gateway) etc.

Check the documentation for other available metrics, more usage examples, and for details on how to interpret the stats – including topics like handlers, labels, stats breakdown by handler and by response code, and interpreting distribution-type metrics aka summaries (e.g. latencies distributions).

Leverage Prometheus: visualize and automate your monitoring

As we’ve seen above /metrics is a self-sufficient endpoint accessible via command line. 

On top of that, the endpoint is compatible with Prometheus. It’s a powerful monitoring framework you can integrate with to visualize the stats and setup automatic notifications.

Here’s a setup sequence we recommend:

  1. Explore the metrics that amppackager provides. Pick the ones you’d like to monitor.
  2. Set up a Prometheus server.
  3. Set up all amppackager replicas as targets for the Prometheus server. Use the multi-target exporter pattern.
  4. Try querying the metrics you chose.
  5. Visualize the metrics via grafana.
  6. Set up alerts that will notify you of abnormal behavior (e.g. latencies growing beyond 60 seconds – see more examples).

Conclusion

amppackager offers a variety of tools to instrument your monitoring and ensure a high level of availability and stability of your signed exchanges serving. From quick manual health checks to sophisticated automation via Prometheus, we encourage you to pick the setup that’s right for you and make monitoring part of your process.

Written by Michael Rybak, Software Engineer, Google, The AMP Project