Skip to content
downpipes docs

Scrape downpipes backup-health metrics into Elastic Observability

Elastic Observability reads downpipes’ backup-health metrics through the Prometheus /metrics scrape. Elastic’s OTLP/HTTP intake accepts binary protobuf only, and JSON-encoded support is an open feature request rather than a shipped capability, so the JSON body this engine’s OTLP push sends does not reach Elastic. Point Elastic Agent’s Prometheus input at the scrape endpoint instead.

The scrape carries no field-mapping work. Elastic’s Prometheus input parses the standard exposition format directly.

What you need

  • An Elastic Agent you run (or an OpenTelemetry Collector feeding Elastic), enrolled in your stack.
  • Owner access to the downpipes console to mint a read-only metrics token.

Set it up

  1. In the console, open Integrations, choose this vendor’s tile, and mint a read-only token in the metrics scrape scope there. Copy the secret once; note the metrics URL shown next to it, which is your console origin plus /metrics.
  2. In Fleet, add the Prometheus integration to your Elastic Agent policy and set it to the Prometheus input, with the metrics URL as the endpoint and a bearer header carrying your minted token.
  3. Roll the policy out to the agent and confirm the downpipe_* metrics arrive in Elastic. The wire format is standard Prometheus exposition, so there is no field mapping to build.
  4. Add the staleness rule on downpipe_backup_last_success_timestamp_seconds in Kibana against each downpipe’s own cadence.
  5. If Cloudflare Access fronts your console hostname, an unattended scrape is turned away at the edge before the engine ever sees the bearer, whatever credential it carries. Before pointing Elastic Agent’s Prometheus input at it, add a self-hosted Access application whose path is exactly /metrics on your console hostname. Give that application a Service Auth policy and issue an Access service token. If the agent can set request headers, it sends CF-Access-Client-Id and CF-Access-Client-Secret on every request, beside the metrics bearer in Authorization. Prefer this arrangement. If the agent cannot set request headers, give the same path-scoped application a Bypass policy instead. The minted credential still gates the scrape either way. On the split topology, point the agent at the engine’s own hostname, which has its own Access policy (the split topology). For the Prometheus form of these headers, see reaching the endpoint.

Good to know

  • Elastic’s OTLP intake accepts protobuf only, so the direct push does not reach it and the scrape is the route that works. The OTLP push page carries the per-vendor state.
  • The only running piece you supply is the Elastic Agent (or a Collector) doing the scraping; there is no downpipes-specific field mapping. The metrics token is time-boxed like every pull credential, defaulting to a year from the mint and capped at 400 days, so once it lapses the scrape returns 401 until you re-mint; the console shows the expiry date beside the credential. Full scrape mechanics are on the /metrics endpoint, and the OTLP-acceptance table is on OTLP metrics push.

Last updated .