Skip to content
downpipes docs

Scrape downpipes backup-health metrics into Elastic Observability

Elastic Observability reads downpipes’ backup-health metrics through the Prometheus /metrics scrape. Elastic’s OTLP/HTTP intake accepts binary protobuf only, and JSON-encoded support is an open feature request rather than a shipped capability, so the JSON body this engine’s OTLP push sends does not reach Elastic. Point Elastic Agent’s Prometheus input at the scrape endpoint instead.

The scrape carries no field-mapping work. Elastic’s Prometheus input parses the standard exposition format directly.

What you need

  • An Elastic Agent you run (or an OpenTelemetry Collector feeding Elastic), enrolled in your stack.
  • Owner access to the downpipes console to mint a read-only metrics token.

Set it up

  1. In the console, open Integrations, choose this vendor’s tile, and mint a read-only token in the metrics scrape scope there. Copy the secret once; note the metrics URL shown next to it, which is your console origin plus /metrics.
  2. In Fleet, add the Prometheus integration to your Elastic Agent policy and set it to the Prometheus input, with the metrics URL as the endpoint and a bearer header carrying your minted token.
  3. Roll the policy out to the agent and confirm the downpipe_* metrics arrive in Elastic. The wire format is standard Prometheus exposition, so there is no field mapping to build.
  4. Add the staleness rule on downpipe_backup_last_success_timestamp_seconds in Kibana against each downpipe’s own cadence.
  5. If Cloudflare Access fronts your console hostname, an unattended scrape is turned away at the edge before the engine ever sees the bearer, whatever credential it carries. Before pointing Elastic Agent’s Prometheus input at it, either give the agent an Access service token to send as extra headers (a path-scoped Access application with a Service Auth policy), or add a narrow Bypass over /metrics; the minted credential still gates the scrape either way. The full remedy is on the /metrics endpoint.

Good to know

  • Elastic’s OTLP intake accepts protobuf only, so the direct push does not reach it and the scrape is the route that works. The OTLP push page carries the per-vendor state.
  • The only running piece you supply is the Elastic Agent (or a Collector) doing the scraping; there is no downpipes-specific field mapping. The metrics token is time-boxed like every pull credential, defaulting to a year from the mint and capped at 400 days, so once it lapses the scrape returns 401 until you re-mint; the console shows the expiry date beside the credential. Full scrape mechanics are on the /metrics endpoint, and the OTLP-acceptance table is on OTLP metrics push.

Last updated .