Prometheus metrics: the /metrics scrape endpoint

GET /metrics is a Prometheus text-exposition scrape surface: point a scraper at it and it answers with the standard # HELP / # TYPE preamble and one sample line per series, the exact shape any Prometheus-compatible tool already parses. Nothing about the wire format needs teaching. This is the Tier 1 surface named on the monitoring overview: the effort is in reaching the endpoint and reading the credential correctly, not in getting your tool to understand what comes back.

The endpoint reads the same run-history and replication state that already drives the backup-failure, backup-stale and replication-degraded alerts described in alert events. It computes nothing new; it renders the same facts in a different shape for a different reader.

Minting the metrics credential

The endpoint is reached by a bearer credential, not a console session, so an unattended scrape running every fifteen or thirty seconds never has to hold a browser cookie or pass a CSRF check. The credential is a third scope on the exact mechanism owner-minted pull credentials documents for vendor diagnostics and the SIEM audit feed: an Owner mints it, the secret is shown once, and only its SHA-384 hash is stored (mintIngestCredential, engine/src/admin/support-ingest.ts).

The metrics scope carries the longest lifetime of the three on purpose: it is meant to sit in a scrape target’s configuration for the long run, the same way a Prometheus job’s bearer file usually does, rather than a lease you renew every quarter. It still expires on the engine’s own clock and is still revocable at any moment, the same as the other two.

Mint it from the console on the Integrations screen. Open a metrics vendor’s tile (Prometheus is the obvious one) and its panel carries the “Metrics scrape (Prometheus)” credential to mint, with the Metrics endpoint URL shown right above it to point a scraper at, a copy button included. The mint is Owner-only, and once a second Owner exists on the account it is dual-control gated the same way minting a diagnostics or audit-feed credential is: the first attempt queues a pending approval and a second Owner acts before the token is issued. The secret appears once; capture it then, because only its hash is stored. (The diagnostics scope, for a support pull, still lives under Settings; the audit-feed and metrics scopes moved onto their Integrations tiles.)

POST /admin/support/credentials HTTP/1.1
Host: console.example.com
Content-Type: application/json

{"scope": "metrics"}

{
  "scope": "metrics",
  "clientId": "dpc_AbCdEfGhI",
  "secret": "dps_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "bearer": "dpc_AbCdEfGhI.dps_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "expiresAt": "2027-07-05T00:00:00.000Z",
  "note": "the secret is shown once and stored only as a hash; revoke and re-mint to rotate"
}

Reaching the endpoint

The engine serves /metrics itself, and in the shipped single-hostname topology the console worker proxies it through to the engine over the ENGINE service binding, alongside /admin/* and /support/* (isEngineSurface, console/src/worker.ts; see the same-origin proxy). So a scrape aimed at https://console.example.com/metrics reaches the engine on the default topology, with no second hostname to stand up. The metrics tile’s panel shows the exact URL to use, derived from your own origin, so you copy it rather than assemble it by hand.

The split topology, where the engine takes its own custom domain (the split topology), works too: point the scraper at the engine’s own hostname, which the engine answers directly. It is no longer required for scraping, though; the default single hostname reaches the endpoint on its own.

Pointing a scraper at it

With the URL from the panel and the bearer above, the configuration is the one every Prometheus-compatible tool already takes for a bearer-secured target.

scrape_configs:
  - job_name: downpipes
    metrics_path: /metrics
    scheme: https
    static_configs:
      - targets: ["console.example.com"]
    authorization:
      type: Bearer
      credentials: "dpc_AbCdEfGhI.dps_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

Every row above reads the identical wire format the same way, standard Prometheus text exposition needing no downpipes-specific translation, and that much genuinely is Tier 1. What still varies is the effort to point a given tool at it. Grafana Cloud’s hosted scraper is the one fully agentless case here: paste the URL and bearer into its UI and there is nothing else to run. Prometheus, Grafana Agent/Alloy, Elastic’s Prometheus input and the Splunk Distribution of the OpenTelemetry Collector are zero-mapping in the same way, but each still needs its own already-running agent or collector pointed at the endpoint, an ordinary “add a scrape target” step rather than a zero-effort one. Datadog and Dynatrace need genuine configuration: Datadog’s OpenMetrics check needs three configuration fields filled in, low effort but not zero, and Dynatrace’s route needs either a YAML extension mapping file or an OpenTelemetry Collector, plus the ActiveGate or Collector agent itself running. New Relic has no agentless pull at all: reaching it means running the remote-write integration or the infrastructure agent yourself, not pointing an existing hosted scraper at a URL the way Grafana Cloud’s does.

The canonical metric set

Every metric is a gauge, labelled by downpipe_id and downpipe_name (and, for the destination-health series, destination as well). A series is present only when the engine has a real fact to report; it is omitted, never fabricated, when there is nothing yet to say.

The classic staleness alert

The endpoint exists to make the standard SRE idiom work against your own thresholds, not the engine’s guess at what counts as stale.

time() - downpipe_backup_last_success_timestamp_seconds{downpipe_id="dp_8f2a1"} > 90000

Set the threshold to your own downpipe’s cadence budget plus headroom, not a fixed number across every downpipe: an hourly downpipe and a weekly one need different thresholds, and only you know which is which. Two companion facts make the rule honest rather than merely convenient.

First, the comparison above never fires for a downpipe that has never had a successful run, because an absent series never satisfies a numeric comparison; pair it with absent(downpipe_backup_last_success_timestamp_seconds{downpipe_id="dp_8f2a1"}) if you also want to catch a downpipe that has run zero times. Second, staleness and failure are different conditions worth alerting on separately: downpipe_backup_success == 0 catches a run that concluded and failed, which can be true even while the last-success timestamp is still recent (an older success followed by a newer failure), so a monitoring stack that only watches the staleness threshold can miss a failing downpipe that has not yet gone stale.

Windowed counters, not a lifetime total

downpipe_backup_recent_attempts, _recent_successes and _recent_failures are gauges, not _total counters, and the name says so on purpose. The engine retains a bounded run-history ring per downpipe (50 runs), and these three metrics count only what that ring currently holds. A count that can shrink as old runs age out of the ring would read to rate() or increase() as a counter reset and produce a phantom spike; treating the value as a level rather than a rate is the correct read, and the gauge type is there to stop a consumer applying the wrong idiom to it.

That fixed window is a count of runs, not a span of time, so its wall-clock coverage depends entirely on the downpipe’s own schedule. Fifty retained runs cover roughly the last two days for a downpipe backing up hourly, and roughly the last fifty days for one backing up daily. A single alert threshold applied uniformly across downpipes on different schedules, for example firing whenever recent_failures crosses some fixed count, therefore means a different amount of wall-clock history depending on which downpipe it is evaluated against. Set the threshold with each downpipe’s own schedule in mind, not as one number copied across every downpipe regardless of its cadence.

Injection safety

A downpipe or destination name is an operator-chosen string, so it is treated as untrusted before it reaches a label value. Every label value is escaped for the Prometheus text-exposition grammar (backslash first, then the double quote, then any line break folded to the two-character \n escape), so a name carrying a quote or a newline can never forge a second series or break out of the line it belongs to (escapeLabelValue, engine/src/admin/metrics.ts).

Failure behaviour

A missing or wrong bearer is a plain 401. A method other than GET is a 404, because the surface has no other verb. A genuine backend fault (the scheduler Durable Object is unreachable, or returns something the handler cannot parse) fails the scrape itself with a 500, rather than rendering a healthy-looking empty body: Prometheus’s own per-target up metric is the honest signal for that failure mode, and a fabricated “nothing configured” response would mask a real outage as an empty account. A successful scrape is recorded on the credential’s pull trail, the same best-effort, fire-and-forget recording every ingest credential keeps, so you can confirm from the console when the scrape last actually ran.

Where this fits

For the zero-agent alternative that pushes the same facts out instead of waiting to be scraped, read OTLP metrics push. For the credential mechanism this endpoint shares with vendor diagnostics and the audit feed, read owner-minted pull credentials. For the canonical, honest catalogue of the alert events the same underlying state also drives, read alert events. For the split-topology and Access-perimeter mechanics referenced above, read topology and wiring the audit feed into your SIEM.

Last updated 5 July 2026.