Skip to content
downpipes docs

How sources survive a deploy, and why never a bare wrangler deploy

A binding-based backup source in downpipes is a Workers binding on the engine: a KV namespace, an R2 bucket, a D1 database or a Secrets Store secret. You attach these from the console, so they live on the running worker and are not written into the engine’s committed wrangler.toml. That difference is the whole subject of this page, because of how Cloudflare applies a deploy. The other source types (Cloudflare config, Workers scripts, Stream and Images) are read over the Cloudflare API using the engine’s read-only discovery token, not a Workers binding, so a deploy never touches them and this hazard does not apply to them.

wrangler deploy replaces the worker’s binding set with exactly what wrangler.toml lists. A deploy from a tree that does not name your console-attached sources therefore removes them, with no error and no prompt, and the next scheduled run of each dropped source fails with a source binding error. This is not a downpipes quirk; it is how the platform treats bindings, and it is why the engine ships its own deploy path.

The safe path is npm run deploy. It reconciles the live worker’s bindings before it deploys, so a code deploy or an upgrade can never silently drop a source. This page sets out the hazard precisely, what the reconcile does, the one flag that opts out of it and when that flag is appropriate, and how to recover if a source has already gone missing.

The hazard, stated precisely

The engine’s wrangler.toml declares no source bindings at all, on purpose. Pinning one source there would make that single source survive a bare wrangler deploy while every other console-attached source was dropped, which is the exact inconsistency the design refuses. Every source is treated identically: none in the committed config, all on the live worker.

So a working tree’s wrangler.toml and the engine’s live binding set diverge the moment you attach your first source from the console. After that:

  • A bare wrangler deploy ships only the bindings hand-listed in wrangler.toml, which is the engine’s own infrastructure (its scheduler and seal Durable Objects, the destination, its secrets) and nothing else. Every console-attached source is removed from the worker.
  • Each removed source then has a downpipe pointing at a binding that no longer exists. The Sources screen flags it as not attached, the preflight check fails and names it, and the next run fails with a source binding error.

Never run a bare wrangler deploy

A bare wrangler deploy (or wrangler deploy with no reconciled config) silently drops every source you attached from the console, because those sources live on the worker and not in wrangler.toml. It is not a supported way to deploy the engine. Always use npm run deploy, which reconciles the live bindings first. If you maintain your own pipeline, the reconcile step described below is mandatory in it.

What npm run deploy does instead

npm run deploy runs scripts/sync-bindings.mjs before it deploys. The reconcile reads the live worker’s bindings from the Cloudflare script-settings API and writes a generated wrangler.deploy.toml that is the committed wrangler.toml plus every live source binding it found. The deploy then runs against that superset config (wrangler deploy -c wrangler.deploy.toml), so the deployed worker keeps its code, its infrastructure and every attached source together.

The shape of the deploy is, in order:

cd engine
npm run deploy
# internally:
#   node scripts/sync-bindings.mjs           # reads live bindings, writes wrangler.deploy.toml
#   npx wrangler deploy -c wrangler.deploy.toml

Because the generated config is a strict superset of wrangler.toml, anything you do declare in the committed file is preserved as well. If you run the engine as infrastructure-as-code and prefer to list some sources directly in wrangler.toml, you may; the reconcile keeps those and simply adds any console-attached ones on top. You can also run the reconcile on its own at any time with npm run sync-bindings to inspect what it would carry forward.

The reconcile reads, it does not guess

The reconcile reads the live bindings from the same script-settings endpoint the console source-attach writes to. It re-sends each existing binding verbatim and adds the sources, so it never reconstructs a binding from assumptions. If a live source binding cannot be rendered for deploy, it refuses rather than drop it silently.

It refuses to deploy blind

The reconcile cannot prove your sources will survive unless it can read the live worker’s bindings. So when it cannot read them it exits non-zero, and because scripts/deploy.sh runs under set -e, the deploy stops there rather than ship a config that would drop your sources.

It has two ways to read them, and it does not need an API token for either. If CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID are both set it reads the script-settings API directly. If either is missing it falls back to your own wrangler session, running wrangler deployments status and wrangler versions view --json to read the active version’s bindings, which return the same binding shape. That fallback is what makes a deploy authenticated only by wrangler login, with no separate token minted at all, still binding-safe. It only refuses when the path it took genuinely failed.

Variable What it is How to get it
CLOUDFLARE_API_TOKEN A token that can read the engine worker’s settings. Optional: without it the reconcile uses your wrangler session instead. CF_API_TOKEN is accepted as an alias Create it from the dashboard “Edit Cloudflare Workers” template, scoped to the account that holds the engine worker
CLOUDFLARE_ACCOUNT_ID The id of that account. Needed only for the token path, since both must be set for it to be taken. CF_ACCOUNT_ID is accepted as an alias The account that holds the engine worker, not necessarily the one that holds your destination bucket

There is one deliberate exception to the refusal. The first time you ever deploy the engine there is no worker yet, so there are no live sources to preserve; the reconcile detects this (the settings read returns a 404, or wrangler reports the script as not found) and proceeds with wrangler.toml unchanged, saying so. A genuine first deploy therefore does not need the opt-out flag below, and a fresh deploy never dead-ends.

# First deploy from a machine with neither a token nor a wrangler session: the reconcile
# cannot read a worker that does not exist yet, so accept the empty reconcile explicitly.
cd engine
DOWNPIPE_ALLOW_BINDING_RESET=1 npm run deploy

DOWNPIPE_ALLOW_BINDING_RESET is not a routine flag

Setting DOWNPIPE_ALLOW_BINDING_RESET=1 tells the reconcile to deploy wrangler.toml as-is even when it cannot read the live bindings. Any console-attached source not also listed in wrangler.toml will be dropped. Use it only for a first deploy with nothing attached yet, or for a deliberate reset where you intend to drop the current bindings. It is never the fix for a deploy that is refusing because it could not read your live bindings; the fix there is to give it a way to read them, either by running wrangler login or by supplying CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID. The flag only takes effect on a read that genuinely failed, so it cannot drop a source that a successful read would have carried forward.

Why the console source-attach uses the same idea

Attaching a source from the console is the mirror image of the deploy reconcile, and it leans on the same safety property. When you attach a source, the engine rewrites its own live bindings through the Cloudflare script-settings API using a one-shot scoped token you paste in the console; the token is used for that one read-modify-write only, is never stored and is never logged.

The write is wrapped in a safety harness so it can never harm the running engine:

  • It reads the engine’s current bindings first and confirms they are genuinely this engine’s (both required Durable Objects are present), refusing on any unexpected shape.
  • It proves the new binding set strictly contains every existing binding, so the additions are the only difference, before it writes anything.
  • After the write it re-reads the bindings and post-verifies that nothing was dropped and the new source landed.
  • If that post-verify finds anything missing it raises a loud, named alarm that points at the redeploy recovery, rather than leaving a silent gap.

Rewriting the worker’s bindings makes Cloudflare deploy a new worker version, so the attach triggers a self-redeploy. That self-redeploy briefly resets the engine’s Durable Object, which is why the audit append and the credential-registry note after an attach are written best-effort with a retry: a reset must never fail an attach that has already landed and post-verified. The new binding takes effect on the next invocation, which the console detects by polling status.

Attaching a key is not the same as attaching a source

Installing the engine’s keys from the console writes Worker secrets, which take effect without a redeploy and do not reset the Durable Object. Only a binding change rewrites the worker’s settings and so triggers the self-redeploy described here. The two paths share the one-shot-token, never-stored discipline but differ in this respect.

Recovery when a source goes missing

If a deploy dropped a binding, or the underlying resource was deleted, the Sources screen shows that source as not attached under “Needs attention”, and the preflight “Source bindings” check fails and names it. Recover it like this.

  1. Confirm what is missing

    Open the Sources screen. Any configured source whose binding is no longer on the worker shows as not attached. The preflight “Source bindings” check names the same source, so you can confirm the cause is a dropped binding rather than a deleted resource.

  2. Re-attach from the console

    Use Re-attach on the Sources screen. The engine re-adds the binding to itself with a one-shot scoped token and post-verifies it survives, exactly as the original attach did. No command line is involved.

  3. Re-run npm run deploy

    Run npm run deploy from the engine directory so the next deploy preserves the re-attached source too. With CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID set, the reconcile reads the now-correct live bindings and carries them forward.

    cd engine
npm run deploy

  4. Verify the run lands

    Trigger a run or wait for the next scheduled one, and confirm it completes without a source binding error. The Sources screen should show the source attached again, and the preflight check should pass.

Why the vendor cannot do this for you

The reconcile runs on your machine, with your credentials, because that is the only place the credentials exist. The vendor holds no standing Cloudflare token and has no inbound path into your account, so there is nobody on the vendor side who could read your live bindings or deploy on your behalf. This is the no-custody model working as designed: the operator drives the reconcile because the operator is the only party who can.

That same property is why the console source-attach asks you to paste a one-shot token rather than using a stored one. The engine never keeps a Cloudflare credential, so each privileged operation collects the scope it needs for that one action and discards it.

Deeper detail: the reconcile's exact decisions

The reconcile derives the worker name from the name field in wrangler.toml (defaulting to downpipe-engine). On the token path it reads .../workers/scripts/{name}/settings, the same endpoint the console attach uses, so the read shape matches the write shape and re-sending existing bindings verbatim is exact. On the wrangler path it reads the active version through wrangler versions view --json, which returns the same binding shape.

Its decision table is small and fail-safe:

  • Both CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID set: read the live bindings over the script-settings API.
  • Either one missing: say so, and read the live bindings through your wrangler session instead. This is a normal path, not a degraded one, and it preserves sources exactly as the token path does.
  • The worker does not exist yet (HTTP 404 on the settings read, or wrangler reporting the script as not found): a first deploy with nothing to preserve, so write wrangler.toml as-is and proceed.
  • The read fails for another reason (an under-scoped token, or no wrangler login session): refuse and exit non-zero unless DOWNPIPE_ALLOW_BINDING_RESET=1, in which case it writes wrangler.toml as-is and proceeds, after noting plainly that any console-attached source not in it will be dropped.
  • The read succeeds: write wrangler.deploy.toml as the committed config plus every live source, log how many sources were carried forward and which a bare deploy would have dropped, and hand the path to wrangler deploy -c wrangler.deploy.toml.
  • The read succeeds but a live source binding cannot be rendered into the deploy config: exit non-zero and name it, rather than drop it silently. DOWNPIPE_ALLOW_BINDING_RESET does not override this one.

That parity holds for reading the bindings and for a pure detach: both need only the one Workers Scripts permission on the script-settings endpoint. Attaching a new source is not quite the same. The console attach also needs read access to that source’s own resource type (Workers KV, R2, D1 or Secrets Store), because Cloudflare will not let the engine bind a namespace, bucket, database or secret the token cannot itself see. The dashboard “Edit Cloudflare Workers” template bundles Workers Scripts, KV and R2, but it predates D1 and Secrets Store, so attaching a D1 or Secrets Store source needs that extra permission added to the token by hand. The reconcile’s read never needs it, since it only re-sends existing bindings verbatim rather than adding a new one.

Where this fits

This page is the canonical home for the deploy-binding safeguard. For the broader deploy and upgrade runbook that calls npm run deploy at each step, see deploy and upgrade the engine. For how a source is attached in the first place and what each source type needs, see connect a source. The custody reasoning that explains why no standing vendor token exists lives in the no-custody trust model.

Last updated .