Set up a downpipe with multiple destinations for failover

Attaching two or more destinations to a downpipe is how you get redundant off-source copies and failover. If the destination a run would normally seal to is down, the run still lands on a healthy one rather than failing outright. This page is the step-by-step task for an operator: add destinations, tick more than one on a downpipe, and understand what each destination receives and when.

The one expectation to set up front: copies fill in shortly after a run, not during it. A run seals to a single destination and then a scheduled pass replicates it to the others, so a freshly run fan-out downpipe legitimately shows fewer than its full copy count for a short while. That is normal and self-correcting.

Before you start

Add your destinations first. A downpipe can only fan out to destinations that already exist, and the console verifies each destination with a write probe at save time, so you cannot attach a destination that is not reachable and writable. Destinations are managed in one place (the Destinations screen), separately from downpipes, because a destination is an account-level archive target that any downpipe can use.

You need two or more destinations configured before the picker offers a choice. With a single destination there is nothing to pick, so the downpipe editor just shows where backups go and links you to Destinations to add another.

This is the real console; the write host has been replaced with example.downpipes.io and the key fields show placeholder examples, not live credentials.

Putting an R2 bucket in a different Cloudflare account

By default the R2 form offers the buckets discovery found in the account the engine runs in, and derives the write endpoint from that account, so there is no Account ID to type. That is the common case and it stays the default.

A second copy in the same Cloudflare account is not a second blast radius, so the form also lets you point a destination somewhere else. Tick This bucket is in a different Cloudflare account and two things change: an Account ID field appears, and the bucket stops being a picker and becomes a text box, because the list you were choosing from describes the engine’s account and not the one you are now naming.

Fill both. The Account ID is the 32-character id shown in the right-hand column of that account’s R2 page in the Cloudflare dashboard, and it becomes the endpoint host <account-id>.r2.cloudflarestorage.com; the bucket name is typed exactly as that account’s dashboard shows it. The access key pair must be an R2 API token minted in the same account, not in the engine’s. The form restates the endpoint the writes will actually go to as you type, so you can check it before saving.

Leaving the tick on with the Account ID empty is refused at the form. It never falls back to the engine’s account, because writing into the one account you have just said you did not want is the mistake the tick exists to prevent.

Naming a destination

Each destination carries a short name so you can tell your destinations apart. It shows up in the Destinations list and in the destination picker when you attach one to a downpipe, and it defaults to the bucket name if you leave it blank. The name is a label only and does not change where backups are written. You can change it later by editing the destination, but editing re-verifies the destination with a live write probe, so it asks for the access key pair again; because the secret is never shown back to you, mint a fresh key first if you did not keep it.

Attach two or more destinations

You can set the destinations either when you create a downpipe in the wizard, or later by editing an existing one. The control is the same in both places: a list of tickboxes, one per destination.

1. Open the destination picker

   In the new-downpipe wizard, the picker appears on the step where you name the downpipe and choose where it goes. In the editor, it is the “Destinations” field. Both are labelled so you can tick more than one for extra copies.

2. Tick two or more destinations

   The first destination you tick is the primary the run seals to. Each of the rest receives a copy of every run. A live summary line tells you what your current selection means (for example, “3 destinations: the first ticked is the primary; the rest each get a copy of every run”), and the default destination is labelled. After you save, the downpipe’s detail view marks the first ticked destination with a “primary” badge and each of the others with a “copy” badge.

3. Check the ordering

   Tick order is the selection order: the first ticked is index 0, the primary. When you edit an existing downpipe, the picker renders your current selection first, in its stored order, so an unrelated edit (a rename, a cadence change) never flips which destination is the primary.

4. Save

   On save, the console assembles the ticked destinations into an ordered list and sends it to the engine, which checks every id is a live destination before accepting the change. Leaving every box unticked clears any pin and falls back to the account default, a single copy.

What the badges mean

What is sent and stored

The form assembles a destinationIds array. Index 0 is the primary, and the remaining entries are the replicas. The wire shape looks like this when two destinations are ticked.

{
  "id": "kv-prod-backup",
  "name": "Production KV",
  "cadenceSeconds": 3600,
  "enabled": true,
  "source": { "type": "kv", "binding": "KV_prod", "include": [], "exclude": [] },
  "destinationIds": ["dest-1a2b3c4d", "dest-9f8e7d6c"]
}

A downpipe created before fan-out existed used a single destinationId field. Those legacy downpipes keep working unchanged. The engine resolves the primary from destinationIds when it is present, then falls back to the old destinationId, then to the account default, so nothing breaks and you can migrate a downpipe to fan-out simply by editing it and ticking a second destination.

How destination credentials are held

A destination needs a credential to write to your bucket, either an S3-style access key or a role to assume. That credential is yours and is stored in your own account, never with the vendor. Two engine settings tighten how it is held at rest.

Creating the destination credential

A plain S3-compatible destination authenticates with an access key id and a secret access key. You mint that pair in your storage provider’s own console, not in downpipes. For AWS S3 that is IAM: create (or reuse) an IAM user, then create an access key under its security credentials. For Cloudflare R2 reached over its S3-compatible endpoint it is an R2 API token. For another S3-compatible store, such as Backblaze B2, Wasabi or MinIO, it is that provider’s equivalent application-key or access-key screen. Whichever it is, the credential stays yours, in your own account.

Scope the key to the one archive bucket, and grant it the operations the engine actually performs against that bucket. On AWS these are the IAM actions below.

One trap is worth calling out. The save-time write probe exercises read, write and delete, so a key missing s3:ListBucket still passes the probe, then fails later when the replication sync or a prune needs to list the bucket. Grant list up front so a key that verifies at save keeps working on every later pass.

To assume a role you paste its Role ARN, the Amazon Resource Name of the IAM role the engine assumes, in the form arn:aws:iam::<account-id>:role/<role-name>. Copy it from that role’s summary page in the AWS IAM console; it is the role’s own ARN, not the principal user’s ARN named in the trust policy. When you assume a role, two more optional fields shape the request, both tied to the role. The External ID must match the sts:ExternalId condition in the role’s trust policy exactly; the match is case-sensitive at AWS. It is the cross-account guard against a confused-deputy call, so where a role’s policy sets an External ID the assume fails without the matching value. The Session duration is the lifetime of each credential the engine mints, a whole number of seconds from 900 to 43200 (15 minutes to 12 hours), default 3600. The console requires a whole number in that range, and because the engine re-mints per run a short duration is safe.

Building the AssumeRole role in AWS

The access key you store is no longer a write key under AssumeRole; it is a principal whose only job is to assume the role, and the temporary credentials do the writes. So you build an IAM role with two things. Its trust policy must let that principal assume it, meaning it trusts the principal user’s ARN for the sts:AssumeRole action, and where you set an External ID it requires the matching sts:ExternalId condition. Its permissions policy must carry the same S3 access on the archive bucket as a plain key does, the object read, write, delete and list from the table above, because the assumed session is what writes each run.

Two settings have to line up or the assume is refused. The region on the destination must be the role’s real AWS region, not R2’s auto, because the engine signs the STS call for that region. And the role’s own Maximum session duration, which AWS defaults to one hour, must be at least the Session duration you set here, or AWS rejects a longer request.

CONFIG_WRAP_KEY and STS AssumeRole are both engine settings on your self-hosted deployment. For tamper resistance at the destination itself, the object-lock and WORM story is in immutability and attestation.

Creating an Object-Lock bucket for WORM

If you set a WORM (write-once, read-many) policy on a destination, the engine writes each object with an object-lock retention until date, and at save time it reads the bucket’s Object-Lock configuration to confirm the bucket actually enforces it. A policy on a bucket that was not created with Object-Lock is silently ignored by the store, so the engine reports it as configured but not enforced rather than claiming a protection it is not applying.

The catch is that S3 Object-Lock can only be enabled when a bucket is created; it cannot be turned on afterwards. On AWS the enable-Object-Lock control is in the Create bucket flow (under the advanced or object-ownership settings), so you choose it at creation. If an existing bucket does not have it, the fix is to create a new bucket with Object-Lock enabled and point the destination at that. A console-set R2 destination is reached over the S3-compatible endpoint and probed the same way, so the engine detects Object-Lock enforcement on R2 identically when the bucket was created to enforce it. The full WORM model, including governance versus compliance mode, is in immutability and attestation.

Addressing style and storage class

Two optional S3 settings shape how the engine writes to an S3-compatible destination. Both are location config, not secrets, and both have a safe default, so you set them only when your provider needs it.

Addressing style chooses how the bucket appears in the request URL. Leave it on auto and the engine uses virtual-hosted addressing for AWS S3 and path-style addressing everywhere else, which is what almost every provider wants. Set path or vhost explicitly only when your S3-compatible provider requires one style: some on-premises and older gateways need path (the bucket in the URL path), while a few require vhost (the bucket in the hostname).

Storage class picks the S3 storage tier each object is written to. The engine writes only to the immediately readable tiers, because a restore has to read an object back without a retrieval delay:

Archival tiers such as Glacier and Deep Archive are deliberately refused: an object there cannot be read back without a restore-from-archive step first, which would make a recovery drill or a real restore fail at the moment you need it. If your retention policy calls for cold storage, use a bucket lifecycle rule to transition older objects after they are written, rather than writing them cold.

Validation and the removal guard

The engine bounds the configuration at its authority boundary, so a malformed or stale selection is refused rather than silently mishandled.

The endpoint cannot point inside your network

An S3-compatible destination takes an endpoint URL, and that field is the one place a destination could be aimed somewhere it has no business reaching. The engine screens the host before it does anything with it. An endpoint whose host is an internal, private, loopback or link-local address is refused with a 400 naming that reason, including the cloud metadata address that would otherwise be the interesting target.

The refusal happens before any request is made. It is checked ahead of the bucket and credential fields and ahead of the live write probe that verifies a destination, so nothing is stored, nothing is contacted, and a refused endpoint leaves no destination behind. The same message reaches you at the console form rather than only over the API.

The screen covers the spellings, not just the obvious one. Loopback, RFC1918 private ranges, carrier-grade NAT, the this-host range, localhost and any .localhost name, IPv6 loopback, unique-local and link-local, an internal IPv4 address smuggled in as an IPv4-mapped IPv6 literal, and the obfuscated decimal and hexadecimal spellings of an IPv4 address that a URL parser collapses back to the same host.

One limit is worth knowing. This screens the address as written. A hostname that looks ordinary and whose DNS resolves to an internal address is not caught here, because catching that needs the name resolved and pinned at the moment of the request rather than checked at save time. If you run split-horizon DNS, do not treat this screen as the thing standing between a mistyped endpoint and your internal network.

Why copies fill in after a run, not during it

This is the behaviour to expect and not be alarmed by. A run does not write to every destination at the same instant. It seals to one destination, that copy is verified at seal time, and only then does a separate scheduled pass copy the finalised run to the other destinations. The replication pass runs after the seal precisely so a replica copy never delays a backup, and it catches each destination up on its whole backlog of missing runs, not just the latest, so a run that landed while one destination was briefly down is still back-filled later.

The map’s “N of M copies” readout reflects this honestly. Right after a run, a fan-out downpipe can legitimately show one of two copies, then move to two of two once replication catches up. The number is derived from proven outcomes: a destination counts as holding a run only when its recorded state matches the latest successful run id, never from a guess based on how stale the run looks. A destination the engine could not reach shows as down, and a reachable destination that is simply behind shows as catching up.

Where this fits

Last updated 21 June 2026.