Skip to content
downpipes docs

Destination providers compared: R2, S3-compatible, Google Cloud and Azure Blob

Four stores can hold your archives, and most of what you configure is the same for all four. This page is the short list of what is not. It exists so the per-provider facts live in one row each, instead of being restated on every page that mentions a destination.

For the walkthrough of the form itself, read choosing a destination. For the operator task of adding and attaching several destinations, read multiple destinations.

The four providers, axis by axis

Each provider gets its own table below, so a long cell never shares a narrow column with three others. Read down one table for one provider, or compare the same row (Endpoint, Credential kinds, and so on) across the four.

Cloudflare R2

AxisWhat R2 does
EndpointDerived from your Cloudflare account id (<account-id>.r2.cloudflarestorage.com). The R2 branch of the form asks for a bucket rather than a host.
Credential kindsAn R2 API token, entered as an S3 access key id and secret.
Regionauto, and the R2 branch of the form does not ask.
Storage classNot asked for on the R2 branch of the form.
AssumeRoleNot offered. R2 has no STS.
ImmutabilityNone, on any bucket, by any path. Over its S3 endpoint, the Object-Lock probe answers 404, and a write carrying x-amz-object-lock-mode: COMPLIANCE answers 501 NotImplemented naming that header. Saving an R2 destination with a mode set is refused.
Offline readerYes, over --s3-endpoint.

S3-compatible

AxisWhat an S3-compatible store does
EndpointYou type the endpoint URL, the bucket and the region. Covers AWS S3, Backblaze B2, Wasabi, MinIO and any other S3-compatible store.
Credential kindsAn access key id and a secret access key, or an STS role to assume.
RegionHonoured, and it must be the bucket’s real region. Under AssumeRole auto is refused, because the STS endpoint is regional.
Storage classSTANDARD, STANDARD_IA, INTELLIGENT_TIERING and ONEZONE_IA. Glacier and Deep Archive are not offered, because an archive you cannot read back without a retrieval step fails you on the day you need it.
AssumeRoleYes, on AWS: a role ARN, an optional external id, and a session duration from 900 to 43200 seconds.
ImmutabilityS3 Object Lock, in governance or compliance mode. The bucket must have been created with Object Lock enabled, which cannot be turned on afterwards. A bucket default retention rule also protects each archive, with or without a mode set here.
Offline readerYes, over --s3-endpoint.

Google Cloud Storage

AxisWhat Google Cloud Storage does
EndpointFixed at https://storage.googleapis.com and shown read-only. Google publishes one S3-interop host and it is neither per-bucket nor per-region.
Credential kindsAn HMAC interoperability key pair for a service account, created in Google Cloud under Cloud Storage settings. A service-account JSON key is a different kind of credential and does not work here.
RegionIgnored. Google leaves the region out of the request signature, so auto is correct and any other value is accepted and disregarded.
Storage classRefused by name. Google’s class names are not Amazon’s and downpipes translates between neither vocabulary. Set the class on the bucket in Google Cloud.
AssumeRoleRefused by name. Google has no STS equivalent, so a role ARN would never be assumed.
ImmutabilityS3 Object Lock over the interop API. The bucket must have been created with per-object retention, which cannot be added later.
Offline readerYes, over --s3-endpoint https://storage.googleapis.com, with the same HMAC pair in AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

Azure Blob Storage

AxisWhat Azure Blob Storage does
Endpointhttps://<account>.blob.core.windows.net, whose first label is the storage account. The US Government (core.usgovcloudapi.net) and China (core.chinacloudapi.cn) suffixes are matched and routed to the Azure client too. It must be https. The Data Lake Storage Gen2 dfs host is refused by name and points you at the same account’s blob host.
Credential kindsThree kinds. A Shared Key is the storage account name plus one of its access keys, and never expires. A SAS token goes where the access key would and is recognised by its structure, with its se expiry read out of it. An Entra service principal authenticates a tenant and application against Microsoft’s identity platform, and is refused on the US Government and China clouds.
RegionUnused. No Azure request carries a region, so a value sent through the API has no effect. The console form hides the box.
Storage classRefused by name. Azure has access tiers (Hot, Cool, Cold and Archive) rather than Amazon’s class names. Set the tier on the container or the storage account in Azure.
AssumeRoleRefused by name. Azure has no STS equivalent, so a role ARN would never be assumed.
ImmutabilityAzure version-level immutability, sent per blob. governance maps to an unlocked policy and compliance to a locked one. The container or the storage account must have version-level immutability enabled, which also needs blob versioning on the account.
Offline readerYes, over --azure-endpoint with --azure-container, which is a second flag pair rather than another value of --s3-endpoint. The credential comes from AZURE_STORAGE_KEY or AZURE_STORAGE_SAS_TOKEN in the environment. prune --apply is the one operation the reader will not perform there.

Each destination also carries a provider badge in the console, derived from the endpoint host, not the button you pressed: R2, S3, GCS or Azure.

What immutability means on each store

The four rows above are not four spellings of one setting, and reading them as one is how an over-credited claim gets made.

On an S3-compatible store and on Google Cloud Storage the mechanism is S3 Object Lock. The engine reads the bucket’s own configuration with a live probe before it credits anything. Google enforces Object Lock through its interoperable API; it does not accept the header and ignore it: a bucket created with per-object retention answers the probe with Object Lock enabled, carries a real retain-until instant on a compliance write, and refuses a delete inside the window.

On Cloudflare R2 there is nothing to reach. R2’s S3 API does not implement the Object-Lock configuration calls, and its CreateBucket rejects the object-lock-enabled header, so no R2 bucket enforces it and none can be made to. Reaching R2 over its S3-compatible endpoint is not a way round that: the same endpoint answers a compliance write with 501 and the name of the header it will not implement.

On Azure the mechanism is Azure’s own, and the mapping is exact rather than a guess. Azure carries two primitives: an immutability policy that holds a blob until a retain-until date and is separately either unlocked or locked, and a legal hold that holds a blob with no expiry until an administrator clears it. The form collects one mode plus one window, which is the policy and not the hold. The two vocabularies then name the same two guarantees.

governance (unlocked Azure policy) means a sufficiently privileged principal can lift the retention; compliance (locked) means nobody can shorten or remove it for the window. Nothing is inferred from the container, because the mode rides on the write as x-ms-immutability-policy-mode with x-ms-immutability-policy-until-date beside it, so compliance sends locked and either gets the strong guarantee or the write is refused. The legal hold is never set: deriving an unexpiring hold from a retention-days answer would invent a promise you never made.

The precondition is Azure’s. A per-blob policy only binds on a container with version-level immutability enabled, so the engine probes for it with one credentialed Get Container Properties read and takes the answer from x-ms-immutable-storage-with-versioning-enabled. A container without it is refused at save, rather than stored and then failing every backup write.

A bucket default retention rule on AWS S3

An AWS S3 bucket can carry an Object Lock default retention rule. The rule applies its window to every object written to the bucket. A destination with no mode set can write to that bucket, and it gets store-enforced retention there.

AWS refuses every write into such a bucket that carries no integrity header. The engine sends the x-amz-checksum-sha256 header, inside the request signature. A destination with a mode set sends it on each object it writes in a single request. A destination with no mode set sends it after AWS refuses a write for the lack of it. It then sends that write again once.

When you save the destination in the console, the save-time write probe meets that refusal first. The engine then records on the destination that the bucket requires a checksum. Every later write then carries the header from the start. Two cases have no such record: a rule added to the bucket after the last save, and a destination set with the deploy-time DEST_* variables. There, each time the engine opens the destination (each slice of a backup run, each canary cycle, each audit-log push), its first write meets the refusal once. The support pack counts each of those refused writes as dest-io-checksum-retry.

A large archive goes up as a multipart upload. For a destination that requires a checksum, the engine creates each upload with the SHA-256 checksum algorithm, and each part carries its own checksum.

The engine does not send the header to other stores because of an Object Lock reading alone. Backblaze B2 reports Object Lock over its S3-compatible API. A B2 destination gets the header only with a mode set here, or after B2 sends the AWS refusal. Cloudflare R2 has no Object Lock, and its S3 API reference lists Content-MD5 and no x-amz-checksum-* header for PutObject.

The offline reader reaches all four

The downpipe Go reader is the break-glass path that needs neither the vendor nor Cloudflare. It reads a local copy of the bucket tree, or the destination itself over one of two network backends. Between them, those two cover every provider in the table above.

--s3-endpoint with --s3-bucket reaches R2, an S3-compatible store and Google Cloud Storage, all signed from AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Google Cloud Storage is reached as an ordinary S3 endpoint (--s3-endpoint https://storage.googleapis.com) with the same HMAC interoperability pair the destination uses.

--azure-endpoint with --azure-container reaches Azure Blob Storage. It is a second pair rather than another value of the first because Azure is not an S3-compatible store behind a different host: it is a different wire protocol, with its own authentication scheme, its own request shape and its own listing document, so no value of --s3-endpoint opens an Azure container. Pass both halves of the pair, since an endpoint with no container is a usage error rather than a guess, and do not name an --s3-endpoint and an --azure-endpoint in the same command: two destinations at once is refused outright rather than resolved by precedence, because an operator who has given both has one of them wrong.

The Azure credential is read from the environment and never from a flag, so it reaches neither a shell history nor a process listing. Set AZURE_STORAGE_KEY (a storage account access key) or AZURE_STORAGE_SAS_TOKEN (a shared access signature), and set exactly one of the two. The storage account name is taken from the endpoint’s first label; set AZURE_STORAGE_ACCOUNT as well only when it is not, which is the case for a custom domain in front of the account. The two credential kinds are interchangeable for reading: the same run restored under a SAS and under an account key produces the same bytes.

downpipe restore --apply --azure-endpoint https://<account>.blob.core.windows.net \
    --azure-container <container> --run <runId> \
    --identity ./keys/identity.key --signer ./keys/signer.pub \
    --min-runlog-index <n> --out ./restored

One thing the reader will not do on Azure is delete. Its Azure backend implements the reads a recovery needs, which are the buffered fetch, the streamed fetch, the existence check and the listing, and it implements no delete at all, so prune --apply against an Azure container refuses on the capability it cannot find rather than on a check somebody has to remember to write. prune without --apply still plans and reports. Retention on an Azure destination stays the engine’s job. A break-glass tool that can delete from a destination is one that can destroy the last copy of the data it exists to recover.

Holding a second copy on another provider is still the 3-2-1 pattern you likely want. The reasons have nothing to do with which flags the reader takes.

Where each setting is made

The console’s destination form asks only what the chosen provider can honour, so a field the save is certain to refuse is hidden rather than left as a trap. The R2 branch hides the whole endpoint block, which is where the region and the storage class live. Google Cloud and Azure keep that block and hide the storage class and the AssumeRole section. On Google Cloud the region box is shown and is inert. On Azure the form also hides the region box and the addressing style.

Every Azure capability the engine has is on the console form. All three credential kinds and the immutability policy are set from the console, and none of them needs the API.

  • A Shared Key is the default. The storage account name goes in the first credential box and one of the account’s access keys in the second.
  • A SAS token needs no separate control. Paste it where the access key goes: the engine recognises it by its structure, requiring both a parameter separator and a sig parameter, so an account key can never be mistaken for one. Its se expiry is read at save, and a token that has already expired is refused there rather than failing on the first backup.
  • An Entra service principal has its own collapsed block below the credentials, asking for the Directory (tenant) ID and the Application (client) ID. Both are public identifiers. The client secret goes in the same credential box, because a destination carries exactly one secret and the client secret is it. Fill both ids or neither: a half-filled principal is refused, at the form and again by the engine, rather than being quietly dropped.
  • An immutability policy is offered for all four providers. Which of them can enforce it is decided by a live probe against your own bucket or container at save, not by the provider’s name.

The engine’s destinations API (POST /admin/destinations, see admin endpoints) accepts the same immutability policy and service principal as the form.

Last updated .