Break-glass recovery with the offline open-source CLI
This page is about the second way to get your data back: offline, with the open-source downpipe reader and your own break-glass key, when the in-account engine cannot read your backups back for you. It is for the person who holds the recovery kit and needs to recover when the usual console path is unavailable. It frames when and why you reach for this channel, and points you at the CLI command reference for the exact commands rather than restating the whole walkthrough here.
Most of the time you never touch this. The normal restore is the console, the engine does the work, and a customer never opens a terminal for it. This channel is the floor underneath that: a recovery path that stays open even if the vendor and Cloudflare both disappear, because everything it needs is the destination bucket bytes, the offline keys from your recovery kit, and a reader you can rebuild from source.
When you need it
The recovery posture, set out in full in choosing your recovery posture, decides which read-back paths are open to you. In the strict break-glass-only posture, the engine holds no operational private key, so it cannot decrypt any archive on its own. A console restore drill in that posture does not fake a result. It reports honestly that there is no standing in-account read-back key, and without a key the console offers the keyless attestation as the strongest claim it can make. The console also carries an in-console break-glass restore panel: you supply your break-glass key in the browser, where it is read and never uploaded, so only the per-run key for the run you are recovering crosses to your own engine to write the data back. A split M-of-N key works there too, reassembled in the same browser flow. Keeping your private in the browser is defence in depth, not a custody requirement: your own engine never gains a standing key that could decrypt an archive on its own, which is the very posture a break-glass-only estate chose, and the vendor receives nothing whichever path you use.
There are two situations that bring you to the offline channel.
| Situation | Why the offline channel is the path |
|---|---|
| You run break-glass-only | By design the engine holds no standing key that decrypts an archive on its own. You can supply your break-glass key in the browser to drive an in-console break-glass restore, and this offline channel remains the read-back path that needs neither the vendor nor Cloudflare, so rehearse it on a schedule |
| The engine or console is unavailable | An account compromise, a lost engine, or any time you cannot trust the in-account path, you still recover from the bucket bytes plus your offline keys with nothing else in the loop |
In the two-recipient posture you usually restore in the console, because the engine holds an operational key that can read archives back and prove them recoverable on its own. Even then, an offline drill is the only test that proves rather than assumes the bucket and your offline key are enough by themselves, so it is worth rehearsing regardless of posture. See prove recoverability.
What the offline reader does
The reader is the downpipe Go command line. The same binary verifies a run, attests to it without any key, and restores it, all from a local copy of the bucket or straight from an S3-compatible endpoint, issuing reads only. The behaviour that matters for a safe recovery is that it never writes by surprise and never launders a bad record as a good one.
| Behaviour | What it means for you |
|---|---|
| Restore is a dry run by default | restore plans the writes and reports any conflict with state already present, and writes nothing until you pass --apply. So you read the plan before anything changes |
| It writes to a file or an env sink, not to live Cloudflare | A recovery materialises your records as files on disk, or as dotenv lines, for you to place back deliberately. The reader does not write to live KV or R2, which keeps the offline path simple and auditable |
| Every record is reassembled and hash-verified | On apply it reassembles each record’s value and checks it against its signed plaintext hash as it writes, and it refuses to overwrite an existing destination key |
| A bad record is reported, not hidden | A per-record reassemble, verify or write failure is recorded and the restore continues to the next record, then exits non-zero. A partial restore is never dressed up as a success |
Because the reader produces files or environment variables rather than touching your live resources, putting the recovered data back into KV, R2 or a worker is a separate, deliberate step you perform. For the boundary between what an archive can and cannot reconstruct, see what restore can and cannot write back.
Proving integrity with no in-account key
The offline channel also gives you an integrity proof when there is no in-account read-back key to rely on, which complements the engine’s own keyless attestation. Two read-only commands cover it without writing anything back. The verify command proves a run whole and genuine end to end against your break-glass identity and the signer public key. The attest command needs no identity and materialises no plaintext: it always checks the shard hashes and the structural completeness of the signed root without ever opening the encrypted data. Its signature and freshness checks depend on what you give it: with --signer, the signature is cryptographically verified and, if you also pass --min-runlog-index, so is freshness against a rolled-back RUNLOG; without --signer it is fully keyless, reports the signature as unchecked, and a rollback pin given without a signer is a structural, unauthenticated check rather than a cryptographic one. See the CLI command reference for the exact guarantee in each mode. Adding --check-bundle to verify (or restore) binds the in-bucket recovery instructions to their signed checksums, so the guidance you are following is itself proven genuine.
A clean verify exits zero; any non-zero exit names one specific verdict to act on. The normative meaning of every code is on exit codes and verification outcomes, so a scheduled drill script can react to the result rather than read prose.
Getting the reader
The quickest way to obtain the binary is go install github.com/downpipes/downpipe/cmd/downpipe@latest, which needs Go 1.26 or newer and a network connection to fetch the module. If you have no network access, or want the offline guarantee, build from a clone instead: the module is github.com/downpipes/downpipe, and every dependency is vendored into the repository so the build itself downloads no module and makes no network call. That offline-from-source property is the recovery promise made concrete: keep a copy of the repository alongside your recovery key, and you can rebuild the reader even if the vendor and Cloudflare are both gone, provided the recovery machine’s own Go already satisfies the pinned toolchain version. See install the tool truthfully for that pin and what to hold in advance so this is not a surprise mid-recovery.
The reader is MIT-licensed, and that is a different licence from the rest of the platform. The engine and the console are source-available under the Elastic License 2.0; the downpipe reader is genuinely open source under the MIT licence, so keeping a copy, rebuilding it, auditing it, or handing it to a third party who recovers on your behalf needs no permission from anyone. That is deliberate. The one component your recovery cannot proceed without is the one carrying the fewest strings.
Confirm the command set before you rely on the binary
go install .../downpipe@latest resolves to a complete reader today, tag v0.2.0, which carries prune, recombine and unseal-export. A tag-drift check in the downpipe repository now fails CI if a tag is ever cut without all three commands present. Even so, before a real recovery, run downpipe --help and confirm prune, recombine and unseal-export are all listed: a cached module resolution or an install done before a fix landed can hand you an old binary even when the tag itself is fine, and that check takes five seconds. Both go install and a plain from-source go build stamp the version as dev, since neither runs the maintainer’s release pipeline, so the version string will not tell you whether the command set is complete. The exact build commands are in install the tool truthfully.
The tool version is a separate thing from the archive format. The binary stamps its own build version, while the on-disk format carries its own semver identity, downpipe/0.1.0, and the two move independently. The reader accepts exactly the downpipe/0.1.x line, so an unstamped dev build opens the archives your engine writes.
What you need in hand
A break-glass recovery needs the destination bucket bytes (a local copy of the whole bucket tree, or read-only credentials for the S3-compatible endpoint), plus three pieces of information from your recovery kit. The first is identity.key, the break-glass private key, which is what decrypts your archives on this offline path and so must live offline and on a printed recovery sheet. The second is signer.pub, the operator signer public key the reader pins to confirm a run was signed by your writer. The third is the anti-rollback pin: the latest trusted RUNLOG index written on the recovery sheet’s blank anti-rollback line, passed to the reader as --min-runlog-index. Skipping the first two stops recovery outright; skipping the third does not, which is what makes it easy to leave out, and the whole point of the pin is to catch a bucket that has been rolled back to an older, validly-signed run. All three were written down at the key ceremony before any backup ran. For how that kit is generated and stored, see the key ceremony and recovery kit.
Where this fits
This page is the recovery-context pointer for the offline channel. For installation, every command, every flag and the canonical step-by-step recovery walkthrough, read the CLI command reference. For the meaning of each exit code a verification produces, read exit codes and verification outcomes. For the posture choice that decides whether this is your only read-back path, read choosing your recovery posture. For the normal in-account journey this sits beneath, read the restore flow.
Last updated .