Prove a backup is recoverable: drills, blind restore tests, keyless attestation, and the canary
Restoring data writes plaintext back into your live account. Proving a backup is recoverable does not write anything to a real resource and does not reveal a single record’s contents. This page is for an auditor or recovery owner who needs to know exactly what assurance each verb gives, where its proof stops, and which one to reach for in which posture.
downpipes has four non-destructive ways to gain that assurance. Three of them act on your real archives and read only: the drill, the blind restore test and the keyless attestation. The fourth, the canary, is structurally different and proves something narrower. Conflating them is the most common mistake, so the contrast is drawn explicitly below.
None of these reveal record contents
The drill, the blind restore test, the keyless attestation and the canary all return counts, coarse reasons and one-way digests only. No branch of any of them ever returns or logs a record value. Only the destructive restore writes plaintext to a real resource.
The four verbs
Reach for a drill to spot-check a real run cheaply, a blind restore test to fully verify a real run without ever seeing plaintext, a keyless attestation when you need proof with no decryption key in play at all, and the canary for the automatic, always-on check that the machinery itself works.
| Verb | What it acts on | What it proves | Role floor |
|---|---|---|---|
| Drill | A sample of a real run | That a real run is reachable and decryptable, on the sampled records | Operator (the drill.run capability floor; a recovery rehearsal is an operator action) |
| Blind restore test | Every in-scope record of a real run | That the whole archive decrypts and each plaintext hash verifies, revealing no plaintext | Viewer (read-only verify) |
| Keyless attestation | A real run, with no key | Signature, completeness and anti-rollback, with no decryption key and no data | Viewer |
| Canary | A synthetic known corpus in an isolated cell | That the engine’s write, seal, read and restore machinery works | Run automatically on a cadence |
The drill
The drill opens a real run with the in-account read-back key, verifies the whole chain, then reads back a stride-sampled set of records and verifies each one in memory, writing nothing. It samples rather than reads everything: a small run is fully exercised, and a larger run has up to eight records read back at an even stride that always includes the first record, so a corruption anywhere in the archive (not just at the head) is exercised while the cost on your metered account stays bounded. The structural chain over every record was already recomputed when the run was opened.
What the drill proves is that a real run is reachable and decryptable, on the records it sampled. It also measures the wall-clock and the verified byte throughput of that read-back, which is the honest signal the RTO estimate is derived from. A drill needs Operator because of the drill.run capability floor: a recovery rehearsal is an operator action, not because it writes anything.
The blind restore test
The blind restore test is the strongest single-archive recoverability proof short of an actual apply. It opens and verifies the run, then decrypts every in-scope record to a discard sink: each record’s plaintext is materialised, its SHA-384 is checked against the signed record hash, the byte count is summed for throughput, and then the bytes are dropped. The plaintext is never returned, never logged and never written.
It returns the records and bytes it verified, a per-record list of any record that failed (so you see every bad record in one pass, not just the first), and a restoreDigest. The digest folds each verified record’s id together with that record’s plaintext hash, in a stable order, so the same archive restoring yields the same digest, while the digest input is only the per-record hash and never a plaintext byte. So a repeat test proves the same bytes restore without the digest ever becoming a confirmation oracle for a guessed value. The blind verify is read-only and open to any authenticated role, including Viewer.
The keyless Tier-0 attestation
The keyless attestation needs neither a decryption key nor any record data. It verifies the manifest signature against the operator-pinned signer, the shard-presence completeness, and the RUNLOG anti-rollback. Because it needs only the public verifier and read access to the destination, it runs even in the break-glass-only posture, where the engine holds no in-account read-back key. That makes it the strongest in-account recoverability claim available when there is nothing to decrypt with: you can still prove a run is signed, complete and not rolled back without any key in account.
It is an engine-side completeness and anti-rollback check. It is not a Cloudflare-side check, and it does not make an archive impossible to tamper with; it makes tampering detectable.
The canary, set apart
The canary is the verb people most often mistake for the others, so be precise about it. The canary writes a fixed, known synthetic corpus into an isolated _CANARY/runs/<runId>/ cell inside the destination, one subdirectory per flight, reads it back, decrypts every record, compares each byte against the known data, restores over the cell, and compares again. It never touches a customer archive or a real binding. A green canary proves the engine’s write, seal, read, decrypt and restore machinery works for that destination, byte for byte, on data whose loss does not matter.
What the canary does not prove is that any specific real run of yours is recoverable. It is a smoke alarm for the path, not a proof of a particular archive. For that, use a blind restore test or a keyless attestation against the real run. The canary has its own page at the canary.
What stamps “restorability last proven”
The console can show “restorability last proven” for a downpipe: who proved it, when, by which method, and the run id. That stamp is written by a passed manual blind restore test or a passed manual keyless attestation. A passed full attended verification, a person-run blind verification that supplies your real break-glass key, stamps it too. It is not written by the scheduled restore test. The scheduled test updates recency and an RTO sample, but it does not claim a fresh affirmative proof, so the “last proven” line never overstates what was actually run by a person.
| Action | Stamps “last proven” | Updates recency | Updates the RTO sample |
|---|---|---|---|
| Manual blind restore test (passed) | Yes | Yes | No |
| Manual keyless attestation (passed) | Yes | Yes | No |
| Attended verification, full (passed) | Yes | Yes | No |
| Attended verification, sampled (passed) | No | Yes, shown with its sample rate | No |
| Scheduled restore test (passed) | No | Yes | Yes, when it measured recoverable work |
Attended verification is the offline-key-only posture’s person-run proof: a full pass stamps “last proven” and a sampled pass updates recency alone, carried with its sample rate. See attended verification.
The scheduled restore test
A scheduled restore test runs per downpipe on a cadence, on by default at weekly. An explicit cadence is floored at sixty seconds in the engine so a malformed or abusive value cannot run a punishing loop on your metered account. Each run drives an in-account drill against the downpipe’s most recent run, updates the recency on the downpipe state, and contributes an RTO sample when it measured recoverable work. It emits a pass as an informational event and a genuine failure as a critical one, so a real integrity or freshness problem is loud while a healthy test is quiet.
Weekly is the default cadence, with a shorter daily option. You can also switch the scheduled test off, which stops only the proof: because the drill acts on an already-completed run rather than on the backup itself, turning it off pauses recoverability testing while backups keep running, and no scheduled test advances the restore-test recency until you re-enable it.
In the break-glass-only posture the engine has no in-account read-back key, so it cannot reopen a run it sealed earlier, which is what this scheduled test does. It still verifies each run as it seals it and still flies its hourly canary; neither needs this key. The scheduled test does not fake a pass and does not raise a false critical. It records an offline-rehearsal-required note and emits an informational event telling you to rehearse recovery offline with the break-glass key. A downpipe with no successful run yet has nothing to read back, which is recorded as an informational “no completed run” note rather than a failure.
Two independent proofs, not one to compare
The engine’s blind restore test and the offline CLI’s discard-sink restore both fold a per-record plaintext hash and are both deterministic and content-sensitive, but they are computed differently. The engine chains a re-hash over each record id and its plaintext hash in record-id order; the offline tool streams each destination key and plaintext hash through one running hash in iteration order. They are two independent recoverability proofs of the same data, not a shared value you are meant to compare across the two implementations. Use each as its own attestation.
How the verbs reach the destination and the keys
The drill and the blind restore test need the in-account read-back key (the operational private key), because they reopen a run the engine sealed earlier, and they honestly report the break-glass-only posture when it is absent rather than erroring. The canary no longer does: it seals its own test data and reads it back with that flight’s own key, so it runs in full in either posture. The keyless attestation needs only the public verifier derived from the signer the engine holds to sign runs, plus read access to the destination, which is why it survives the break-glass-only posture. None of the four ever consults the licence: recoverability assurance runs at the community tier. Reports and attestations are not verified in the browser; any cryptographic verification of them is done out of band, because the verifying key is not present in the console.
Where this fits
- The restore flow is the destructive path these verbs let you avoid running blind.
- The canary is the synthetic known-answer check described above, in full.
- Recovery postures explains the break-glass-only posture where keyless attestation is the strongest in-account claim.
- Reports covers how assurance evidence is packaged, and why full verification is out of band.
- Verify at seal is the read-back check that runs at backup time, complementing these recovery-time verbs.
Last updated .