D1 backup and restore in depth: the resumable record sequence, the consistent snapshot, and the fresh-database restore contract

D1 is the most involved source downpipes backs up, and this page is the deep dive into how it works on both halves: how a database is exported as a resumable, consistent sequence of bounded records, and how those records are replayed into a live database. It is written for a developer who needs to know precisely what a D1 backup guarantees, what its limits are, and what the restore contract obliges them to do. Every figure here comes from the source, not from any older note.

The short version is that a D1 database is captured as a resumable sequence of bounded records, not one whole-database value: one header record, many row-page records, then one schema record, each small enough to hold in memory on its own and checkpointed between so a crawl can resume after any one of them. The whole sequence is read through one consistent session so the tables do not tear under concurrent writes. Restore replays the sequence with bound parameters into a fresh database, atomic per batch rather than across the whole load. The rest of this page makes each of those claims exact.

A resumable sequence: header, row pages, schema

A D1 database is backed up as a resumable sequence of bounded records, not as one whole-database record sealed in a single invocation, in a strict order with a checkpoint mark after every record so a slice can stop and resume between any two of them (D1Source.crawlFrom, engine/src/sources/d1.ts):

1. One header record, body format downpipe-d1-header/1 (D1_HEADER_FORMAT, engine/src/sources/d1-format.ts): every table’s exact CREATE TABLE text and its ordered column names, no rows. Small by construction, DDL and column names only.
2. Many row-page records, body format downpipe-d1-rows/1 (D1_ROWS_FORMAT): one keyset-ordered page of one table’s rows per record, carrying the table name and its columns so each record is self-contained.
3. One schema record, body format downpipe-d1-schema/1 (D1_SCHEMA_FORMAT): the CREATE INDEX, CREATE TRIGGER and CREATE VIEW statements, applied last.

This is a SQL-level export, not the deprecated binary D1Database.dump(): a binary file cannot be replayed through the runtime binding because there is no load() call, whereas a header-plus-row-pages-plus-schema sequence can be re-created and re-inserted, which is what makes a real in-account restore possible.

Each record carries a fixed shape.

A reader that does not understand one of these three format strings, or the legacy downpipe-d1-json/1 described below, refuses the body rather than mis-replaying it.

Cell values keep their SQLite type. A string, a number and null pass through as JSON; the two kinds JSON cannot hold directly are tagged. A BLOB becomes { "$blob": "<base64url>" }, and an integer outside the JSON-safe range becomes { "$int": "<decimal string>" } so a value above 2^53 is not lossily narrowed (D1Cell, cellFromValue, engine/src/sources/d1-format.ts). Each record carries its format string on its own body and on the record’s d1Format descriptor, so a restore can refuse a shape it cannot replay.

Bounded records, checkpointed between, not one buffered dump

Checkpointing and resume

Each record in the sequence is a complete, bounded Uint8Array, not a re-openable stream fed into the seal window by window: the header is small by construction, and each row page is bounded by its own byte guard. The schema is read first (small by construction, since it is data definition text and column names only, never rows), then each table is read in keyset-ordered pages, and a mark checkpoint follows every record, header, each row page, and the schema, so the sliced seal can stop after any record and a resumed slice picks up exactly where it left off, re-reading no row already emitted and never re-emitting the header (D1Source.crawlFrom, engine/src/sources/d1.ts).

Sizing guards and byte limits

Author the sizing from this record sequence, not from a buffered whole-body figure. A multi-gigabyte database is backed up rather than rejected: the former whole-body reject no longer applies, because no single record ever holds more than one keyset page of rows. The two enforced guards are the per-record byte cap and the run’s own segment and subrequest budget, the same as every other source.

The row page is byte-bounded, not a fixed row count: the keyset pager picks each page’s row LIMIT from the measured row width toward a target serialised size of D1_PAGE_TARGET_BYTES, 8 MiB, and never exceeds the D1_ROWS_PER_PAGE ceiling of 2000 rows. A narrow-row table ramps up to that ceiling (few subrequests); a wide-row table shrinks to a handful of rows per page so a page stays well under the 96 MiB per-page bound (engine/src/sources/d1-reader.ts).

A sequentially-consistent snapshot, on the production binding

D1 is the one source that gives a coherent cross-table snapshot, and it is a real guarantee with a stated condition. The whole export reads through one D1 Session anchored with withSession("first-primary") (makeSession, engine/src/sources/d1-reader.ts). The first query goes to the primary, which holds the newest committed state, and establishes a bookmark; every later read on that same session is constrained to a replica at or after that bookmark. That bookmark is threaded through the mark after every record (engine/src/sources/d1-token.ts), so a resumed slice re-opens the identical snapshot rather than a fresh one. The schema read and every table page, across every slice, therefore see one sequentially-consistent point-in-time snapshot, so a database under concurrent write load is exported torn-free rather than mixing pre-write and post-write rows across tables.

State the condition plainly, because the snapshot is not unconditional in every environment. The cross-table snapshot holds on the production D1 binding, which supports withSession. On an older or local binding that does not expose withSession, the export degrades gracefully to a best-effort read against the bare database: the reads are still ordered keyset pages, but the single cross-table snapshot is then best-effort rather than sequentially consistent. The guarantee is “sequentially consistent on the production binding, best-effort without withSession”, never a perfect snapshot everywhere.

The selector does not narrow what is read within a single database. It selects which databases a downpipe covers, expressed by configuring one D1 source per database, so the record sequence for one D1 source always covers the whole of that database.

How tables are read

Each table is read in whichever way preserves both consistency and bounded memory.

A table with a rowid is read in keyset-ordered pages. The query is WHERE _rowid_ > ?1 ORDER BY _rowid_ LIMIT ?2, carrying the last rowid as the cursor, so memory holds at most one page, which becomes one row-page record (pageTableRows, engine/src/sources/d1-reader.ts). The selected row shape is the table’s own columns plus a trailing rowid used only as the cursor and stripped before the cells are emitted, so the dumped columns are byte-identical to a plain SELECT *. The ORDER BY _rowid_ is the stable keyset order a resumed crawl reproduces, which is what makes the emitted rows deterministic across a fresh crawl and a resume.

A table without a rowid is read in a single pass. A WITHOUT ROWID table has no rowid to page by, so it is read with one SELECT * and emitted as one row-page record, still under the same session and still byte-identical; these tables are uncommon and small in practice (lookup and configuration tables), and such a table cannot resume mid-table.

Internal tables are excluded from both backup and restore. Tables whose name starts with sqlite_ (SQLite internals such as the sequence and statistics tables, and the schema table itself) or _cf_ (D1’s own bookkeeping) are skipped (isInternalTable, engine/src/sources/d1-reader.ts). Re-creating or re-inserting these would either error or corrupt the managed database, so the backup never captures them and the restore never touches them.

The legacy whole-database encoder: test-only now, and why no D1 record needs multi-segment sealing

encodeD1Backup and its streaming counterpart encodeD1BackupStream (engine/src/sources/d1-format.ts) build the single whole-database downpipe-d1-json/1 body: the format the live crawl produced before resumability. Neither has a production caller today. Grepping the source tree turns up only d1-format.ts itself and the test files engine/test/validate-restore.ts and engine/test/validate-d1-restore.ts; D1Source.crawlFrom never calls either. decodeD1Record still accepts a downpipe-d1-json/1 body on restore (the full record kind), so an archive written before resumability keeps restoring unchanged, but the live crawl now produces only the header/rows/schema sequence described above.

The streaming form still matters as a reference check: encodeD1BackupStream hand-serialises the same JSON shape JSON.stringify produces for the typed body, with the same key order, escaping each cell with JSON.stringify (which is context-free, so a value stringified alone is byte-identical to the same value inside the parent object). The validator pins it byte-for-byte against the buffered encodeD1Backup over the same content (engine/test/validate-d1-restore.ts), which is what keeps the legacy decoder honest, not a claim about what the live crawl emits.

A row page is capped at the 96 MiB D1_PAGE_BYTE_LIMIT by the reader’s own guard; the header and schema records carry no such enforced cap, but stay small by construction, since each holds only table/column names and DDL text, never row data. The seal’s chained multi-segment path only engages once a single record’s serialised size crosses MAX_STREAM_SEGMENT_BYTES, 1 GiB (engine/src/dest/types.ts), so no D1 record reaches that ceiling today: the largest possible record, a full 96 MiB row page, is well under a tenth of the size that would trigger multi-segment sealing.

Restore: bound parameters, not interpolation

Restore decodes and shape-checks each record first, so a malformed or foreign body fails before any statement is built (decodeD1Record, engine/src/sources/d1-format.ts). decodeD1Record dispatches on the format string, so the legacy whole-database body and the three resumable header/rows/schema bodies are all shape-checked before anything is applied. The bytes have already passed the archive’s plaintext-hash check, so this is a shape guard rather than a trust boundary: it confirms the format string is one this reader understands, every table has a CREATE TABLE statement, every row has one cell per column, and every cell is a well-formed value.

Replay then re-creates the schema and re-inserts every row with parameterised statements (D1RestoreSink, engine/src/dest/restore-sink.ts). Table and column names are quoted identifiers (they cannot be bound as parameters), but every row value is a placeholder bound through bind(), never interpolated into SQL text. A tagged BLOB is bound as an ArrayBuffer and a tagged big integer as a bigint, so the value’s type survives and the value can never become SQL (cellToBind, engine/src/sources/d1-format.ts).

The validator proves this directly. It seeds a row whose value is the string Robert'); DROP TABLE users;-- and a BLOB of raw bytes, restores the database, and asserts that no row value appears in any prepared SQL string and that every INSERT uses placeholder parameters only (engine/test/validate-d1-restore.ts). A hostile value lands verbatim as data, never as a statement.

The replay runs in two phases for a reason, and with the resumable sequence that order rides the record kinds themselves. The header record creates every table (and runs the once-only fresh-target check, below) before any row is processed; each rows record then appends into an already-created table; and only after the schema record, last in the sequence, are the non-table objects (indexes, triggers, views) applied, so a trigger never fires on a load row and an index is built once over the final data rather than maintained row by row. The legacy whole-body apply runs the same two phases inside one record, byte-behaviour unchanged, so an archive written before resumability restores the same way it always did (D1RestoreSink, engine/src/dest/restore-sink.ts).

Atomicity is per batch, not across the restore

This is the contract to internalise, because it shapes how you should run a D1 restore. D1 exposes one transaction primitive to a Worker, db.batch(), which runs its statements as a single implicit transaction. There is no cross-call begin and commit, so a whole restore is not one transaction. The sink splits a large set into bounded batches of D1_INSERT_BATCH, 50 statements, and each batch is individually atomic (runBatches, engine/src/dest/restore-sink.ts).

A schema with more than 50 tables, or a table larger than the batch size, therefore spans several batches, and a fault part way through leaves a partially-loaded database. That is why the contract is to restore into a fresh, empty database, and why a write failure is reported so plainly: the failure reason for a D1 record says the target may be inconsistent and to drop it and retry into a fresh database, rather than the generic access error used for an idempotent KV or R2 key (engine/src/admin/restore.ts).

Dry run by default, and it writes back in-account

A D1 restore is a dry run unless an apply is confirmed, and the dry run does real work: the sink in dry mode decodes and shape-checks each record’s body and writes nothing, so a malformed record that would not replay is caught at dry-run with zero writes rather than passing “dry-run clean” and failing on apply (engine/src/admin/restore.ts). The validator confirms a dry run leaves the target database untouched: no tables, no schema, no batches (engine/test/validate-d1-restore.ts).

Unlike Secrets and Workers, whose restore is out of band, D1 writes back in account through the live database binding. The same reserved-binding guard that protects KV and R2 applies: a restore can never be pointed at a reserved binding such as the engine’s own signer key, and the validator proves a reserved D1 target is refused with nothing written (engine/test/validate-d1-restore.ts). The wider restore safety model, the dual-control approval an apply requires, the verify-everything-before-writing discipline, and the freshness handling, is shared across all sources and is documented on the recovery pages rather than repeated here.

What is verified, and what verification means

The archive proves each record’s bytes are intact: restore re-verifies each record’s plaintext SHA-384 against the signed record hash before a byte is written. For a D1 record that proves the bytes are exactly what was sealed, but it does not by itself prove they decode into a replayable header, row-page or schema record, since the body shape is interpreted by the D1 sink and not by the archive format. That is precisely why the dry-run path exercises the decoder: a body that hashes correctly but names a foreign format is caught before any write. Full verification of an archive end to end, including a recoverability proof, is covered on the recovery pages and happens out of band of the browser.

Where this fits

These pages set the D1 deep dive in its wider context.

Last updated 3 July 2026.