D1 backup and restore in depth: the resumable record sequence, the consistent snapshot, and the fresh-database restore contract

D1 is the most involved source downpipes backs up, and this page is the deep dive into how it works on both halves: how a database is exported as a resumable, consistent sequence of bounded records, and how those records are replayed into a live database. It is written for developers who need to know what a D1 backup guarantees, what its limits are, and what the restore contract requires.

The short version is that a D1 database is captured as a resumable sequence of bounded records, not one whole-database value: one header record, many row-page records, then one schema record, each small enough to hold in memory on its own and checkpointed between so a crawl can resume after any one of them. The whole sequence is read through one consistent session so the tables do not tear under concurrent writes. Restore replays the sequence with bound parameters into a fresh database, atomic per batch rather than across the whole load. The rest of this page gives the detail for each.

A resumable sequence: header, row pages, schema

A D1 database is backed up as a resumable sequence of bounded records, not as one whole-database record sealed in a single invocation. The records land in strict order, with a checkpoint mark after every record, so a slice can stop and resume between any two of them (D1Source.crawlFrom, engine/src/sources/d1.ts):

1. One header record, body format downpipe-d1-header/1 (D1_HEADER_FORMAT, engine/src/sources/d1-format.ts): every table’s exact CREATE TABLE text and its ordered column names, no rows. Small by construction, DDL and column names only.
2. Many row-page records, body format downpipe-d1-rows/1 (D1_ROWS_FORMAT): one keyset-ordered page of one table’s rows per record, carrying the table name and its columns so each record is self-contained.
3. One schema record, body format downpipe-d1-schema/1 (D1_SCHEMA_FORMAT): the CREATE INDEX, CREATE TRIGGER and CREATE VIEW statements, applied last.

This is a SQL-level export, not the deprecated binary D1Database.dump(): a binary file cannot be replayed through the runtime binding because there is no load() call, whereas a header-plus-row-pages-plus-schema sequence can be re-created and re-inserted, which is what makes a real in-account restore possible.

Each record carries a fixed shape.

A reader that does not understand one of these three format strings refuses the body rather than mis-replaying it.

Cell values keep their SQLite type. A string, a number and null pass through as JSON; the two kinds JSON cannot hold directly are tagged. A BLOB becomes { "$blob": "<base64url>" }, and an integer outside the JSON-safe range becomes { "$int": "<decimal string>" } so a value above 2^53 is not lossily narrowed (D1Cell, cellFromValue, engine/src/sources/d1-format.ts). Each record carries its format string on its own body and on the record’s d1Format descriptor, so a restore can refuse a shape it cannot replay.

Bounded records, checkpointed between, not one buffered dump

Checkpointing and resume

Each record in the sequence is a complete, bounded Uint8Array, not a re-openable stream fed into the seal window by window: the header is small by construction, and each row page is bounded by its own byte guard. The schema is read first (small by construction, since it is data definition text and column names only, never rows), then each table is read in keyset-ordered pages, and a mark checkpoint follows every record, header, each row page, and the schema, so the sliced seal can stop after any record and a resumed slice picks up exactly where it left off, re-reading no row already emitted and never re-emitting the header (D1Source.crawlFrom, engine/src/sources/d1.ts).

A table with a rowid pages by its rowid, and the checkpoint mark carries the last rowid of the page. From engine 0.3.6, a WITHOUT ROWID table pages by its PRIMARY KEY in the same way. The mark carries the key values of the last row, so the table also resumes mid-table (pageByPrimaryKey, engine/src/sources/d1-reader.ts). The checkpoint wraps the mark with AES-256-GCM before it stores it, key values included. Up to engine 0.3.5, a WITHOUT ROWID table is read in one page and cannot resume mid-table. The run fails if that table’s rows exceed the 96 MiB page limit.

From engine 0.3.6, a row page whose last key encodes to more than 4 KiB gets no mark (D1_KEY_CURSOR_MAX_BYTES, engine/src/sources/d1-reader.ts). This keeps a large key value out of the checkpoint. The crawl reads on to the next page that ends on a smaller key, so a slice cannot stop between those two pages. A resumed table also continues the page numbers of its record names, so no name repeats one that an earlier slice sealed.

Sizing guards and byte limits

Size a D1 backup from this record sequence. A multi-gigabyte database is backed up rather than rejected, because no single record ever holds more than one keyset page of rows. The two enforced guards are the per-record byte cap and the run’s own segment and subrequest budget, the same as every other source.

The row page is byte-bounded, not a fixed row count: the keyset pager picks each page’s row LIMIT from the measured row width toward a target serialised size of D1_PAGE_TARGET_BYTES, 8 MiB, and never exceeds the D1_ROWS_PER_PAGE ceiling of 2000 rows. A narrow-row table ramps up to that ceiling (few subrequests); a wide-row table shrinks to a handful of rows per page so a page stays well under the 96 MiB per-page bound (engine/src/sources/d1-reader.ts).

A sequentially-consistent snapshot, on the production binding

D1 is the one source that gives a coherent cross-table snapshot, and it is a real guarantee with a stated condition. The whole export reads through one D1 Session anchored with withSession("first-primary") (makeSession, engine/src/sources/d1-reader.ts). The first query goes to the primary, which holds the newest committed state, and establishes a bookmark; every later read on that same session is constrained to a replica at or after that bookmark. That bookmark is threaded through the mark after every record (engine/src/sources/d1-token.ts), so a resumed slice re-opens the identical snapshot rather than a fresh one. The schema read and every table page, across every slice, therefore see one sequentially-consistent point-in-time snapshot, so a database under concurrent write load is exported torn-free rather than mixing pre-write and post-write rows across tables.

The snapshot is not unconditional in every environment. The cross-table snapshot holds on the production D1 binding, which supports withSession. On an older or local binding that does not expose withSession, the export degrades gracefully to a best-effort read against the bare database: the reads are still ordered keyset pages, but the single cross-table snapshot is then best-effort rather than sequentially consistent. The guarantee is “sequentially consistent on the production binding, best-effort without withSession”, never a perfect snapshot everywhere.

The selector does not narrow what is read within a single database. It selects which databases a downpipe covers, expressed by configuring one D1 source per database, so the record sequence for one D1 source always covers the whole of that database.

How tables are read

Each table is read in whichever way preserves both consistency and bounded memory.

A table with a rowid is read in keyset-ordered pages. The query is WHERE _rowid_ > CAST(?1 AS INTEGER) ORDER BY _rowid_ LIMIT ?2, carrying the last rowid as the cursor, so memory holds at most one page, which becomes one row-page record (pageTableRows, engine/src/sources/d1-reader.ts). The query selects an integer-safe projection of the table’s own columns. It adds a trailing rowid for the cursor and strips it before it emits the cells. The dumped columns are exactly the table’s declared columns, and an integer above 2^53 keeps every digit. The ORDER BY _rowid_ is the stable keyset order a resumed crawl reproduces, making the emitted rows deterministic across a fresh crawl and a resume.

From engine 0.3.6, a WITHOUT ROWID table is also read in keyset-ordered pages, by its PRIMARY KEY. The engine takes the table kind from the table’s CREATE TABLE text, not from a failed rowid read. The query is WHERE (k1, k2) > (?1, ?2) ORDER BY k1, k2 LIMIT ?3, with each key column compared under the collation of the key’s own index (pageByPrimaryKey, engine/src/sources/d1-reader.ts). The cursor is the key values of the last row, so the table resumes mid-table like a rowid table. If the engine cannot read the key columns, it reads that table in a single pass, as engine 0.3.5 does.

Up to engine 0.3.5, a table without a rowid is read in a single pass. A WITHOUT ROWID table has no rowid to page by, so the engine reads it with one query over all its rows and emits it as one row-page record. The read stays under the same session, but such a table cannot resume mid-table.

Internal tables are excluded from both backup and restore. Tables whose name starts with sqlite_ (SQLite internals such as the sequence and statistics tables, and the schema table itself) or _cf_ (D1’s own bookkeeping) are skipped (isInternalTable, engine/src/sources/d1-reader.ts). Re-creating or re-inserting these would either error or corrupt the managed database, so the backup never captures them and the restore never touches them.

Why no D1 record needs multi-segment sealing

A row page is capped at the 96 MiB D1_PAGE_BYTE_LIMIT by the reader’s own guard; the header and schema records carry no such enforced cap, but stay small by construction, since each holds only table/column names and DDL text, never row data. The seal’s chained multi-segment path only engages once a single record’s serialised size crosses MAX_STREAM_SEGMENT_BYTES, 1 GiB (engine/src/dest/types.ts), so no D1 record reaches that ceiling: the largest possible record, a full 96 MiB row page, is well under a tenth of the size that would trigger multi-segment sealing.

Restore: bound parameters, not interpolation

Restore decodes and shape-checks each record first, so a malformed or foreign body fails before any statement is built (decodeD1Record, engine/src/sources/d1-format.ts). decodeD1Record dispatches on the format string, so the header, rows and schema bodies are each shape-checked before anything is applied. The bytes have already passed the archive’s plaintext-hash check, so this is a shape guard rather than a trust boundary: it confirms the format string is one this reader understands, every table has a CREATE TABLE statement, every row has one cell per column, and every cell is a well-formed value.

Replay then re-creates the schema and re-inserts every row with parameterised statements (D1RestoreSink, engine/src/dest/restore-sink.ts). Table and column names are quoted identifiers (they cannot be bound as parameters), but every row value is a placeholder bound through bind(), never interpolated into SQL text. A tagged BLOB is bound as an ArrayBuffer (cellToBind, engine/src/sources/d1-format.ts). D1 refuses a bigint bind, so a tagged big integer is bound as its decimal string under a CAST(? AS INTEGER) placeholder (insertRows, engine/src/dest/restore-sink.ts). The value’s type survives, and the value can never become SQL.

No row value appears in any prepared SQL string, and every INSERT uses placeholder parameters only, so a value such as Robert'); DROP TABLE users;-- lands verbatim as data, never as a statement.

The replay runs in two phases, and the record kinds carry that order. The header record creates every table (and runs the once-only fresh-target check, below) before any row is processed; each rows record then appends into an already-created table; and only after the schema record, last in the sequence, are the non-table objects (indexes, triggers, views) applied, so a trigger never fires on a load row and an index is built once over the final data rather than maintained row by row (D1RestoreSink, engine/src/dest/restore-sink.ts).

Atomicity is per batch, not across the restore

This is the contract to internalise, because it shapes how you should run a D1 restore. D1 exposes one transaction primitive to a Worker, db.batch(), which runs its statements as a single implicit transaction. There is no cross-call begin and commit, so a whole restore is not one transaction. The sink splits a large set into bounded batches of D1_INSERT_BATCH, 50 statements, and each batch is individually atomic (runBatches, engine/src/dest/restore-sink.ts).

A schema with more than 50 tables, or a table larger than the batch size, therefore spans several batches. A fault part way through leaves a partially-loaded database. That is why the contract is to restore into a fresh, empty database, and why a write failure carries a specific reason: the failure reason for a D1 record says the target may be inconsistent and to drop it and retry into a fresh database, rather than the generic access error used for an idempotent KV or R2 key (engine/src/admin/restore-apply.ts).

Dry run by default, and it writes back in-account

A D1 restore is a dry run unless an apply is confirmed, and the dry run does real work: the sink in dry mode decodes and shape-checks each record’s body and writes nothing, so a malformed record that would not replay is caught at dry-run with zero writes rather than passing “dry-run clean” and failing on apply (engine/src/admin/restore.ts). A dry run leaves the target database untouched: no tables, no schema, no batches.

Unlike Secrets and Workers, whose restore is out of band, D1 writes back in account through the live database binding. The same reserved-binding guard that protects KV and R2 applies: a restore can never be pointed at a reserved binding such as the engine’s own signer key, and a reserved D1 target is refused with nothing written. The wider restore safety model, the apply’s dual-control approval, the verify-everything-before-writing discipline, and the freshness handling are shared across all sources. They are documented on the recovery pages rather than repeated here.

What is verified, and what verification means

The archive proves each record’s bytes are intact: restore re-verifies each record’s plaintext SHA-384 against the signed record hash before a byte is written. For a D1 record that proves the bytes are exactly what was sealed, but it does not by itself prove they decode into a replayable header, row-page or schema record, since the body shape is interpreted by the D1 sink and not by the archive format. That is why the dry-run path exercises the decoder: a body that hashes correctly but names a foreign format is caught before any write. Full verification of an archive end to end, including a recoverability proof, happens out of band of the browser. The recovery pages cover it.

Where this fits

These pages set the D1 deep dive in its wider context.

Last updated 25 September 2026.