Scale and limits: sliced runs, the slice budget, and the ceilings

The engine runs inside a Cloudflare Worker, and a Worker invocation has finite budgets: a cap on outbound subrequests and a ceiling on CPU and wall-clock time. A backup that cannot finish inside one invocation is not rejected and does not silently truncate. It is split into slices that each finalise within a per-invocation budget, and the run is carried across invocations on a durable-alarm chain. This page is for an operator or developer who needs the concrete figures: how big one slice gets, what one object can be, and how storage grows as runs accumulate.

It is the companion to the engine architecture, which owns the runtime model and the dispatch floor; this page owns the numbers. Every figure here is pinned to a named constant in the engine source, so prefer it over any prose that quotes a different value.

Sliced runs: how a large backup crosses invocations

A backup run is the crawl of a source, the encryption of every record, and the write of an immutable archive to your destination. A small run completes inside the cron tick’s first inline slice, never wraps a per-run key, and behaves exactly like a single-pass seal. A run that cannot complete in that one invocation is handed to a per-downpipe Durable Object, RunSealDO, which advances it one slice at a time.

The mechanics that make this safe to do are worth stating plainly.

Each alarm invocation of RunSealDO gets a fresh platform subrequest budget. That is the whole reason a large run is chained through alarms rather than looped inside one invocation that would hit the cap. On every alarm the object heartbeats the scheduler’s in-flight lease (so a long run is never reclaimed as crashed mid-flight), seals one budget’s worth of work, persists the advanced checkpoint and that slice’s output in one atomic write, then re-arms the next alarm. When the source is exhausted and there is headroom left, that same invocation finalises the run: it writes the root manifest, the signature and the recovery bundle, appends to the account-wide RUNLOG, reads the archive back to verify it, and posts the completion.

Two correctness properties hold throughout. The alarms are durable, so a slice that crashes resumes from its last checkpoint rather than restarting the whole run, and content addressing makes the re-seal idempotent (a segment already written is detected and skipped). The destination chosen when the run started is pinned for the life of that run, so a mid-run destination change can never split one archive across two stores.

The slice budget

SliceBudget (engine/src/seal/budget.ts) bounds one invocation’s share of the platform limits. The two limits that actually kill a seal are the per-invocation subrequest cap (every KV get, R2 get and put, S3 fetch and Durable Object fetch counts toward it) and the CPU and wall-clock. The budget is spent by the sources, the destination and the pipeline as they work; when it runs low the slice checkpoints between records and hands the run to the next invocation.

The defaults are deliberately conservative. The paid-plan subrequest cap is 1000, and a cron invocation also spends on the scheduler Durable Object, the notification passes and any other due downpipes, so a slice never assumes it owns the whole cap. The 700 default leaves room for that sharing and for the finalise reserve.

The reserve is not a guessed number. It is derived from the worst-case sub-terms of the actual finalisation code paths, so that editing a retry count cannot silently break the arithmetic: the final flush, the root manifest, the signature and four recovery-bundle objects, each destination put retried up to three attempts (21); the RUNLOG append loop of up to six contended read-then-put rounds plus the re-read and the signature put (14); the RUNLOG lock acquire-and-release attempts (4); and the completion fetch (1). That sums to 40. The slice yields as soon as the remaining budget falls to the reserve or the wall-clock is consumed, and it checks that boundary only between records, never mid-record, so a record’s segments and its manifest line are always committed together.

Tuning the budget

Two environment knobs override the defaults: SCALE_SLICE_SUBREQUESTS and SCALE_SLICE_WALL_MS. They are read per slice. An invalid value (not a positive number) falls back to the default rather than throwing, and an over-set value is clamped to the platform-survivable ceiling rather than honoured. This matters: past roughly 940 subrequests the yield signal could fire only after the 1000-cap had already killed the invocation, and a cap death persists no checkpoint progress, so an over-set knob would convert a large run into a permanently unfinishable one. The clamp keeps the yield able to fire before the platform cap. You should rarely need to touch either knob.

The segment and object ceilings

The archive is built from content-addressed segments. The hard ceiling is on one segment, not on one run.

A sealed segment streams to the destination in one put. There is no multipart path to split an oversized segment, so a value is capped to MAX_STREAM_SEGMENT_BYTES before streaming, and both the R2 and the S3 destinations assert the same ceiling on the put as defence in depth. (A maximal one-gibibyte plaintext segment seals to slightly more than a gibibyte once its per-chunk tags are added; the S3 path uploads that as roughly 33 parts of 32 MiB each.)

A value larger than the segment ceiling is not rejected outright, but it is not unbounded either. It is sealed as a chained multi-segment record: the value is windowed into one-gibibyte slices, each window addressed and sealed as its own segment, and the record line carries the ordered segment list with the whole-record plaintext digest and size. Both the standalone reader and the engine reassemble these chains on restore. Byte sources (R2, S3-compatible, Images, Stream) chain up to a named record ceiling, not indefinitely: MAX_SINGLE_RECORD_CONTENT_BYTES (roughly 3.66 GiB) for a value with a stable etag that can be mid-record-resumed across slices, or the lower MAX_SINGLE_SLICE_RECORD_BYTES (roughly 1.84 GiB) for a value with no stable etag, which must complete its chain inside one slice. A value past its applicable ceiling is not sealed at all: it is skipped with a loud incompleteness marker (recordsIncomplete surfaces it; the rest of the backup completes), recoverable out of band with the downpipe CLI rather than wedging the run. Both ceilings are defined in engine/src/seal/budget.ts and applied in engine/src/sources/byte-fetch.ts.

D1 is the source with an explicit total cap, and it is worth being precise because an older note had it wrong. A D1 database is not buffered whole in memory: the dump is produced incrementally, the schema up front and then each table read in keyset-ordered pages, and fed into the seal as a re-openable stream, exactly the way a large object is sealed in windows. Peak memory is one row page plus the current sealed chunk, never the whole database, so a multi-gigabyte D1 is backed up rather than rejected. The former whole-body one-gibibyte reject is gone, and so is any single whole-export size gate: D1_EXPORT_SIZE_LIMIT (256 GiB) is now only a nominal constant, no longer enforced as a sizing pass. The streamed total is bounded by the run’s segment and subrequest budget like every other source. The one live D1 ceiling is D1_PAGE_BYTE_LIMIT, a 96 MiB bound on one in-memory row page that refuses a single pathological row too large to window.

Storage growth: no cross-run deduplication today

Within a single resumed run, content addressing deduplicates: a segment already written is skipped on the re-seal. There is no deduplication across runs. Each run derives a fresh per-run key, so its content addresses differ from every other run’s, and each run stores a full per-run snapshot.

The consequence for capacity planning is direct. Storage grows with how often you run, not with how much actually changed between runs. A daily downpipe over a mostly-static dataset still writes a full snapshot per day. Retention pruning is what bounds this over time: a superseded run’s tree is deleted and then the segments no retained run still references are garbage-collected, manifest-driven, so a delete only ever touches an object the planner has proven unreferenced. Size your destination and your retention window together, and see predicting cost for the model that turns run frequency and retention into a storage estimate.

The RUNLOG grows; the audit log is capped

Two account-wide signed logs grow as you operate, and they behave differently. Knowing which is bounded and which is not is the difference between a tidy account and a surprise.

The audit log is bounded. AUDIT_CAP is 10000 entries; at the cap the log rolls over the oldest entries first, and the console prompts an export as the count nears the cap so nothing is lost silently. The scheduled-restore drill-evidence log is bounded too, at DRILL_EVIDENCE_CAP of 500 rows, rolling oldest-first.

The RUNLOG is not bounded the same way. It is the account-wide, signed, append-only record of every run, and it grows over time without a self-pruning cap of its own. It is small per entry, so this is slow, but on a long-lived account with many frequent downpipes it is a real, monotonic growth you should plan for rather than assume away. There is no setting today that trims it. If you need it smaller, the lever is run frequency.

The dispatch floor is a scale and scheduling consequence

The only thing that begins a new backup is the engine’s reconciliation cron, declared in engine/wrangler.toml as crons = ["*/15 * * * *"]. It ticks every fifteen minutes. The scheduler’s own alarm currently only re-arms a wakeup; it does not dispatch. An already-started large run continues on the seal Durable Object’s alarm chain between ticks, but a new run only ever starts on a cron tick.

This is the source of the effective backup floor. Because the cron is the dispatch driver and it ticks every fifteen minutes, the most often a downpipe can actually begin a run is roughly every fifteen minutes. The console reflects this honestly by flooring the schedule selector at Hourly rather than offering a finer cadence it could not keep. The fifteen-minute figure is the cron tick, not a tunable knob, so do not treat it as a recovery-point objective you can dial down. The freshness consequence is developed in full on objectives, RPO and RTO, and the framing of the floor as a backup-scheduling property appears again on the backing-up overview.

Where this fits

For the runtime model behind slicing (the two Durable Objects, the lease, the two alarm chains), read the engine architecture. For what one run does end to end, see the anatomy of a backup run. For how often a backup can run and what that means for recovery, see objectives, RPO and RTO. For turning run frequency and retention into a storage and cost estimate, see predicting cost and retention and pruning.

Last updated 4 August 2026.