Storage
Semantic estate storage for snapshots, SQL blobs, and state manifests
What Storage Does
- Publish immutable estate artifacts -
estate.json, snapshots, SQL blobs, and state manifests - List and read by content hash - filenames are not proof
- Refuse path-level get/put/delete - parsers, not drivers, admit trusted values
Available Writers
| Writer | Storage | Best For |
|---|---|---|
| Filesystem | Local files | Development, CI/CD, version control |
| Durable Object | A SQLite-backed Durable Object’s ctx.storage |
One history per tenant object |
| R2 | An R2 bucket, one key prefix per estate | Histories shared outside one object |
ObjectStoreEstateStorage |
Strongly consistent object store with conditional create | Shared remote estates |
MemoryEstateStorage |
Process memory | Tests and storage conformance fixtures |
Semantic Interface
Storage is a reader/writer of content-addressed objects, not a path store:
interface MigrationStorageReader {
readEstate(): Promise<Uint8Array | null>;
listStates(): Promise<readonly string[]>;
listSnapshots(): Promise<readonly string[]>;
listSql(): Promise<readonly string[]>;
readState(id: string): Promise<Uint8Array | null>;
readSnapshot(hash: string): Promise<Uint8Array | null>;
readSql(hash: string): Promise<Uint8Array | null>;
}
interface MigrationStorageWriter extends MigrationStorageReader {
publishEstate(bytes: Uint8Array): Promise<PublishResult>;
publishSnapshot(hash: string, bytes: Uint8Array): Promise<PublishResult>;
publishSql(hash: string, bytes: Uint8Array): Promise<PublishResult>;
publishState(id: string, bytes: Uint8Array): Promise<PublishResult>;
}
interface PublishResult {
outcome: "created" | "identical";
}
Reader inventories and reads must be strongly consistent. Writer publication is conditional create: publishing identical bytes at the same identity is idempotent, while different bytes at that identity are corruption. The state manifest is published last and is the only visibility boundary for a new state.
readSnapshot, publishSnapshot, and listSnapshots cover both full schema
documents and incremental delta artifacts. Storage handles exact bytes and
physical content addresses; the migration engine authenticates and reconstructs
them. A full document’s address is its schema hash. A delta’s address uses a
separate hash domain, and the state manifest records it alongside the unchanged
full-schema hash. Custom storage implementations need no new methods.
Copy all snapshot artifacts, including transitive delta bases. Publication of a
new delta uses already durable bases, then publishes the delta and SQL before
the state manifest. Interrupted publication can leave unused artifacts, which
check reports without deleting them.
Eventually consistent or last-writer-wins storage cannot implement the writer contract directly. It needs an external single-writer or conditional publication owner. Workers KV is therefore refused as a writable estate; strongly consistent object storage with conditional writes can be used.
Custom writable storage should run the exported
createStorageConformanceSuite() against a fresh writer. The suite proves
idempotent identical publication, corruption refusal, committed-manifest
inventory, immutable estate publication, one winner among concurrent creates,
null for absent artifacts, listing as soon as a publish returns, and that a
listed state’s snapshot and SQL are already readable. It does not upgrade a storage
service whose consistency or conditional-write contract is weaker than the
interface promises.
Caching
Storage is the source of truth. Every command lists the estate’s states again and reads every artifact it needs again, so the next command sees a state published by another process or a history restored from a backup.
Only the parse is cached. Each state manifest and snapshot is parsed once per
process (one Worker isolate, one Node process), keyed by its own content hash.
It is never keyed by the estate hash, because every estate with the same
target has the same estate.json. A cached parse is used only when the bytes
just read are identical to the bytes it was parsed from. Changed bytes under a
known name are parsed again and refused, exactly as without the cache. Two
tenants share a cached value only when they store the same artifact, byte for
byte, and shared values are frozen. SQL blobs are not cached: they are not
parsed, and hashing their bytes is their identity check.
The cache holds at most 8 MiB of artifact bytes. That covers the snapshots and manifests of a 200-state estate of twenty 15-column tables; the parsed values take about as much memory again. The least recently used entries are dropped first. An estate larger than the bound still works, but its commands parse every artifact, as without the cache.
Directory Structure
The filesystem writer uses this estate layout:
migrations/
├── estate.json
├── snapshots/
│ └── <snapshot-hash>.json
├── sql/
│ └── <sql-hash>.sql
└── states/
└── <state-id>.json
Use a writer with createMigrationClient:
import { createMigrationClient } from "viborm/migrations";
import { createFsStorageWriter } from "viborm/migrations/storage/fs";
const migrations = createMigrationClient(client, {
storage: createFsStorageWriter("./migrations"),
});