Skip to content
VibORM
Esc
↑↓navigate↵open⌘Jpreview
On this page

Storage

Semantic estate storage for snapshots, SQL blobs, and state manifests

What Storage Does

  • Publish immutable estate artifacts - estate.json, snapshots, SQL blobs, and state manifests
  • List and read by content hash - filenames are not proof
  • Refuse path-level get/put/delete - parsers, not drivers, admit trusted values

Available Writers

Writer Storage Best For
Filesystem Local files Development, CI/CD, version control
Durable Object A SQLite-backed Durable Object’s ctx.storage One history per tenant object
R2 An R2 bucket, one key prefix per estate Histories shared outside one object
ObjectStoreEstateStorage Strongly consistent object store with conditional create Shared remote estates
MemoryEstateStorage Process memory Tests and storage conformance fixtures

Semantic Interface

Storage is a reader/writer of content-addressed objects, not a path store:

interface MigrationStorageReader {
  readEstate(): Promise<Uint8Array | null>;
  listStates(): Promise<readonly string[]>;
  listSnapshots(): Promise<readonly string[]>;
  listSql(): Promise<readonly string[]>;
  readState(id: string): Promise<Uint8Array | null>;
  readSnapshot(hash: string): Promise<Uint8Array | null>;
  readSql(hash: string): Promise<Uint8Array | null>;
}

interface MigrationStorageWriter extends MigrationStorageReader {
  publishEstate(bytes: Uint8Array): Promise<PublishResult>;
  publishSnapshot(hash: string, bytes: Uint8Array): Promise<PublishResult>;
  publishSql(hash: string, bytes: Uint8Array): Promise<PublishResult>;
  publishState(id: string, bytes: Uint8Array): Promise<PublishResult>;
}

interface PublishResult {
  outcome: "created" | "identical";
}

Reader inventories and reads must be strongly consistent. Writer publication is conditional create: publishing identical bytes at the same identity is idempotent, while different bytes at that identity are corruption. The state manifest is published last and is the only visibility boundary for a new state.

readSnapshot, publishSnapshot, and listSnapshots cover both full schema documents and incremental delta artifacts. Storage handles exact bytes and physical content addresses; the migration engine authenticates and reconstructs them. A full document’s address is its schema hash. A delta’s address uses a separate hash domain, and the state manifest records it alongside the unchanged full-schema hash. Custom storage implementations need no new methods.

Copy all snapshot artifacts, including transitive delta bases. Publication of a new delta uses already durable bases, then publishes the delta and SQL before the state manifest. Interrupted publication can leave unused artifacts, which check reports without deleting them.

Eventually consistent or last-writer-wins storage cannot implement the writer contract directly. It needs an external single-writer or conditional publication owner. Workers KV is therefore refused as a writable estate; strongly consistent object storage with conditional writes can be used.

Custom writable storage should run the exported createStorageConformanceSuite() against a fresh writer. The suite proves idempotent identical publication, corruption refusal, committed-manifest inventory, immutable estate publication, one winner among concurrent creates, null for absent artifacts, listing as soon as a publish returns, and that a listed state’s snapshot and SQL are already readable. It does not upgrade a storage service whose consistency or conditional-write contract is weaker than the interface promises.

Caching

Storage is the source of truth. Every command lists the estate’s states again and reads every artifact it needs again, so the next command sees a state published by another process or a history restored from a backup.

Only the parse is cached. Each state manifest and snapshot is parsed once per process (one Worker isolate, one Node process), keyed by its own content hash. It is never keyed by the estate hash, because every estate with the same target has the same estate.json. A cached parse is used only when the bytes just read are identical to the bytes it was parsed from. Changed bytes under a known name are parsed again and refused, exactly as without the cache. Two tenants share a cached value only when they store the same artifact, byte for byte, and shared values are frozen. SQL blobs are not cached: they are not parsed, and hashing their bytes is their identity check.

The cache holds at most 8 MiB of artifact bytes. That covers the snapshots and manifests of a 200-state estate of twenty 15-column tables; the parsed values take about as much memory again. The least recently used entries are dropped first. An estate larger than the bound still works, but its commands parse every artifact, as without the cache.

Directory Structure

The filesystem writer uses this estate layout:

migrations/
├── estate.json
├── snapshots/
│   └── <snapshot-hash>.json
├── sql/
│   └── <sql-hash>.sql
└── states/
    └── <state-id>.json

Use a writer with createMigrationClient:

import { createMigrationClient } from "viborm/migrations";
import { createFsStorageWriter } from "viborm/migrations/storage/fs";

const migrations = createMigrationClient(client, {
  storage: createFsStorageWriter("./migrations"),
});

Next Steps

Was this page helpful?