diff --git a/CHANGELOG.md b/CHANGELOG.md index 1dd9c368..1807a23f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,8 +7,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **WarpStream** — composable async stream primitive built on `AsyncIterable`. Domain concept for "data flow over time." Supports `pipe`, `tee`, `mux`, `demux`, `drain`, `reduce`, `forEach`, `collect`. Natural backpressure via `for await`, error propagation via async iterator protocol, cooperative cancellation via `AbortSignal`. +- **Stream transforms** — `CborEncodeTransform`, `CborDecodeTransform`, `GitBlobWriteTransform`, `TreeAssemblerSink`, `IndexShardEncodeTransform` — composable infrastructure pipeline stages for encode → blobWrite → treeAssemble. +- **Artifact record classes** — `CheckpointArtifact` family (`StateArtifact`, `FrontierArtifact`, `AppliedVVArtifact`), `IndexShard` family (`MetaShard`, `EdgeShard`, `LabelShard`, `PropertyShard`, `ReceiptShard`), `PatchEntry`, `ProvenanceEntry`. Runtime-backed domain nouns with constructor validation and `instanceof` dispatch. +- **`PatchJournalPort.scanPatchRange()`** — streaming alternative to `loadPatchRange()`. Returns `WarpStream` for incremental patch consumption. Commit walking moved from SyncProtocol into the adapter. +- **`StateHashService`** — standalone canonical state hash computation. Separately callable by checkpoint creation, comparison, materialization, and verification. +- **Index builder `yieldShards()`** — `LogicalBitmapIndexBuilder` and `PropertyIndexBuilder` yield `IndexShard` record instances via generators. Proven byte-identical to legacy `serialize()` path. +- **`LogicalIndexBuildService.buildStream()`** — returns `WarpStream` merging both builders via `mux()`. +- **Hex tripwire test** — 36 automated checks scanning domain files for forbidden codec imports/usage. Fails loud if domain touches `defaultCodec`, `cbor-x`, or `codec.encode()`/`codec.decode()`. +- **Golden fixtures** — known CBOR bytes for patches, checkpoints, VV, frontier, index shards. Wire format stability proven across refactor. + +### Fixed + +- **CborPatchJournalAdapter.readPatch()** — now throws `EncryptionError` when `encrypted=true` but no `patchBlobStorage` is configured. Previously fell through to the unencrypted `blobPort.readBlob()` path, returning wrong data. +- **Writer constructor** — `patchJournal` is now a required parameter. Previously optional, which let `beginPatch()` succeed but `commit()` hard-fail with a confusing `PersistenceError` from `PatchBuilderV2`. + ### Changed +- **CheckpointStorePort collapsed** — 7 micro-methods (`writeState`, `readState`, etc.) replaced with 2 semantic operations: `writeCheckpoint(record)` and `readCheckpoint(treeOids)`. A checkpoint is one domain event, not a bag of individual blob writes. +- **SyncProtocol uses stream scan** — `processSyncRequest()` prefers `patchJournal.scanPatchRange()` (streaming) over `loadPatchRange()` (array). Falls back to legacy path when unavailable. +- **PatchBuilderV2, SyncProtocol, Writer** — codec-free. Patch persistence goes through `PatchJournalPort`; domain never imports `defaultCodec` or calls `codec.encode()`/`codec.decode()`. +- **CheckpointService** — routes through `CheckpointStorePort` when available. Legacy codec-based paths remain as fallback. - **The Method** — introduced `METHOD.md` as the development process framework. Filesystem-native backlog (`docs/method/backlog/`) with lane directories (`inbox/`, `asap/`, `up-next/`, `cool-ideas/`, `bad-code/`). Legend-prefixed filenames (`PROTO_`, `TRUST_`, `VIZ_`, `TUI_`, `DX_`, `PERF_`). Sequential cycle numbering (`docs/design//`). Dual-audience design docs (sponsor human + sponsor agent). Replaced B-number system entirely. - **Backlog migration** — all 49 B-number and OG items migrated from `BACKLOG/` to `docs/method/backlog/` lanes. Tech debt journal (`.claude/bad_code.md`) split into 10 individual files in `bad-code/`. Cool ideas journal split into 13 individual files in `cool-ideas/`. `docs/release.md` moved to `docs/method/release.md`. `BACKLOG/` directory removed. diff --git a/contracts/type-surface.m8.json b/contracts/type-surface.m8.json index 95bd66b5..5e8e8090 100644 --- a/contracts/type-surface.m8.json +++ b/contracts/type-surface.m8.json @@ -1237,6 +1237,18 @@ "WriterError": { "kind": "class" }, + "WorldlineSelector": { + "kind": "class" + }, + "LiveSelector": { + "kind": "class" + }, + "CoordinateSelector": { + "kind": "class" + }, + "StrandSelector": { + "kind": "class" + }, "buildWarpStateIndex": { "kind": "function", "async": true diff --git a/docs/BEARING.md b/docs/BEARING.md index f6634f18..000997ce 100644 --- a/docs/BEARING.md +++ b/docs/BEARING.md @@ -2,6 +2,18 @@ Updated at cycle boundaries. Not mid-cycle. +## Invariants + +1. **HEXAGONAL** — domain never imports infrastructure +2. **DETERMINISTIC** — same patches, any order → same materialized state +3. **APPEND-ONLY** — Git history never rewritten +4. **MULTI-WRITER** — each writer owns its own ref, no coordination +5. **RUNTIME-TRUTH** — domain concepts are classes with validated invariants (SSJS P1) +6. **BOUNDARY-HONESTY** — untrusted input validated at boundaries +7. **TRAVERSAL-TRUTH** — unbounded data flows through streams (traversal); + bounded truth crosses ports (contracts). Never conflated. Persistence + ordering is canonical regardless of stream timing. + ## Where are we going? Structural decomposition of `domain/services/` — 83 files in a flat @@ -15,9 +27,13 @@ analysis, 10 cohesive groups identified, no circular dependencies. ## What feels wrong? -- WorldlineSource is still a tagged object, not a subclass hierarchy. -- `defaultCodec.js` lives in `domain/utils/` but imports `cbor-x` - directly — a hexagonal boundary violation. +- ~~WorldlineSource~~ Shipped as WorldlineSelector hierarchy (cycle 0007). +- 20 domain services do serialization directly (`codec.encode()`/ + `codec.decode()`). The fix is a two-stage boundary: artifact-level + ports (PatchJournalPort, CheckpointStorePort, etc.) that speak + domain types, backed by codec-owning adapters over the raw Git + ports. Strangler refactor, patches first. + See `NDNM_defaultcodec-to-infrastructure.md`. - The two legends (CLEAN_CODE, NO_DOGS_NO_MASTERS) overlap significantly. May need consolidation or clearer boundaries. - JoinReducer is imported by 8 of 10 service clusters — it is the diff --git a/docs/design/0007-viewpoint-design/viewpoint-design.md b/docs/design/0007-viewpoint-design/viewpoint-design.md index c070be80..8f4f015e 100644 --- a/docs/design/0007-viewpoint-design/viewpoint-design.md +++ b/docs/design/0007-viewpoint-design/viewpoint-design.md @@ -414,7 +414,7 @@ change shape. The inbound boundary factory. Accepts plain objects with a `kind` discriminant and returns the appropriate class instance: -```javascript +```text WorldlineSelector.from({ kind: 'live' }) → new LiveSelector() @@ -526,7 +526,7 @@ const selector = WorldlineSelector.from(source).clone(); **Before (materializeSource dispatch):** -```javascript +```text if (source.kind === 'live') { ... } if (source.kind === 'coordinate') { ... } return await materializeStrandSource(...); @@ -534,7 +534,7 @@ return await materializeStrandSource(...); **After:** -```javascript +```text if (source instanceof LiveSelector) { ... } if (source instanceof CoordinateSelector) { ... } return await materializeStrandSource(...); @@ -542,7 +542,7 @@ return await materializeStrandSource(...); **Before (Worldline.source getter):** -```javascript +```text get source() { return cloneWorldlineSource(this._source); } @@ -550,7 +550,7 @@ get source() { **After:** -```javascript +```text get source() { return this._source.toDTO(); // plain object for public API } @@ -611,7 +611,7 @@ export { Add the selector class declarations. Keep the existing `WorldlineSource` type and interfaces unchanged: -```typescript +```text // NEW — selector classes export class WorldlineSelector { clone(): WorldlineSelector; diff --git a/docs/design/0008-stream-architecture/stream-architecture.md b/docs/design/0008-stream-architecture/stream-architecture.md new file mode 100644 index 00000000..9a7706f1 --- /dev/null +++ b/docs/design/0008-stream-architecture/stream-architecture.md @@ -0,0 +1,234 @@ +# Cycle 0008 — Stream Architecture + +**Sponsor (human):** James +**Sponsor (agent):** Claude +**Status:** DESIGN + +## Hill + +A developer can pipe domain objects through a composable stream +pipeline where encoding, persistence, and tree assembly are transforms +and sinks — never called directly by domain code. Semantic ports +remain for bounded single-artifact operations. Artifact records carry +runtime identity. The system is memory-bounded for unbounded datasets. + +## The Rule + +Streams are for scale. Ports are for meaning. Artifacts are the nouns. +Paths are infrastructure. + +## Playback Questions + +1. Does the pipeline produce byte-identical output to the legacy path? +2. Does a constrained-heap test complete for a dataset that would + otherwise OOM? +3. Do semantic ports still tell you WHAT is being persisted and WHAT + lifecycle rules apply? +4. Is CBOR vocabulary absent from domain nouns? +5. Does every artifact record class add runtime identity, not just a name? + +## Non-Goals + +- CborStream or any codec-named class in the domain +- Marker stream subclasses that don't add flow behavior +- Melting separate ports/services into one generic pipe +- Replacing bounded single-artifact reads with streams + +--- + +## Architecture + +### One Stream Container + +```text +WarpStream — domain primitive + pipe / tee / mux / demux / drain / reduce / forEach / collect + [Symbol.asyncIterator]() +``` + +No domain subclasses. Identity lives on elements, not the container. + +### Semantic Ports + +Ports define what is being persisted and what lifecycle rules apply. +Bounded operations stay `Promise`. Unbounded operations return or +accept `WarpStream`. + +**PatchJournalPort** (keep, extend) +```text +writePatch(patch) → Promise bounded write +readPatch(oid) → Promise bounded read +scanPatchRange(...) → WarpStream unbounded scan (NEW) +``` + +**CheckpointStorePort** (collapse micro-methods) +```text +writeCheckpoint(record) → Promise one call +readCheckpoint(sha) → Promise bounded read +``` +Adapter internally streams artifacts through the pipeline. + +**IndexStorePort** (NEW, streaming) +```text +writeShards(stream) → Promise WarpStream → tree OID +scanShards(...) → WarpStream unbounded read +``` + +**ProvenanceStorePort** (NEW, separate concept) +```text +scanEntries(...) → WarpStream +writeIndex(index) → Promise +``` +Own port. Physical colocation under checkpoint tree ≠ semantic +ownership. Checkpoint = recovery. Provenance = causal/query/verification. +Different jobs, different lifecycle, different consumers. + +**StateHashService** (separate callable, not buried in adapter) +```text +compute(state) → Promise +``` +Used by verification, comparison, detached checks, AND checkpoint +creation. Not exclusively inside writeCheckpoint(). + +### Artifact Records + +Runtime identity on elements, not containers (P1/P7). + +**CheckpointArtifact** — closed subclass family +```text +CheckpointArtifact (abstract base) + common: checkpointRef, schemaVersion + +StateArtifact extends CheckpointArtifact + payload: { state: WarpStateV5 } + +FrontierArtifact extends CheckpointArtifact + payload: { frontier: Map } + +AppliedVVArtifact extends CheckpointArtifact + payload: { appliedVV: VersionVector } +``` +No paths. No CBOR. No blob OIDs. No adapter trivia. + +**IndexShard** — subtype family (not one generic class) +```text +IndexShard (base) + common: indexFamily, shardId, schemaVersion + +MetaShard extends IndexShard + payload: { nodeToGlobal, alive, nextLocalId } + +EdgeShard extends IndexShard + payload: { direction, shardKey, buckets } + +LabelShard extends IndexShard + payload: { labels: [string, number][] } + +PropertyShard extends IndexShard + payload: { entries: [string, Record][] } +``` +The code already treats shard families differently (isMetaShard, +isEdgeShard, classifyShards). One mega-shard class is just `any` +with better PR. + +**PatchEntry** — `{ patch: PatchV2, sha: string }` + +**ProvenanceEntry** — `{ nodeId, patchShas }` + +### Path Mapping + +Adapter owns it. Full stop. Domain produces artifact records. +Adapter maps to Git tree paths at the last responsible moment. + +```text +StateArtifact → 'state.cbor' +FrontierArtifact → 'frontier.cbor' +MetaShard → 'meta_XX.cbor' +EdgeShard → '{fwd|rev}_XX.cbor' +``` + +Static mapping table or instanceof dispatcher in the adapter. +No `.path()` on domain objects. Paths are storage convention. + +Domain owns meaning. Adapter owns layout. + +### Infrastructure Transforms + +```text +CborEncodeTransform artifact → [path, bytes] +CborDecodeTransform [path, bytes] → artifact +GitBlobWriteTransform [path, bytes] → [path, oid] +TreeAssemblerSink [path, oid] → finalize → treeOid +``` + +Encode → blobWrite → treeAssemble stays entirely in infrastructure. +CBOR is boundary vocabulary — never a domain noun. + +### Pipeline Examples + +```js +// Index write (unbounded, streaming) +await indexStore.writeShards( + WarpStream.from(builder.yieldShards()) +); +// Adapter internally: stream → encode → blobWrite → treeAssemble + +// Checkpoint write (bounded, one call) +await checkpointStore.writeCheckpoint({ + state, frontier, appliedVV, stateHash, provenanceIndex +}); +// Adapter internally: yield artifacts → encode → blobWrite → tree + +// Patch scan (unbounded) +const patches = patchJournal.scanPatchRange(writerRef, fromSha, toSha); +for await (const entry of patches) { + reducer.apply(entry.patch); +} +``` + +### Ordering Guarantee + +`WarpStream.mux()` interleaves by arrival order. Async completion +timing must not bleed into tree assembly. `TreeAssemblerSink` sorts +entries before `writeTree()`. Deterministic Git trees don't care +which blob write finished first. + +--- + +## Migration Plan + +### Phase 1 — Artifact records + streaming ports + +- CheckpointArtifact family (StateArtifact, FrontierArtifact, + AppliedVVArtifact) +- IndexShard family (MetaShard, EdgeShard, LabelShard, PropertyShard) +- PatchEntry, ProvenanceEntry records +- IndexStorePort with writeShards/scanShards +- PatchJournalPort.scanPatchRange() +- StateHashService +- ProvenanceStorePort + +### Phase 2 — Wire write paths + +- CheckpointStorePort collapse → writeCheckpoint(record) +- Index builders: yieldShards() returns IndexShard subclass instances +- SyncProtocol: consume scanPatchRange() instead of loadPatchRange() + +### Phase 3 — P5 cleanup + +- Remove defaultCodec from all domain files +- Delete defaultCodec.js, canonicalCbor.js +- Expand tripwire to all migrated files + +### Phase 4 — Memory-bounded witnesses + +- Constrained-heap tests +- Naming audit for slurp APIs + +--- + +## Accessibility / Localization / Agent-Inspectability + +- **Agent-Inspectability**: Artifact records are `instanceof`- + dispatchable. WarpStream carries AbortSignal. Sink.consume() + returns typed results. diff --git a/docs/method/backlog/asap/NDNM_defaultcodec-to-infrastructure.md b/docs/method/backlog/asap/NDNM_defaultcodec-to-infrastructure.md index 774540af..504795a8 100644 --- a/docs/method/backlog/asap/NDNM_defaultcodec-to-infrastructure.md +++ b/docs/method/backlog/asap/NDNM_defaultcodec-to-infrastructure.md @@ -1,18 +1,208 @@ -# Move defaultCodec.js to infrastructure +# Dissolve serialization from domain (P5) -**Effort:** S +**Effort:** L ## Problem -`src/domain/utils/defaultCodec.js` imports `cbor-x` directly — a -concrete codec dependency inside `src/domain/`. This is a P5 violation: -"Serialization Is the Codec's Problem." The domain layer should speak -only through the CodecPort. +20 domain services import `defaultCodec` and call `codec.encode()` / +`codec.decode()` directly. This is a P5 violation: "Serialization Is +the Codec's Problem." Domain services should work with domain objects, +not bytes. Serialization belongs at the infrastructure boundary. -## Fix +`defaultCodec` is a singleton pretending to be dependency injection. +The `codec` constructor param is theater — every service can bypass +its caller and import the global directly. -Move `defaultCodec.js` to `src/infrastructure/codecs/DefaultCodecAdapter.js`. -Update all domain imports. The domain's fallback becomes a lazy import -of the infrastructure adapter (same pattern as `CasBlobAdapter`). +## Wrong Fixes (Cycle 0007) -Flagged in the Systems-Style audit (PR #75 session). +1. **Move `defaultCodec.js` to infrastructure.** Changes where the file + lives, not where the boundary is. Domain services still call + `encode()`. + +2. **Thread codec through constructors.** Dependency-passing theater. + Domain services still call `encode()`, they just receive the codec + from their parent instead of importing it. Constructor injection is + not absolution. + +3. **Move serializer services to infrastructure.** Keeps the serializers + alive, just in a different folder. And risks creating a god object + if everything lands in GitGraphAdapter. + +4. **Dissolve serializers into GitGraphAdapter.** Cures one god object + (domain doing serialization) by creating another (infrastructure + doing everything). GitGraphAdapter is Git plumbing. It stays Git + plumbing. + +## Right Fix: Two-Stage Boundary + +If a domain service needs a codec, the boundary is in the wrong place. +Bytes are sewage. Keep them in the pipes. + +### Stage 1 — Domain-facing artifact ports + +Ports that speak domain artifacts and lifecycle semantics. Named by +what the caller means, not how Git stores it. + +| Port | Speaks | Lifecycle | +|---|---|---| +| `PatchJournalPort` | PatchV2 ops | Append-only | +| `CheckpointStorePort` | Checkpoint records | Replace-latest | +| `IndexStorePort` | Index shard structures | Tree-structured, per-shard | +| `ProvenanceStorePort` | Provenance mappings | Alongside checkpoint | +| `BtrStorePort` | BTR records | Tamper-evident chain | + +### Stage 2 — Infrastructure adapters (codec owners) + +Adapters that turn domain artifacts into bytes over the raw Git ports. +Each adapter owns its codec instance. + +| Adapter | Uses | +|---|---| +| `CborPatchJournalAdapter` | CommitPort, BlobPort, RefPort, CborCodec | +| `CborCheckpointStoreAdapter` | CommitPort, BlobPort, RefPort, CborCodec | +| `CborIndexStoreAdapter` | TreePort, BlobPort, CborCodec | +| `CborProvenanceStoreAdapter` | BlobPort, CborCodec | +| `CborBtrStoreAdapter` | CommitPort, BlobPort, CborCodec | + +### Existing raw Git ports (unchanged) + +`CommitPort`, `BlobPort`, `TreePort`, `RefPort`, `ConfigPort` stay as +infrastructure-level primitives. They speak bytes. That's correct — +they ARE about bytes. Domain services just stop talking to them +directly for artifact persistence. + +## Critical: Split Semantic Projection from Byte Encoding + +Some "serializer" files contain two concerns jammed together: + +- `StateSerializerV5`: `projectStateV5()` (domain — semantic + projection of visible state) + `serializeStateV5()` (boundary — + byte encoding). Split these apart. +- `CheckpointSerializerV5`: `computeAppliedVV()` (domain logic) + + `serializeAppliedVV()` / `deserializeAppliedVV()` (boundary logic). + +Domain projection logic stays in domain. Byte encoding goes behind +the adapter. + +## Boundary Records + +Named by what they ARE, not how they're stored: + +- `PatchRecord` +- `CheckpointRecord` +- `IndexShardRecord` +- `BtrRecord` + +## Strangler Refactor — Cut Plan + +One artifact family per slice. Prove the architecture with the two +biggest seams before touching the weirder storage families. + +### Slice 1: Patches + +- Add `PatchJournalPort` +- Move patch encode/decode out of domain callers +- Wire `Writer` / `SyncProtocol` / `PatchBuilderV2` through it +- Kill patch-related `defaultCodec` usage + +### Slice 2: Checkpoints + +- Split `computeAppliedVV` from checkpoint byte encoding +- Add `CheckpointStorePort` +- Move checkpoint encode/decode behind adapter + +### Slice 3: Indexes + +- Separate "build shard structure" from "encode shard bytes" +- Keep algorithmic builders if they're truly algorithmic +- Add `IndexStorePort` + +### Slice 4: Provenance + BTR + +- Same pattern +- Keep tamper-evident semantics visible in the port contract +- Hide bytes behind adapter + +## Hard Gates + +- **Hex Tripwire Test**: one test that recursively scans `src/domain/` + for forbidden imports (`cbor-x`, `defaultCodec`, `.encode()` / + `.decode()` on persistence codecs). Added at the start, ratcheted + down per slice. +- **Golden Blob Museum**: check in canonical patch/checkpoint/index + fixtures from real repo data. Require exact round-trip compatibility. + Proves refactor didn't change wire format. +- **ESLint rule**: ban `defaultCodec` imports under + `src/domain/services/`. +- **Design matrix**: artifact → domain type → boundary record → port → + adapter → underlying raw ports. Lives in the cycle design doc. + +## The 20 Offenders + +### state/ (split projection from encoding) + +- CheckpointSerializerV5.js +- StateSerializerV5.js + +### index/ (split structure building from shard encoding) + +- BitmapIndexBuilder.js +- StreamingBitmapIndexBuilder.js +- IncrementalIndexUpdater.js +- LogicalIndexBuildService.js +- IndexRebuildService.js +- LogicalIndexReader.js +- PropertyIndexReader.js +- IndexStalenessChecker.js +- LogicalBitmapIndexBuilder.js +- PropertyIndexBuilder.js + +### provenance/ (serialize BTRs and provenance payloads) + +- BoundaryTransitionRecord.js +- ProvenanceIndex.js + +### sync/ (serialize sync protocol messages) + +- SyncProtocol.js + +### services root (various serialization needs) + +- Frontier.js (serialize/deserialize frontier) +- PatchBuilderV2.js (encode patch ops) +- MaterializedViewService.js (orchestrates index serialization) +- WormholeService.js (compress/decompress wormholes) + +### warp/ (writer encodes patches) + +- Writer.js + +### utils/ (dead code) + +- canonicalCbor.js (unused — delete) + +## Progress + +### Shipped (Slices 1-2) + +- **Patches**: PatchJournalPort + CborPatchJournalAdapter. PatchBuilderV2, + SyncProtocol, Writer are codec-free. 27 tripwire checks. +- **Checkpoints**: CheckpointStorePort + CborCheckpointStoreAdapter. + CheckpointService routes through port. 9 tripwire checks. + +### Remaining (Slices 3-4) → Stream Architecture Cycle + +Index files (12) and provenance/BTR files need the stream architecture, +not more per-artifact ports. Collection APIs that return graph-scale +aggregates must become `AsyncIterable`. Single bounded +artifacts (Slices 1-2) are correctly handled by semantic ports. + +See `PERF_stream-architecture.md` for the stream cycle proposal. + +## Source + +Cycle 0007 defaultCodec migration attempt (failed). Root cause analysis +identified the P5 violation. Corrected 2026-04-04: the fix is a +two-stage boundary with artifact-level ports, not file relocation or +serializer migration. Slices 3-4 deferred to stream architecture cycle +(2026-04-04). diff --git a/docs/method/backlog/asap/PERF_stream-architecture.md b/docs/method/backlog/asap/PERF_stream-architecture.md new file mode 100644 index 00000000..07d47def --- /dev/null +++ b/docs/method/backlog/asap/PERF_stream-architecture.md @@ -0,0 +1,83 @@ +# Stream Architecture — Honest APIs for Unbounded Data + +**Effort:** XL + +## Invariant: TRAVERSAL-TRUTH + +Unbounded data flows through streams (traversal). Bounded truth +crosses ports (contracts). Never conflated. Persistence ordering +is canonical regardless of stream timing. + +A stream is a linear projection of a worldline traversal. A port +defines what must be true at a boundary. These are orthogonal +axes — not competing abstractions. + +### Violations + +- A port returning `AsyncIterable` for a bounded artifact (lying + about traversal — a single patch IS bounded) +- A stream establishing truth without a port (bypassing contracts) +- Domain code consuming raw stream elements without artifact identity + (vibes pipeline — `AsyncIterable>`) +- Persistence ordering depending on async completion timing + (non-deterministic truth — finalization must restore canonical order) + +### Standing Playback Question + +Does this code route unbounded data through streams and bounded +truth through ports? Is persistence ordering canonical? + +### Connection to the Papers + +| DSM Property | Paper | Mechanism | +|---|---|---| +| Artifacts are first-class | I (WARP inductive def) | P1 runtime-backed forms | +| Ports define truth boundaries | III (boundary encoding) | Hexagonal architecture | +| Streams are worldline projections | II (tick sequences) | AsyncIterable traversal | +| Ordering restored before persistence | II (tick-level confluence) | TreeAssemblerSink sorts | +| Concurrency semantically erased | II (admissible batches) | Transforms are pure | +| Replay produces identical results | III (computational holography) | Deterministic finalization | + +A stream IS a worldline projection: +- `scanPatchRange(from, to)` = projecting a worldline segment +- `mux(writerA, writerB)` = merging worldlines (materialization) +- Backpressure = causal ordering (can't consume tick N+1 before N) +- Stream identity = frontier position (version vector advances) +- Observer `O: Hist(U,R) → Tr` (Paper IV) = stream transform + +## The Two Cases + +### Case 1 — Bounded artifact (port) + +A single patch, checkpoint, shard. The semantic object is +reasonable. The port speaks `Promise`: + +```text +readPatch(oid) → Promise +writeCheckpoint(record) → Promise +``` + +### Case 2 — Unbounded traversal (stream) + +Patch history, index shards, provenance walks. The dataset can +exceed memory. The API speaks `AsyncIterable`: + +```text +scanPatchRange(...) → WarpStream +yieldShards() → Generator +``` + +The API shape tells the caller: you can't slurp this. + +## Naming Convention + +| Name | Meaning | +|---|---| +| `scan*`, `stream*`, `enumerate*` | Honest: incremental, unbounded-safe | +| `get*List()`, `getAll*()` | Dangerous: whole-materialization | +| `collect()` | Explicitly dangerous opt-in | + +## Source + +P5 codec dissolution → stream architecture design (2026-04-04). +Formalized as TRAVERSAL-TRUTH invariant. diff --git a/docs/method/backlog/asap/PERF_stream-subclass-hierarchy.md b/docs/method/backlog/asap/PERF_stream-subclass-hierarchy.md new file mode 100644 index 00000000..770a192a --- /dev/null +++ b/docs/method/backlog/asap/PERF_stream-subclass-hierarchy.md @@ -0,0 +1,20 @@ +# Artifact record classes + streaming port methods + +**Effort:** M + +Runtime identity on ELEMENTS, not stream containers. No CborStream +in domain. No marker subclasses of WarpStream. + +Artifact records: +- CheckpointArtifact (State | Frontier | AppliedVV) — for checkpoint + write pipeline +- IndexShard — for index write pipeline +- PatchEntry — for patch scan stream +- ProvenanceEntry — for provenance scan stream + +Streaming port methods: +- PatchJournalPort.scanRange() → WarpStream +- IndexStorePort.writeShards(WarpStream) → treeOid +- IndexStorePort.scanShards() → WarpStream + +See cycle 0008 design doc. diff --git a/docs/method/backlog/asap/PERF_stream-write-migration.md b/docs/method/backlog/asap/PERF_stream-write-migration.md new file mode 100644 index 00000000..2b9438ee --- /dev/null +++ b/docs/method/backlog/asap/PERF_stream-write-migration.md @@ -0,0 +1,19 @@ +# Migrate write paths to stream pipeline + +**Effort:** L + +First streaming wins — the graph-scale liars: + +1. loadPatchRange() → scanPatchRange() returning WarpStream +2. index serialize() → yieldShards() through WarpStream pipeline + (already proven byte-identical for LogicalBitmapIndexBuilder) +3. Collapse CheckpointStorePort micro-methods into + writeCheckpoint(record) — adapter streams artifacts internally + +Keep PatchJournalPort for bounded single-artifact writes. Add +scanRange() for unbounded reads. CheckpointStorePort gets surgery +(collapse, not deletion). + +Encode → blobWrite → treeAssemble stays in infrastructure. + +See cycle 0008 design doc. diff --git a/docs/method/backlog/cool-ideas/DX_artifact-store-stack-diagram.md b/docs/method/backlog/cool-ideas/DX_artifact-store-stack-diagram.md new file mode 100644 index 00000000..3ffb037b --- /dev/null +++ b/docs/method/backlog/cool-ideas/DX_artifact-store-stack-diagram.md @@ -0,0 +1,24 @@ +# Artifact Store Stack Diagram + +A single doc showing the full persistence stack: + +```text +Domain Service + ↓ domain objects +Artifact Port (PatchJournalPort, CheckpointStorePort, ...) + ↓ domain objects +Codec Adapter (CborPatchJournalAdapter, ...) + ↓ bytes +Raw Git Port (BlobPort, TreePort, CommitPort, RefPort) + ↓ bytes +GitGraphAdapter + ↓ git plumbing calls +Git +``` + +Lives in the design doc for the P5 dissolution cycle. Updated as each +slice lands. + +## Source + +P5 codec dissolution planning (2026-04-04). diff --git a/docs/method/backlog/cool-ideas/DX_golden-blob-museum.md b/docs/method/backlog/cool-ideas/DX_golden-blob-museum.md new file mode 100644 index 00000000..bcb8f1c4 --- /dev/null +++ b/docs/method/backlog/cool-ideas/DX_golden-blob-museum.md @@ -0,0 +1,17 @@ +# Golden Blob Museum + +Check in canonical patch/checkpoint/index fixtures extracted from real +repo data. Require exact round-trip compatibility: same bytes in, same +domain objects out. Proves refactors don't change wire format +accidentally. + +Fixtures should cover: +- PatchV2 (schema:2) with all op types +- Checkpoint (V5 full state) +- Index shards (meta, fwd, rev, props) +- ProvenanceIndex +- BoundaryTransitionRecord + +## Source + +P5 codec dissolution planning (2026-04-04). diff --git a/docs/method/backlog/cool-ideas/DX_hex-tripwire-test.md b/docs/method/backlog/cool-ideas/DX_hex-tripwire-test.md new file mode 100644 index 00000000..0ad2b287 --- /dev/null +++ b/docs/method/backlog/cool-ideas/DX_hex-tripwire-test.md @@ -0,0 +1,12 @@ +# Hex Tripwire Test + +One test that recursively scans `src/domain/` for forbidden +imports/usages: `cbor-x`, `defaultCodec`, `.encode()` / `.decode()` +on persistence codecs. Cheap, brutal, effective. + +Add at the start of the P5 dissolution work. Ratchet the allowed +count down per slice as each artifact family is migrated. + +## Source + +P5 codec dissolution planning (2026-04-04). diff --git a/docs/method/backlog/cool-ideas/DX_serializer-exorcism-commit-series.md b/docs/method/backlog/cool-ideas/DX_serializer-exorcism-commit-series.md new file mode 100644 index 00000000..2e3f7d86 --- /dev/null +++ b/docs/method/backlog/cool-ideas/DX_serializer-exorcism-commit-series.md @@ -0,0 +1,18 @@ +# Serializer Exorcism Commit Series + +One artifact family per commit, each with a before/after boundary +diff. Makes the git history teach the rule: "domain stops speaking +bytes." + +Commit series: +1. Patches: PatchJournalPort + CborPatchJournalAdapter +2. Checkpoints: CheckpointStorePort + CborCheckpointStoreAdapter +3. Indexes: IndexStorePort + CborIndexStoreAdapter +4. Provenance + BTR: remaining store ports + adapters + +Each commit should read as a self-contained lesson in boundary +placement. + +## Source + +P5 codec dissolution planning (2026-04-04). diff --git a/docs/method/backlog/up-next/PERF_stream-cleanup.md b/docs/method/backlog/up-next/PERF_stream-cleanup.md new file mode 100644 index 00000000..23b6245c --- /dev/null +++ b/docs/method/backlog/up-next/PERF_stream-cleanup.md @@ -0,0 +1,13 @@ +# Remove per-artifact ports + defaultCodec + +**Effort:** M + +After write and read paths are migrated to stream pipeline: + +- Remove PatchJournalPort, CborPatchJournalAdapter +- Remove CheckpointStorePort, CborCheckpointStoreAdapter +- Remove defaultCodec from all domain files +- Delete defaultCodec.js, canonicalCbor.js +- Expand tripwire to all migrated files + +See cycle 0008 design doc. diff --git a/docs/method/backlog/up-next/PERF_stream-memory-tests.md b/docs/method/backlog/up-next/PERF_stream-memory-tests.md new file mode 100644 index 00000000..44e8e0a0 --- /dev/null +++ b/docs/method/backlog/up-next/PERF_stream-memory-tests.md @@ -0,0 +1,15 @@ +# Memory-bounded stream witnesses + +**Effort:** M + +Constrained-heap tests (`--max-old-space-size=64`) proving the stream +architecture is memory-bounded: + +1. Build index with 1M nodes via streaming pipeline +2. Materialize graph with 100K patches via patch stream +3. Checkpoint large state via CborStream pipeline + +If anything buffers the full dataset, it blows up. The test IS the +architecture proof. + +See cycle 0008 design doc. diff --git a/docs/method/backlog/up-next/PERF_stream-read-migration.md b/docs/method/backlog/up-next/PERF_stream-read-migration.md new file mode 100644 index 00000000..660afab7 --- /dev/null +++ b/docs/method/backlog/up-next/PERF_stream-read-migration.md @@ -0,0 +1,11 @@ +# Migrate read paths + unbounded scans to streams + +**Effort:** L + +- Unbounded reads become AsyncIterable: scanPatches() → PatchStream, + scanIndexShards() → IndexShardStream +- Bounded single-artifact reads stay Promise +- Index readers decode via CborDecodeTransform pipeline +- Naming audit: rename slurp APIs to collect*() (poison pill) + +See cycle 0008 design doc. diff --git a/docs/method/backlog/up-next/PROTO_WESLEY_receipt-envelope-boundary.md b/docs/method/backlog/up-next/PROTO_WESLEY_receipt-envelope-boundary.md new file mode 100644 index 00000000..dab534e3 --- /dev/null +++ b/docs/method/backlog/up-next/PROTO_WESLEY_receipt-envelope-boundary.md @@ -0,0 +1,21 @@ +# WESLEY Receipt Envelope Boundary + +Coordination: `WESLEY_protocol_surface_cutover` + +The current Continuum hill is trying to prove one boring shared contract +family, likely around receipts and nearby causal-envelope nouns. That only +works if `git-warp` names which receipt and provenance fields are substrate +facts and which are debugger or runtime projections. + +Wesley should not guess these nouns from the outside, and `warp-ttd` should not +smuggle debugger policy back into the substrate envelope. + +Work: + +- freeze the minimal substrate-owned receipt and provenance anchors external + consumers may depend on +- keep adapter and debugger projections out of the substrate contract +- expose stable names, digests, or version hooks Wesley can target without + reinterpreting substrate semantics +- coordinate with `PROTO_playback-head-alignment` so external consumers follow + stable read nouns instead of inventing them early diff --git a/docs/method/retro/0007-viewpoint-design/viewpoint-design.md b/docs/method/retro/0007-viewpoint-design/viewpoint-design.md new file mode 100644 index 00000000..6105a580 --- /dev/null +++ b/docs/method/retro/0007-viewpoint-design/viewpoint-design.md @@ -0,0 +1,125 @@ +# Cycle 0007 Retro — WorldlineSelector + defaultCodec + +## Outcome + +**Partial.** + +WorldlineSelector hierarchy: shipped (PR #77). All 5,203 tests pass. +defaultCodec migration: failed. Reverted. Backlog item rewritten. + +## What went well + +### WorldlineSelector + +- RED first. 51 tests written before implementation. +- Real `extends` — `instanceof WorldlineSelector` works. +- Constructor validation — rejects bad ceiling, empty strandId, + non-object frontier. +- `#frontier` private field with defensive copy getter — real Map + immutability, not fake `Object.freeze`. +- `toDTO()` bridge — public API unchanged, internal code clean. +- Self-review caught 7 issues including a behavioral regression + (toDTO ceiling omission), double-clone waste, and registry + hijack vector. All fixed before merge. +- Theory alignment: noun audit (cycle 0006) informed the naming. + "WorldlineSelector" is the brutally literal name. "Viewpoint" + was rejected. The design doc maps the concept against all 7 + papers. + +### Design process + +- Cycle 0005 failure (fake classes, no validation, kind tags kept) + directly informed cycle 0007's design. The retro worked. +- Human sponsor caught the "Viewpoint is weird" problem and pushed + for the observer/writer distinction that clarified the concept. +- The noun audit (cycle 0006) was the right intermediate step — + design before code. + +## What went wrong + +### defaultCodec + +Attempted to move `defaultCodec.js` to infrastructure. Three +approaches tried, all wrong: + +1. **Re-export shim** — "leaves stanky tech debt behind." Hides the + concrete dependency behind indirection without fixing the design. +2. **Thread codec through constructors** — 348 test failures. The + codec injection chain is incomplete: WarpRuntime passes codec to + some services, but many leaf services (index builders, serializers) + construct sub-services without threading codec through. +3. **Revert to shim after failure** — "I didn't say go back to the + shim." + +The root cause: **the problem was misdiagnosed.** The original backlog +item said "move defaultCodec.js to infrastructure" — a file move. +The real P5 violation is that 20 domain services call +`codec.encode()`/`codec.decode()` directly. Domain services are doing +serialization. Moving the file doesn't fix that. + +`defaultCodec` is a singleton pretending to be dependency injection. +Every service can bypass its caller by importing the global. The +`codec` constructor param is theater. + +### Speed over understanding (again) + +Same failure mode as cycle 0005. Jumped to implementation without +understanding why the code is shaped the way it is. The design doc +was written to justify the approach ("shim is fine" then "thread it +through"), not to understand the problem. + +## What the redo needs + +This is now an L-effort architectural item, not an S-effort file +move. Backlog item rewritten as +`NDNM_defaultcodec-to-infrastructure.md` with the full audit of +20 offending services and a phased approach. + +**Corrected 2026-04-04:** The original redo plan (below, struck) was +still wrong — it kept serializer services alive, just in a different +folder. "Dissolve into adapters" was also wrong — it would create a +god object in GitGraphAdapter. + +~~1. Delete dead code (canonicalCbor.js)~~ +~~2. Audit which services' primary concern IS serialization~~ +~~3. Move serialization-primary services to infrastructure~~ +~~4. For the rest, delegate serialization to adapters~~ +~~5. When no domain service imports defaultCodec, delete it~~ + +The real fix is a two-stage boundary: domain-facing artifact ports +that speak domain types (PatchJournalPort, CheckpointStorePort, etc.) +backed by infrastructure adapters that own the codec and talk to the +raw Git ports. Serializer files that mix domain logic with byte +encoding get split — projection stays in domain, encoding goes behind +the adapter. Strangler refactor: patches first, then checkpoints, +then indexes, then provenance/BTR. + +See `NDNM_defaultcodec-to-infrastructure.md` for the full plan. + +## Drift check + +WorldlineSelector: no drift from design doc. Shipped as designed. +defaultCodec: massive drift — the design doc was rewritten three +times during the cycle, which is itself a signal that the problem +wasn't understood. + +## New debt + +- `canonicalCbor.js` was dead code (imported by nothing, tested but + unused). Deleted in this PR. + +## Cool ideas + +- The observer/writer distinction (observer = π, writer = full optic + Ω) could inform how other codebase nouns evolve. Writers own + frontiers and produce witnesses. Observers just project. +- The `toDTO()` bridge pattern (internal classes, external plain + objects) could apply to other domain types that have public API + surface — clean internal modeling without breaking consumers. + +## Backlog maintenance + +- `NDNM_defaultcodec-to-infrastructure` rewritten from S to L, + reframed as architectural serialization extraction +- `NDNM_worldlinesource-to-viewpoint-hierarchy` consumed by this + cycle (WorldlineSelector shipped) diff --git a/eslint.config.js b/eslint.config.js index 62c60ce0..e8ea192a 100644 --- a/eslint.config.js +++ b/eslint.config.js @@ -262,6 +262,7 @@ export default tseslint.config( "src/domain/warp/PatchSession.js", "src/domain/utils/EventId.js", "src/domain/types/WarpTypesV2.js", + "src/domain/types/WorldlineSelector.js", "src/visualization/renderers/ascii/graph.js", "src/domain/services/KeyCodec.js", "src/domain/services/dag/DagTraversal.js", @@ -294,6 +295,11 @@ export default tseslint.config( "src/domain/services/state/StateReaderV5.js", "src/domain/services/sync/SyncAuthService.js", "src/infrastructure/adapters/GitGraphAdapter.js", + "src/infrastructure/adapters/CborCheckpointStoreAdapter.js", + "src/infrastructure/adapters/CborPatchJournalAdapter.js", + "src/infrastructure/adapters/IndexShardEncodeTransform.js", + "src/domain/stream/WarpStream.js", + "src/domain/artifacts/IndexShard.js", "src/visualization/renderers/ascii/path.js", "src/domain/services/strand/StrandService.js", "src/domain/services/query/AdjacencyNeighborProvider.js", diff --git a/index.d.ts b/index.d.ts index b397c175..d56b35e6 100644 --- a/index.d.ts +++ b/index.d.ts @@ -1440,6 +1440,42 @@ export interface StrandObserverSource { /** Union of observer source types for worldline creation. */ export type WorldlineSource = LiveObserverSource | CoordinateObserverSource | StrandObserverSource; +/** Abstract base for worldline selectors. */ +export class WorldlineSelector { + /** Deep-clone this selector. */ + clone(): WorldlineSelector; + /** Convert to a plain DTO matching the WorldlineSource shape. */ + toDTO(): WorldlineSource; + /** Normalize a raw source or plain object into a selector instance. */ + static from(raw: WorldlineSelector | WorldlineSource | null | undefined): WorldlineSelector; +} + +/** Worldline selector for the canonical (live) worldline. */ +export class LiveSelector extends WorldlineSelector { + constructor(ceiling?: number | null); + readonly ceiling: number | null; + clone(): LiveSelector; + toDTO(): LiveObserverSource; +} + +/** Worldline selector for a hypothetical worldline at specific writer tips. */ +export class CoordinateSelector extends WorldlineSelector { + constructor(frontier: Map | Record, ceiling?: number | null); + readonly frontier: Map; + readonly ceiling: number | null; + clone(): CoordinateSelector; + toDTO(): CoordinateObserverSource; +} + +/** Worldline selector for one writer's isolated worldline. */ +export class StrandSelector extends WorldlineSelector { + constructor(strandId: string, ceiling?: number | null); + readonly strandId: string; + readonly ceiling: number | null; + clone(): StrandSelector; + toDTO(): StrandObserverSource; +} + /** Options for creating a worldline handle. */ export interface WorldlineOptions { source?: WorldlineSource; diff --git a/index.js b/index.js index a7f009d5..d1bb2c9b 100644 --- a/index.js +++ b/index.js @@ -94,6 +94,10 @@ import { migrateV4toV5 } from './src/domain/services/MigrationService.js'; import QueryBuilder from './src/domain/services/query/QueryBuilder.js'; import Observer from './src/domain/services/query/Observer.js'; import Worldline from './src/domain/services/Worldline.js'; +import WorldlineSelector from './src/domain/types/WorldlineSelector.js'; +import LiveSelector from './src/domain/types/LiveSelector.js'; +import CoordinateSelector from './src/domain/types/CoordinateSelector.js'; +import StrandSelector from './src/domain/types/StrandSelector.js'; import { computeTranslationCost } from './src/domain/services/TranslationCost.js'; import { encodeEdgePropKey, @@ -237,6 +241,10 @@ export { WarpApp, WarpCore, Worldline, + WorldlineSelector, + LiveSelector, + CoordinateSelector, + StrandSelector, QueryBuilder, Observer, PatchBuilderV2, diff --git a/src/domain/WarpRuntime.js b/src/domain/WarpRuntime.js index 145505c4..e3f1be5f 100644 --- a/src/domain/WarpRuntime.js +++ b/src/domain/WarpRuntime.js @@ -30,6 +30,7 @@ import CheckpointController from './services/controllers/CheckpointController.js import SyncTrustGate from './services/sync/SyncTrustGate.js'; import { AuditVerifierService } from './services/audit/AuditVerifierService.js'; import MaterializedViewService from './services/MaterializedViewService.js'; +import StateHashService from './services/state/StateHashService.js'; import InMemoryBlobStorageAdapter from './utils/defaultBlobStorage.js'; // checkpoint.methods.js replaced by CheckpointController (imported above) // patch.methods.js replaced by PatchController (imported above) @@ -360,6 +361,15 @@ export default class WarpRuntime { /** @type {import('./services/EffectPipeline.js').EffectPipeline|null} */ this._effectPipeline = null; + + /** @type {import('../ports/PatchJournalPort.js').default|null} */ + this._patchJournal = null; + + /** @type {import('../ports/CheckpointStorePort.js').default|null} */ + this._checkpointStore = null; + + /** @type {StateHashService|null} */ + this._stateHashService = null; } /** @@ -478,7 +488,7 @@ export default class WarpRuntime { /** * Opens a multi-writer graph. * - * @param {{ persistence: CorePersistence, graphName: string, writerId: string, gcPolicy?: Record, adjacencyCacheSize?: number, checkpointPolicy?: {every: number}, autoMaterialize?: boolean, onDeleteWithData?: 'reject'|'cascade'|'warn', logger?: import('../ports/LoggerPort.js').default, clock?: import('../ports/ClockPort.js').default, crypto?: import('../ports/CryptoPort.js').default, codec?: import('../ports/CodecPort.js').default, seekCache?: import('../ports/SeekCachePort.js').default, audit?: boolean, blobStorage?: import('../ports/BlobStoragePort.js').default, patchBlobStorage?: import('../ports/BlobStoragePort.js').default, trust?: { mode?: 'off'|'log-only'|'enforce', pin?: string|null }, effectPipeline?: import('./services/EffectPipeline.js').EffectPipeline, effectSinks?: Array, externalizationPolicy?: import('./types/ExternalizationPolicy.js').ExternalizationPolicy }} options + * @param {{ persistence: CorePersistence, graphName: string, writerId: string, gcPolicy?: Record, adjacencyCacheSize?: number, checkpointPolicy?: {every: number}, autoMaterialize?: boolean, onDeleteWithData?: 'reject'|'cascade'|'warn', logger?: import('../ports/LoggerPort.js').default, clock?: import('../ports/ClockPort.js').default, crypto?: import('../ports/CryptoPort.js').default, codec?: import('../ports/CodecPort.js').default, seekCache?: import('../ports/SeekCachePort.js').default, audit?: boolean, blobStorage?: import('../ports/BlobStoragePort.js').default, patchBlobStorage?: import('../ports/BlobStoragePort.js').default, patchJournal?: import('../ports/PatchJournalPort.js').default | null, checkpointStore?: import('../ports/CheckpointStorePort.js').default | null, trust?: { mode?: 'off'|'log-only'|'enforce', pin?: string|null }, effectPipeline?: import('./services/EffectPipeline.js').EffectPipeline, effectSinks?: Array, externalizationPolicy?: import('./types/ExternalizationPolicy.js').ExternalizationPolicy }} options * @returns {Promise} The opened graph instance * @throws {WarpError} If graphName, writerId, checkpointPolicy, or onDeleteWithData is invalid * @@ -491,7 +501,7 @@ export default class WarpRuntime { */ // TODO(OG): split open() validation/bootstrapping; legacy hotspot kept explicit until the API redesign cycle. // eslint-disable-next-line max-lines-per-function, complexity - static async open({ persistence, graphName, writerId, gcPolicy = {}, adjacencyCacheSize, checkpointPolicy, autoMaterialize, onDeleteWithData, logger, clock, crypto, codec, seekCache, audit, blobStorage, patchBlobStorage, trust, effectPipeline, effectSinks, externalizationPolicy }) { + static async open({ persistence, graphName, writerId, gcPolicy = {}, adjacencyCacheSize, checkpointPolicy, autoMaterialize, onDeleteWithData, logger, clock, crypto, codec, seekCache, audit, blobStorage, patchBlobStorage, patchJournal, checkpointStore, trust, effectPipeline, effectSinks, externalizationPolicy }) { // Validate inputs validateGraphName(graphName); validateWriterId(writerId); @@ -535,6 +545,44 @@ export default class WarpRuntime { const graph = new WarpRuntime({ persistence, graphName, writerId, gcPolicy, ...(adjacencyCacheSize !== undefined ? { adjacencyCacheSize } : {}), ...(checkpointPolicy !== undefined ? { checkpointPolicy } : {}), ...(autoMaterialize !== undefined ? { autoMaterialize } : {}), ...(onDeleteWithData !== undefined ? { onDeleteWithData } : {}), ...(logger !== undefined ? { logger } : {}), ...(clock !== undefined ? { clock } : {}), ...(crypto !== undefined ? { crypto } : {}), ...(codec !== undefined ? { codec } : {}), ...(seekCache !== undefined ? { seekCache } : {}), ...(audit !== undefined ? { audit } : {}), blobStorage: resolvedBlobStorage, ...(patchBlobStorage !== undefined ? { patchBlobStorage } : {}), ...(trust !== undefined ? { trust } : {}) }); + // Auto-construct patchJournal when none provided: uses the same dynamic import + // pattern as autoConstructBlobStorage to keep infrastructure imports out of the + // module's top-level scope. + if (patchJournal !== undefined && patchJournal !== null) { + graph._patchJournal = /** @type {import('../ports/PatchJournalPort.js').default} */ (patchJournal); + } else { + const { CborPatchJournalAdapter } = await import( + /* webpackIgnore: true */ '../infrastructure/adapters/CborPatchJournalAdapter.js' + ); + graph._patchJournal = new CborPatchJournalAdapter({ + codec: graph._codec, + blobPort: /** @type {import('../ports/BlobPort.js').default} */ (persistence), + commitPort: /** @type {import('../ports/CommitPort.js').default} */ (persistence), + ...(patchBlobStorage !== undefined && patchBlobStorage !== null ? { patchBlobStorage } : {}), + }); + } + + // Auto-construct checkpointStore when none provided: same pattern as patchJournal. + if (checkpointStore !== undefined && checkpointStore !== null) { + graph._checkpointStore = /** @type {import('../ports/CheckpointStorePort.js').default} */ (checkpointStore); + } else { + const { CborCheckpointStoreAdapter } = await import( + /* webpackIgnore: true */ '../infrastructure/adapters/CborCheckpointStoreAdapter.js' + ); + graph._checkpointStore = new CborCheckpointStoreAdapter({ + codec: graph._codec, + blobPort: /** @type {import('../ports/BlobPort.js').default} */ (persistence), + }); + } + + // Auto-construct StateHashService from codec + crypto (only when crypto is available) + if (graph._crypto !== undefined && graph._crypto !== null) { + graph._stateHashService = new StateHashService({ + codec: graph._codec, + crypto: graph._crypto, + }); + } + // Validate migration boundary await graph._validateMigrationBoundary(); diff --git a/src/domain/artifacts/CheckpointArtifact.js b/src/domain/artifacts/CheckpointArtifact.js new file mode 100644 index 00000000..27c0c8bf --- /dev/null +++ b/src/domain/artifacts/CheckpointArtifact.js @@ -0,0 +1,90 @@ +import WarpError from '../errors/WarpError.js'; + +/** + * Abstract base class for checkpoint artifacts. + * + * A checkpoint is one domain event with multiple persistence artifacts. + * Each artifact carries a domain payload. The adapter maps artifacts to + * Git tree paths at the last responsible moment. + * + * Subclasses: StateArtifact, FrontierArtifact, AppliedVVArtifact. + * + * @abstract + */ +export class CheckpointArtifact { + /** + * Creates a CheckpointArtifact. + * + * @param {{ schemaVersion: number }} fields + */ + constructor({ schemaVersion }) { + if (typeof schemaVersion !== 'number' || !Number.isInteger(schemaVersion) || schemaVersion < 1) { + throw new WarpError( + `CheckpointArtifact schemaVersion must be a positive integer, got ${JSON.stringify(schemaVersion)}`, + 'E_INVALID_ARTIFACT', + ); + } + /** @type {number} */ + this.schemaVersion = schemaVersion; + } +} + +/** + * Carries the full CRDT state for checkpoint recovery. + */ +export class StateArtifact extends CheckpointArtifact { + /** + * Creates a StateArtifact. + * + * @param {{ schemaVersion: number, state: import('../services/JoinReducer.js').WarpStateV5 }} fields + */ + constructor({ schemaVersion, state }) { + super({ schemaVersion }); + if (state === null || state === undefined) { + throw new WarpError('StateArtifact requires a state', 'E_INVALID_ARTIFACT'); + } + /** @type {import('../services/JoinReducer.js').WarpStateV5} */ + this.state = state; + Object.freeze(this); + } +} + +/** + * Carries the writer frontier for checkpoint recovery. + */ +export class FrontierArtifact extends CheckpointArtifact { + /** + * Creates a FrontierArtifact. + * + * @param {{ schemaVersion: number, frontier: Map }} fields + */ + constructor({ schemaVersion, frontier }) { + super({ schemaVersion }); + if (!(frontier instanceof Map)) { + throw new WarpError('FrontierArtifact requires a Map frontier', 'E_INVALID_ARTIFACT'); + } + /** @type {Map} */ + this.frontier = frontier; + Object.freeze(this); + } +} + +/** + * Carries the applied version vector for checkpoint recovery. + */ +export class AppliedVVArtifact extends CheckpointArtifact { + /** + * Creates an AppliedVVArtifact. + * + * @param {{ schemaVersion: number, appliedVV: import('../crdt/VersionVector.js').default }} fields + */ + constructor({ schemaVersion, appliedVV }) { + super({ schemaVersion }); + if (appliedVV === null || appliedVV === undefined) { + throw new WarpError('AppliedVVArtifact requires an appliedVV', 'E_INVALID_ARTIFACT'); + } + /** @type {import('../crdt/VersionVector.js').default} */ + this.appliedVV = appliedVV; + Object.freeze(this); + } +} diff --git a/src/domain/artifacts/IndexShard.js b/src/domain/artifacts/IndexShard.js new file mode 100644 index 00000000..7bcb4119 --- /dev/null +++ b/src/domain/artifacts/IndexShard.js @@ -0,0 +1,142 @@ +import WarpError from '../errors/WarpError.js'; + +/** + * Abstract base class for index shards. + * + * Index builders produce IndexShard subclass instances. The adapter + * maps each subclass to a Git tree path and CBOR-encodes it. The + * domain never knows about paths or encoding. + * + * Subclasses: MetaShard, EdgeShard, LabelShard, PropertyShard, + * ReceiptShard. + * + * @abstract + */ +export class IndexShard { + /** + * Creates an IndexShard. + * + * @param {{ shardKey: string, schemaVersion: number }} fields + */ + constructor({ shardKey, schemaVersion }) { + if (typeof shardKey !== 'string' || shardKey.length === 0) { + throw new WarpError( + `IndexShard shardKey must be a non-empty string, got ${JSON.stringify(shardKey)}`, + 'E_INVALID_SHARD', + ); + } + if (typeof schemaVersion !== 'number' || !Number.isInteger(schemaVersion) || schemaVersion < 1) { + throw new WarpError( + `IndexShard schemaVersion must be a positive integer, got ${JSON.stringify(schemaVersion)}`, + 'E_INVALID_SHARD', + ); + } + /** @type {string} */ + this.shardKey = shardKey; + /** @type {number} */ + this.schemaVersion = schemaVersion; + } +} + +/** + * Node-to-global-ID mappings + alive bitmap for a shard. + */ +export class MetaShard extends IndexShard { + /** + * Creates a MetaShard. + * + * @param {{ shardKey: string, schemaVersion?: number, nodeToGlobal: Array<[string, number]>, nextLocalId: number, alive: Uint8Array }} fields + */ + constructor({ shardKey, schemaVersion = 1, nodeToGlobal, nextLocalId, alive }) { + super({ shardKey, schemaVersion }); + /** @type {Array<[string, number]>} */ + this.nodeToGlobal = nodeToGlobal; + /** @type {number} */ + this.nextLocalId = nextLocalId; + /** @type {Uint8Array} */ + this.alive = alive; + Object.freeze(this); + } +} + +/** + * Forward or reverse edge bitmaps for a shard. + */ +export class EdgeShard extends IndexShard { + /** + * Creates an EdgeShard. + * + * @param {{ shardKey: string, schemaVersion?: number, direction: 'fwd'|'rev', buckets: Record> }} fields + */ + constructor({ shardKey, schemaVersion = 1, direction, buckets }) { + super({ shardKey, schemaVersion }); + if (direction !== 'fwd' && direction !== 'rev') { + throw new WarpError( + `EdgeShard direction must be 'fwd' or 'rev', got ${JSON.stringify(direction)}`, + 'E_INVALID_SHARD', + ); + } + /** @type {'fwd'|'rev'} */ + this.direction = direction; + /** @type {Record>} */ + this.buckets = buckets; + Object.freeze(this); + } +} + +/** + * Label registry (append-only label-to-ID mapping). + */ +export class LabelShard extends IndexShard { + /** + * Creates a LabelShard. + * + * @param {{ shardKey?: string, schemaVersion?: number, labels: Array<[string, number]> }} fields + */ + constructor({ shardKey = 'global', schemaVersion = 1, labels }) { + super({ shardKey, schemaVersion }); + /** @type {Array<[string, number]>} */ + this.labels = labels; + Object.freeze(this); + } +} + +/** + * Property index data for a shard. + */ +export class PropertyShard extends IndexShard { + /** + * Creates a PropertyShard. + * + * @param {{ shardKey: string, schemaVersion?: number, entries: Array<[string, Record]> }} fields + */ + constructor({ shardKey, schemaVersion = 1, entries }) { + super({ shardKey, schemaVersion }); + /** @type {Array<[string, Record]>} */ + this.entries = entries; + Object.freeze(this); + } +} + +/** + * Build metadata receipt. + */ +export class ReceiptShard extends IndexShard { + /** + * Creates a ReceiptShard. + * + * @param {{ shardKey?: string, schemaVersion?: number, version: number, nodeCount: number, labelCount: number, shardCount: number }} fields + */ + constructor({ shardKey = 'receipt', schemaVersion = 1, version, nodeCount, labelCount, shardCount }) { + super({ shardKey, schemaVersion }); + /** @type {number} */ + this.version = version; + /** @type {number} */ + this.nodeCount = nodeCount; + /** @type {number} */ + this.labelCount = labelCount; + /** @type {number} */ + this.shardCount = shardCount; + Object.freeze(this); + } +} diff --git a/src/domain/artifacts/PatchEntry.js b/src/domain/artifacts/PatchEntry.js new file mode 100644 index 00000000..70e4071a --- /dev/null +++ b/src/domain/artifacts/PatchEntry.js @@ -0,0 +1,28 @@ +import WarpError from '../errors/WarpError.js'; + +/** + * A patch entry from a patch scan stream. + * + * Pairs a decoded PatchV2 with its commit SHA. This is the semantic + * unit yielded by PatchJournalPort.scanPatchRange(). + */ +export default class PatchEntry { + /** + * Creates a PatchEntry. + * + * @param {{ patch: import('../types/WarpTypesV2.js').PatchV2, sha: string }} fields + */ + constructor({ patch, sha }) { + if (patch === null || patch === undefined) { + throw new WarpError('PatchEntry requires a patch', 'E_INVALID_ENTRY'); + } + if (typeof sha !== 'string' || sha.length === 0) { + throw new WarpError('PatchEntry requires a non-empty sha', 'E_INVALID_ENTRY'); + } + /** @type {import('../types/WarpTypesV2.js').PatchV2} */ + this.patch = patch; + /** @type {string} */ + this.sha = sha; + Object.freeze(this); + } +} diff --git a/src/domain/artifacts/ProvenanceEntry.js b/src/domain/artifacts/ProvenanceEntry.js new file mode 100644 index 00000000..5e12b792 --- /dev/null +++ b/src/domain/artifacts/ProvenanceEntry.js @@ -0,0 +1,29 @@ +import WarpError from '../errors/WarpError.js'; + +/** + * A provenance entry from a provenance scan stream. + * + * Maps an entity (node or edge) to the set of patch SHAs that + * produced it. This is the semantic unit yielded by + * ProvenanceStorePort.scanEntries(). + */ +export default class ProvenanceEntry { + /** + * Creates a ProvenanceEntry. + * + * @param {{ entityId: string, patchShas: Set }} fields + */ + constructor({ entityId, patchShas }) { + if (typeof entityId !== 'string' || entityId.length === 0) { + throw new WarpError('ProvenanceEntry requires a non-empty entityId', 'E_INVALID_ENTRY'); + } + if (!(patchShas instanceof Set)) { + throw new WarpError('ProvenanceEntry requires a Set of patchShas', 'E_INVALID_ENTRY'); + } + /** @type {string} */ + this.entityId = entityId; + /** @type {Set} */ + this.patchShas = patchShas; + Object.freeze(this); + } +} diff --git a/src/domain/services/PatchBuilderV2.js b/src/domain/services/PatchBuilderV2.js index 952b9fa3..f8941424 100644 --- a/src/domain/services/PatchBuilderV2.js +++ b/src/domain/services/PatchBuilderV2.js @@ -11,7 +11,6 @@ * @see WARP v5 Spec */ -import defaultCodec from '../utils/defaultCodec.js'; import nullLogger from '../utils/nullLogger.js'; import { vvSerialize } from '../crdt/VersionVector.js'; import { orsetGetDots, orsetContains, orsetElements } from '../crdt/ORSet.js'; @@ -40,6 +39,7 @@ import WriterError from '../errors/WriterError.js'; import { isStreamingInput, normalizeToAsyncIterable } from '../utils/streamUtils.js'; import { canonicalStringify } from '../utils/canonicalStringify.js'; import PatchError from '../errors/PatchError.js'; +import PersistenceError from '../errors/PersistenceError.js'; /** * Inspects materialized state for edges and properties attached to a node. @@ -164,9 +164,9 @@ export class PatchBuilderV2 { /** * Creates a new PatchBuilderV2. * - * @param {{ persistence: import('../../ports/CommitPort.js').default & import('../../ports/BlobPort.js').default & import('../../ports/TreePort.js').default & import('../../ports/RefPort.js').default, graphName: string, writerId: string, lamport: number, versionVector: import('../crdt/VersionVector.js').default, getCurrentState: () => import('./JoinReducer.js').WarpStateV5 | null, expectedParentSha?: string|null, targetRefPath?: string, onCommitSuccess?: ((result: {patch: import('../types/WarpTypesV2.js').PatchV2, sha: string}) => void | Promise)|null, onDeleteWithData?: 'reject'|'cascade'|'warn', codec?: import('../../ports/CodecPort.js').default, logger?: import('../../ports/LoggerPort.js').default, blobStorage?: import('../../ports/BlobStoragePort.js').default, patchBlobStorage?: import('../../ports/BlobStoragePort.js').default }} options + * @param {{ persistence: import('../../ports/CommitPort.js').default & import('../../ports/BlobPort.js').default & import('../../ports/TreePort.js').default & import('../../ports/RefPort.js').default, graphName: string, writerId: string, lamport: number, versionVector: import('../crdt/VersionVector.js').default, getCurrentState: () => import('./JoinReducer.js').WarpStateV5 | null, expectedParentSha?: string|null, targetRefPath?: string, onCommitSuccess?: ((result: {patch: import('../types/WarpTypesV2.js').PatchV2, sha: string}) => void | Promise)|null, onDeleteWithData?: 'reject'|'cascade'|'warn', patchJournal?: import('../../ports/PatchJournalPort.js').default, logger?: import('../../ports/LoggerPort.js').default, blobStorage?: import('../../ports/BlobStoragePort.js').default }} options */ - constructor({ persistence, graphName, writerId, lamport, versionVector, getCurrentState, expectedParentSha = null, targetRefPath, onCommitSuccess = null, onDeleteWithData = 'warn', codec, logger, blobStorage, patchBlobStorage }) { + constructor({ persistence, graphName, writerId, lamport, versionVector, getCurrentState, expectedParentSha = null, targetRefPath, onCommitSuccess = null, onDeleteWithData = 'warn', patchJournal, logger, blobStorage }) { /** @type {import('../../ports/CommitPort.js').default & import('../../ports/BlobPort.js').default & import('../../ports/TreePort.js').default & import('../../ports/RefPort.js').default} */ this._persistence = /** @type {import('../../ports/CommitPort.js').default & import('../../ports/BlobPort.js').default & import('../../ports/TreePort.js').default & import('../../ports/RefPort.js').default} */ (persistence); @@ -217,8 +217,8 @@ export class PatchBuilderV2 { /** @type {'reject'|'cascade'|'warn'} */ this._onDeleteWithData = onDeleteWithData; - /** @type {import('../../ports/CodecPort.js').default} */ - this._codec = codec || defaultCodec; + /** @type {import('../../ports/PatchJournalPort.js').default|null} */ + this._patchJournal = patchJournal || null; /** @type {import('../../ports/LoggerPort.js').default} */ this._logger = logger || nullLogger; @@ -233,9 +233,6 @@ export class PatchBuilderV2 { /** @type {import('../../ports/BlobStoragePort.js').default|null} */ this._blobStorage = blobStorage || null; - /** @type {import('../../ports/BlobStoragePort.js').default|null} */ - this._patchBlobStorage = patchBlobStorage || null; - /** * Observed operands — entities whose current state was consulted to build * this patch. @@ -985,11 +982,11 @@ export class PatchBuilderV2 { writes: [...this._writes].sort(), }); - // 6. Encode patch as CBOR and write as a Git blob (or encrypted CAS asset) - const patchCbor = this._codec.encode(patch); - const patchBlobOid = this._patchBlobStorage - ? await this._patchBlobStorage.store(patchCbor, { slug: `${this._graphName}/${this._writerId}/patch` }) - : await this._persistence.writeBlob(patchCbor); + // 6. Persist patch via PatchJournalPort (adapter owns encoding). + if (this._patchJournal === null || this._patchJournal === undefined) { + throw new PersistenceError('patchJournal is required for committing patches', 'E_MISSING_JOURNAL'); + } + const patchBlobOid = await this._patchJournal.writePatch(patch); // 7. Create tree with the patch blob + any content blobs (deduplicated) // Format for mktree: "mode type oid\tpath" @@ -1011,7 +1008,8 @@ export class PatchBuilderV2 { // "encrypted" is a legacy wire name meaning "patch blob stored externally // via patchBlobStorage" (see ADR-0002). The flag tells readers to retrieve // the blob via BlobStoragePort instead of reading it directly from Git. - encrypted: !!this._patchBlobStorage, + // eslint-disable-next-line @typescript-eslint/strict-boolean-expressions -- WarpRuntime options are untyped; cast narrows + encrypted: this._patchJournal ? this._patchJournal.usesExternalStorage : false, }); const parents = (parentCommit !== null && parentCommit !== '') ? [parentCommit] : []; const newCommitSha = await this._persistence.commitNodeWithTree({ diff --git a/src/domain/services/Worldline.js b/src/domain/services/Worldline.js index a7effa7c..f9fd38ed 100644 --- a/src/domain/services/Worldline.js +++ b/src/domain/services/Worldline.js @@ -12,6 +12,9 @@ import QueryBuilder from './query/QueryBuilder.js'; import LogicalTraversal from './query/LogicalTraversal.js'; import { toInternalStrandShape } from '../utils/strandPublicShape.js'; import { callInternalRuntimeMethod } from '../utils/callInternalRuntimeMethod.js'; +import WorldlineSelector from '../types/WorldlineSelector.js'; +import LiveSelector from '../types/LiveSelector.js'; +import CoordinateSelector from '../types/CoordinateSelector.js'; /** @import { ObserverConfig, WorldlineOptions, WorldlineSource } from '../../../index.js' */ @@ -33,47 +36,13 @@ import { callInternalRuntimeMethod } from '../utils/callInternalRuntimeMethod.js */ /** - * Deep-clones a worldline source descriptor, normalizing null/undefined to live. + * Converts a raw source descriptor to a WorldlineSelector and clones it. * - * @param {WorldlineSource|{ kind: 'strand', strandId: string, ceiling?: number|null }|undefined|null} source - * @returns {WorldlineSource} + * @param {WorldlineSelector|WorldlineSource|{ kind: string, [key: string]: unknown }|undefined|null} source + * @returns {import('../types/WorldlineSelector.js').default} */ -function cloneWorldlineSource(source) { - const value = source ?? { kind: 'live' }; - - if (value.kind === 'live') { - return cloneLiveSource(value); - } - if (value.kind === 'coordinate') { - return cloneCoordinateSource(value); - } - return { kind: 'strand', strandId: value.strandId, ceiling: value.ceiling ?? null }; -} - -/** - * Clones a live source, preserving ceiling only if present. - * - * @param {{ kind: 'live', ceiling?: number|null }} value - * @returns {WorldlineSource} - */ -function cloneLiveSource(value) { - return 'ceiling' in value - ? { kind: 'live', ceiling: value.ceiling ?? null } - : { kind: 'live' }; -} - -/** - * Clones a coordinate source, deep-copying the frontier. - * - * @param {{ kind: 'coordinate', frontier: Map|Record, ceiling?: number|null }} value - * @returns {WorldlineSource} - */ -function cloneCoordinateSource(value) { - return { - kind: 'coordinate', - frontier: value.frontier instanceof Map ? new Map(value.frontier) : { ...value.frontier }, - ceiling: value.ceiling ?? null, - }; +function toSelector(source) { + return WorldlineSelector.from(/** @type {WorldlineSelector|{ kind: string, [key: string]: unknown }|null|undefined} */ (source)).clone(); } /** @@ -114,7 +83,7 @@ function buildDetachedOpenOptions(graph) { * Collects optional nullable fields, converting null to undefined for .open() compatibility. * * @param {WarpRuntime} graph - * @returns {{ checkpointPolicy?: unknown, logger?: unknown, seekCache?: unknown, blobStorage?: unknown, patchBlobStorage?: unknown }} + * @returns {{ checkpointPolicy?: unknown, logger?: unknown, seekCache?: unknown, blobStorage?: unknown, patchBlobStorage?: unknown, patchJournal?: unknown, checkpointStore?: unknown }} */ function nullableOpenFields(graph) { return { @@ -123,6 +92,8 @@ function nullableOpenFields(graph) { seekCache: orUndefined(graph._seekCache), blobStorage: orUndefined(graph._blobStorage), patchBlobStorage: orUndefined(graph._patchBlobStorage), + patchJournal: orUndefined(graph._patchJournal), + checkpointStore: orUndefined(graph._checkpointStore), }; } @@ -213,20 +184,20 @@ async function materializeStrandSource(graph, source, collectReceipts) { * Dispatches materialization to the handler for the source's kind. * * @param {WarpRuntime} graph - * @param {WorldlineSource} source + * @param {import('../types/WorldlineSelector.js').default} source * @param {boolean} collectReceipts * @returns {Promise} */ async function materializeSource(graph, source, collectReceipts) { - if (source.kind === 'live') { - return await materializeLiveSource(graph, source, collectReceipts); + if (source instanceof LiveSelector) { + return await materializeLiveSource(graph, /** @type {{ kind: 'live', ceiling?: number|null }} */ (/** @type {unknown} */ (source)), collectReceipts); } - if (source.kind === 'coordinate') { - return await materializeCoordinateSource(graph, source, collectReceipts); + if (source instanceof CoordinateSelector) { + return await materializeCoordinateSource(graph, /** @type {{ kind: 'coordinate', frontier: Map, ceiling?: number|null }} */ (/** @type {unknown} */ (source)), collectReceipts); } - return await materializeStrandSource(graph, source, collectReceipts); + return await materializeStrandSource(graph, /** @type {{ kind: 'strand', strandId: string, ceiling?: number|null }} */ (/** @type {unknown} */ (source)), collectReceipts); } /** @@ -236,14 +207,14 @@ export default class Worldline { /** * Creates a Worldline pinned to the given graph and source descriptor. * - * @param {{ graph: WarpRuntime, source?: WorldlineSource }} options + * @param {{ graph: WarpRuntime, source?: import('../types/WorldlineSelector.js').default }} options */ constructor({ graph, source }) { /** @type {WarpRuntime} */ this._graph = graph; - /** @type {WorldlineSource} */ - this._source = cloneWorldlineSource(source); + /** @type {import('../types/WorldlineSelector.js').default} */ + this._source = toSelector(source); /** @type {Promise|null} */ this._delegateObserverPromise = null; @@ -265,7 +236,7 @@ export default class Worldline { * @returns {WorldlineSource} */ get source() { - return cloneWorldlineSource(this._source); + return /** @type {WorldlineSource} */ (/** @type {WorldlineSource} */ (this._source.toDTO())); } /** @@ -279,9 +250,7 @@ export default class Worldline { async seek(options = undefined) { return await Promise.resolve(new Worldline({ graph: this._graph, - source: cloneWorldlineSource( - cloneWorldlineSource(options?.source || this._source), - ), + source: toSelector(options?.source || this._source), })); } @@ -307,7 +276,7 @@ export default class Worldline { if (!this._delegateObserverPromise) { this._delegateObserverPromise = this._graph.observer( { match: '*' }, - { source: cloneWorldlineSource(this._source) }, + { source: /** @type {WorldlineSource} */ (this._source.toDTO()) }, ); } return await this._delegateObserverPromise; @@ -388,13 +357,13 @@ export default class Worldline { if (typeof nameOrConfig === 'string') { // eslint-disable-next-line @typescript-eslint/no-unsafe-return -- return through defineProperty delegation; type is declared in @returns return await this._graph.observer(nameOrConfig, config, { - source: cloneWorldlineSource(this._source), + source: /** @type {WorldlineSource} */ (this._source.toDTO()), }); } // eslint-disable-next-line @typescript-eslint/no-unsafe-return -- return through defineProperty delegation; type is declared in @returns return await this._graph.observer(nameOrConfig, { - source: cloneWorldlineSource(this._source), + source: /** @type {WorldlineSource} */ (this._source.toDTO()), }); } } diff --git a/src/domain/services/controllers/CheckpointController.js b/src/domain/services/controllers/CheckpointController.js index 1dc4e21d..2fe71c88 100644 --- a/src/domain/services/controllers/CheckpointController.js +++ b/src/domain/services/controllers/CheckpointController.js @@ -86,6 +86,9 @@ export default class CheckpointController { /** @type {CorePersistence} */ const persistence = h._persistence; + /** @type {import('../../../ports/CheckpointStorePort.js').default|null} */ + const checkpointStore = /** @type {import('../../../ports/CheckpointStorePort.js').default|null} */ (h._checkpointStore); + const stateHashService = /** @type {import('../state/StateHashService.js').default|null} */ (h._stateHashService); const checkpointSha = await createCheckpointCommit({ persistence, graphName: h._graphName, @@ -96,6 +99,8 @@ export default class CheckpointController { crypto: h._crypto, codec: h._codec, ...(indexTree ? { indexTree } : {}), + ...(checkpointStore ? { checkpointStore } : {}), + ...(stateHashService ? { stateHashService } : {}), }); const checkpointRef = buildCheckpointRef(h._graphName); @@ -158,7 +163,9 @@ export default class CheckpointController { } try { - return await loadCheckpoint(h._persistence, checkpointSha, { codec: h._codec }); + /** @type {import('../../../ports/CheckpointStorePort.js').default|null} */ + const checkpointStore = /** @type {import('../../../ports/CheckpointStorePort.js').default|null} */ (h._checkpointStore); + return await loadCheckpoint(h._persistence, checkpointSha, { codec: h._codec, ...(checkpointStore ? { checkpointStore } : {}) }); } catch (err) { const msg = err instanceof Error ? err.message : ''; if ( diff --git a/src/domain/services/controllers/ComparisonController.js b/src/domain/services/controllers/ComparisonController.js index caa924a0..bd835482 100644 --- a/src/domain/services/controllers/ComparisonController.js +++ b/src/domain/services/controllers/ComparisonController.js @@ -761,6 +761,21 @@ function buildStrandMetadata(strandId, descriptor) { }; } +/** + * Computes the canonical state hash, preferring StateHashService when available. + * + * @param {import('../../WarpRuntime.js').default} graph + * @param {WarpStateV5} state + * @returns {Promise} + */ +async function computeStateHashForGraph(graph, state) { + const svc = /** @type {import('../state/StateHashService.js').default|null} */ (graph._stateHashService); + if (svc) { + return await svc.compute(state); + } + return await computeStateHashV5(state, { crypto: graph._crypto, codec: graph._codec }); +} + /** * Finalizes one side of a coordinate comparison with digests and summary. * @@ -783,8 +798,7 @@ async function finalizeSide(graph, params, scope) { const visiblePatchFrontier = patchFrontierFromEntries(scopedPatchEntries); const visibleLamportFrontier = lamportFrontierFromEntries(scopedPatchEntries); const reader = createStateReaderV5(scopedState); - - const stateHash = await computeStateHashV5(scopedState, { crypto: graph._crypto, codec: graph._codec }); + const stateHash = await computeStateHashForGraph(graph, scopedState); const patchShas = uniqueSortedPatchShas(scopedPatchEntries); return new ResolvedComparisonSide({ diff --git a/src/domain/services/controllers/MaterializeController.js b/src/domain/services/controllers/MaterializeController.js index 717655b1..35a0556b 100644 --- a/src/domain/services/controllers/MaterializeController.js +++ b/src/domain/services/controllers/MaterializeController.js @@ -144,6 +144,8 @@ async function openDetachedReadGraph(host) { ...(host._blobStorage ? { blobStorage: host._blobStorage } : {}), ...(host._patchBlobStorage ? { patchBlobStorage: host._patchBlobStorage } : {}), ...(host._trustConfig !== undefined ? { trust: host._trustConfig } : {}), + ...(host._checkpointStore ? { checkpointStore: host._checkpointStore } : {}), + ...(host._patchJournal !== undefined && host._patchJournal !== null ? { patchJournal: /** @type {import('../../../ports/PatchJournalPort.js').default} */ (host._patchJournal) } : {}), }); } @@ -622,7 +624,10 @@ export default class MaterializeController { h._stateDirty = false; h._versionVector = state.observedFrontier.clone(); - const stateHash = await computeStateHashV5(state, { crypto: h._crypto, codec: h._codec }); + const stateHashService = /** @type {import('../state/StateHashService.js').default|null} */ (h._stateHashService); + const stateHash = stateHashService + ? await stateHashService.compute(state) + : await computeStateHashV5(state, { crypto: h._crypto, codec: h._codec }); let adjacency; if (h._adjacencyCache) { diff --git a/src/domain/services/controllers/PatchController.js b/src/domain/services/controllers/PatchController.js index 31c2d2a0..9f1f8c83 100644 --- a/src/domain/services/controllers/PatchController.js +++ b/src/domain/services/controllers/PatchController.js @@ -53,10 +53,9 @@ export default class PatchController { expectedParentSha: parentSha, onDeleteWithData: h._onDeleteWithData, onCommitSuccess: /** Post-commit callback. @param {{patch?: import('../../types/WarpTypesV2.js').PatchV2, sha?: string}} opts */ (opts) => this._onPatchCommitted(h._writerId, opts), - codec: h._codec, + ...(h._patchJournal !== null && h._patchJournal !== undefined ? { patchJournal: /** @type {import('../../../ports/PatchJournalPort.js').default} */ (h._patchJournal) } : {}), ...(h._logger !== null && h._logger !== undefined ? { logger: h._logger } : {}), ...(h._blobStorage !== null && h._blobStorage !== undefined ? { blobStorage: h._blobStorage } : {}), - ...(h._patchBlobStorage !== null && h._patchBlobStorage !== undefined ? { patchBlobStorage: h._patchBlobStorage } : {}), }); } @@ -161,8 +160,23 @@ export default class PatchController { } const patchMeta = decodePatchMessage(message); - const patchBuffer = await this._readPatchBlob(patchMeta); - const decoded = /** @type {import('../../types/WarpTypesV2.js').PatchV2} */ (h._codec.decode(patchBuffer)); + /** @type {import('../../../ports/PatchJournalPort.js').default | null | undefined} */ + const journal = /** @type {import('../../../ports/PatchJournalPort.js').default | null | undefined} */ (h._patchJournal); + if (journal === null || journal === undefined) { + // Legacy fallback: read the patch blob directly and decode with the codec + const raw = await h._persistence.readBlob(patchMeta.patchOid); + const decoded = /** @type {import('../../types/WarpTypesV2.js').PatchV2} */ (h._codec.decode(raw)); + patches.push({ patch: decoded, sha: currentSha }); + if (Array.isArray(nodeInfo.parents) && nodeInfo.parents.length > 0) { + currentSha = nodeInfo.parents[0] ?? ''; + } else { + break; + } + continue; + } + const decoded = /** @type {import('../../types/WarpTypesV2.js').PatchV2} */ ( + await journal.readPatch(patchMeta.patchOid, { encrypted: patchMeta.encrypted }) + ); patches.push({ patch: decoded, sha: currentSha }); @@ -281,7 +295,8 @@ export default class PatchController { /** @type {CorePersistence} */ const persistence = h._persistence; - return new Writer({ + /** @type {ConstructorParameters[0]} */ + const writerOpts = { persistence, graphName: h._graphName, writerId: resolvedWriterId, @@ -289,11 +304,11 @@ export default class PatchController { getCurrentState: /** Returns the cached CRDT state. @returns {import('../JoinReducer.js').WarpStateV5|null} */ () => h._cachedState, onDeleteWithData: h._onDeleteWithData, onCommitSuccess: /** Post-commit callback. @type {(result: {patch: import('../../types/WarpTypesV2.js').PatchV2, sha: string}) => void} */ ((opts) => this._onPatchCommitted(resolvedWriterId, opts)), - codec: h._codec, - ...(h._logger !== null && h._logger !== undefined ? { logger: h._logger } : {}), - ...(h._blobStorage !== null && h._blobStorage !== undefined ? { blobStorage: h._blobStorage } : {}), - ...(h._patchBlobStorage !== null && h._patchBlobStorage !== undefined ? { patchBlobStorage: h._patchBlobStorage } : {}), - }); + patchJournal: /** @type {import('../../../ports/PatchJournalPort.js').default} */ (h._patchJournal), + }; + if (h._logger !== null && h._logger !== undefined) { writerOpts.logger = h._logger; } + if (h._blobStorage !== null && h._blobStorage !== undefined) { writerOpts.blobStorage = h._blobStorage; } + return new Writer(writerOpts); } /** diff --git a/src/domain/services/controllers/QueryController.js b/src/domain/services/controllers/QueryController.js index af599680..4ad70721 100644 --- a/src/domain/services/controllers/QueryController.js +++ b/src/domain/services/controllers/QueryController.js @@ -30,6 +30,10 @@ import { computeTranslationCost } from '../TranslationCost.js'; import { computeStateHashV5 } from '../state/StateSerializerV5.js'; import { toInternalStrandShape } from '../../utils/strandPublicShape.js'; import { callInternalRuntimeMethod } from '../../utils/callInternalRuntimeMethod.js'; +import WorldlineSelector from '../../types/WorldlineSelector.js'; +import LiveSelector from '../../types/LiveSelector.js'; +import CoordinateSelector from '../../types/CoordinateSelector.js'; +import StrandSelector from '../../types/StrandSelector.js'; /** * The host interface that QueryController depends on. @@ -55,41 +59,16 @@ import { callInternalRuntimeMethod } from '../../utils/callInternalRuntimeMethod */ /** - * Deep-clones an observer source descriptor for defensive copies. + * Converts a raw source to a WorldlineSelector, or returns null. * - * @param {ObserverOptions['source']|{ - * kind: 'strand', - * strandId: string, - * ceiling?: number|null - * }|undefined} source - * @returns {ObserverOptions['source']} + * @param {WorldlineSelector|{ kind: string, [key: string]: unknown }|undefined} source + * @returns {WorldlineSelector|undefined} */ -function cloneObserverSource(source) { +function toSelector(source) { if (!source) { return undefined; } - - if (source.kind === 'live') { - return 'ceiling' in source - ? { kind: 'live', ceiling: source.ceiling ?? null } - : { kind: 'live' }; - } - - if (source.kind === 'coordinate') { - return { - kind: 'coordinate', - frontier: source.frontier instanceof Map - ? new Map(source.frontier) - : { ...source.frontier }, - ceiling: source.ceiling ?? null, - }; - } - - return { - kind: 'strand', - strandId: source.strandId, - ceiling: source.ceiling ?? null, - }; + return WorldlineSelector.from(source).clone(); } /** @@ -117,6 +96,8 @@ async function openDetachedObserverGraph(graph) { ...(graph._blobStorage ? { blobStorage: graph._blobStorage } : {}), ...(graph._patchBlobStorage ? { patchBlobStorage: graph._patchBlobStorage } : {}), ...(graph._trustConfig !== undefined && graph._trustConfig !== null ? { trust: graph._trustConfig } : {}), + ...(graph._patchJournal !== undefined && graph._patchJournal !== null ? { patchJournal: /** @type {import('../../../ports/PatchJournalPort.js').default} */ (graph._patchJournal) } : {}), + ...(graph._checkpointStore ? { checkpointStore: graph._checkpointStore } : {}), }); } @@ -142,10 +123,10 @@ async function snapshotCurrentMaterialized(graph) { * @returns {Promise<{ state: import('../state/WarpStateV5.js').default, stateHash: string }>} */ async function snapshotReturnedState(graph, state) { - const stateHash = await computeStateHashV5(state, { - crypto: graph._crypto, - codec: graph._codec, - }); + const stateHashService = /** @type {import('../state/StateHashService.js').default|null} */ (graph._stateHashService); + const stateHash = stateHashService + ? await stateHashService.compute(state) + : await computeStateHashV5(state, { crypto: graph._crypto, codec: graph._codec }); return { state: cloneStateV5(state), stateHash, @@ -160,13 +141,13 @@ async function snapshotReturnedState(graph, state) { * @returns {Promise<{ state: import('../state/WarpStateV5.js').default, stateHash: string }>} */ async function resolveObserverSnapshot(graph, options) { - const source = cloneObserverSource(options?.source); + const source = toSelector(options?.source); if (!source) { await graph._ensureFreshState(); return await snapshotCurrentMaterialized(graph); } - if (source.kind === 'live') { + if (source instanceof LiveSelector) { const detached = await openDetachedObserverGraph(graph); const state = /** @type {import('../state/WarpStateV5.js').default} */ (await detached.materialize({ ceiling: source.ceiling ?? null, @@ -174,7 +155,7 @@ async function resolveObserverSnapshot(graph, options) { return await snapshotReturnedState(detached, state); } - if (source.kind === 'coordinate') { + if (source instanceof CoordinateSelector) { const detached = await openDetachedObserverGraph(graph); const state = /** @type {import('../state/WarpStateV5.js').default} */ (await detached.materializeCoordinate({ frontier: source.frontier, @@ -183,10 +164,10 @@ async function resolveObserverSnapshot(graph, options) { return await snapshotReturnedState(detached, state); } - if (source.kind === 'strand') { + if (source instanceof StrandSelector) { const detached = await openDetachedObserverGraph(graph); const internalSource = /** @type {{ strandId: string, ceiling?: number|null }} */ ( - /** @type {unknown} */ (toInternalStrandShape(source)) + /** @type {unknown} */ (toInternalStrandShape(source.toDTO())) ); const state = /** @type {import('../state/WarpStateV5.js').default} */ ( await callInternalRuntimeMethod(detached, 'materializeStrand', internalSource.strandId, { @@ -196,7 +177,7 @@ async function resolveObserverSnapshot(graph, options) { return await snapshotReturnedState(detached, state); } - throw new Error(`unknown observer source kind: ${/** @type {{ kind?: unknown }} */ (source).kind}`); + throw new Error(`unknown observer source kind: ${source.constructor.name}`); } /** @@ -507,7 +488,7 @@ function query() { function worldline(options = undefined) { return new Worldline({ graph: this._host, - source: cloneObserverSource(options?.source) || { kind: 'live' }, + source: toSelector(options?.source) || new LiveSelector(), }); } @@ -561,7 +542,7 @@ async function observer(nameOrConfig, configOrOptions = undefined, maybeOptions config, graph: this._host, snapshot, - source: cloneObserverSource(options?.source) || { kind: 'live' }, + source: /** @type {import('../../../../index.js').WorldlineSource} */ (/** @type {unknown} */ (toSelector(options?.source) || new LiveSelector())), }); } diff --git a/src/domain/services/controllers/SyncController.js b/src/domain/services/controllers/SyncController.js index d6779159..9f40d9cf 100644 --- a/src/domain/services/controllers/SyncController.js +++ b/src/domain/services/controllers/SyncController.js @@ -48,6 +48,7 @@ import SyncTrustGate from '../sync/SyncTrustGate.js'; * @property {import('../../../ports/CodecPort.js').default} _codec * @property {import('../../../ports/CryptoPort.js').default} _crypto * @property {import('../../../ports/LoggerPort.js').default|null} _logger + * @property {import('../../../ports/PatchJournalPort.js').default|null} [_patchJournal] * @property {import('../../../ports/BlobStoragePort.js').default|null} [_patchBlobStorage] * @property {number} _patchesSinceCheckpoint * @property {(op: string, t0: number, opts?: {metrics?: string, error?: Error}) => void} _logTiming @@ -335,7 +336,10 @@ export default class SyncController { localFrontier, persistence, this._host._graphName, - /** @type {Record} */ ({ codec: this._host._codec, logger: this._host._logger || undefined, patchBlobStorage: this._host._patchBlobStorage || undefined }) + /** @type {Record} */ ({ + ...(this._host._patchJournal !== null && this._host._patchJournal !== undefined ? { patchJournal: this._host._patchJournal } : {}), + ...(this._host._logger !== null && this._host._logger !== undefined ? { logger: this._host._logger } : {}), + }) ); } diff --git a/src/domain/services/index/LogicalBitmapIndexBuilder.js b/src/domain/services/index/LogicalBitmapIndexBuilder.js index 385f9112..095428fa 100644 --- a/src/domain/services/index/LogicalBitmapIndexBuilder.js +++ b/src/domain/services/index/LogicalBitmapIndexBuilder.js @@ -18,6 +18,7 @@ import defaultCodec from '../../utils/defaultCodec.js'; import computeShardKey from '../../utils/shardKey.js'; import { getRoaringBitmap32 } from '../../utils/roaring.js'; import { ShardIdOverflowError } from '../../errors/index.js'; +import { MetaShard, EdgeShard, LabelShard, ReceiptShard } from '../../artifacts/IndexShard.js'; /** Maximum local IDs per shard (2^24). */ const MAX_LOCAL_ID = 1 << 24; @@ -273,6 +274,90 @@ export default class LogicalBitmapIndexBuilder { return tree; } + /** + * Yields IndexShard instances without encoding. + * + * This is the stream-compatible alternative to `serialize()`. Pipe the + * output through the adapter's encode → blobWrite → treeAssemble + * pipeline to persist. + * + * @returns {Generator} + */ + *yieldShards() { + const allShardKeys = new Set([...this._shardNextLocal.keys()]); + + // Meta shards + for (const shardKey of allShardKeys) { + const nodeToGlobal = (this._shardNodes.get(shardKey) ?? []) + .slice() + .sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0)); + + const aliveBitmap = this._aliveBitmaps.get(shardKey); + const aliveBytes = aliveBitmap ? aliveBitmap.serialize(true) : new Uint8Array(0); + + yield new MetaShard({ + shardKey, + nodeToGlobal, + nextLocalId: this._shardNextLocal.get(shardKey) ?? 0, + alive: aliveBytes, + }); + } + + // Labels registry + /** @type {Array<[string, number]>} */ + const labelRegistry = []; + for (const [label, id] of this._labelToId) { + labelRegistry.push([label, id]); + } + yield new LabelShard({ labels: labelRegistry }); + + // Forward/reverse edge shards + yield* this._yieldEdgeShards('fwd', this._fwdBitmaps); + yield* this._yieldEdgeShards('rev', this._revBitmaps); + + // Receipt + yield new ReceiptShard({ + version: 1, + nodeCount: this._nodeToGlobal.size, + labelCount: this._labelToId.size, + shardCount: allShardKeys.size, + }); + } + + /** + * Yields EdgeShard instances for a direction without encoding. + * + * @param {'fwd'|'rev'} direction + * @param {Map} bitmaps + * @returns {Generator} + * @private + */ + *_yieldEdgeShards(direction, bitmaps) { + /** @type {Map>>} */ + const byShardKey = new Map(); + + for (const [key, bitmap] of bitmaps) { + const firstColon = key.indexOf(':'); + const secondColon = key.indexOf(':', firstColon + 1); + const shardKey = key.substring(0, firstColon); + const bucketName = key.substring(firstColon + 1, secondColon); + const globalIdStr = key.substring(secondColon + 1); + + if (!byShardKey.has(shardKey)) { + byShardKey.set(shardKey, {}); + } + const shardData = /** @type {Record>} */ (byShardKey.get(shardKey)); + if (!shardData[bucketName]) { + shardData[bucketName] = {}; + } + shardData[bucketName][globalIdStr] = bitmap.serialize(true); + } + + for (const [shardKey, shardData] of byShardKey) { + yield new EdgeShard({ shardKey, direction, buckets: shardData }); + } + } + /** * Serializes forward or reverse edge bitmaps into the output tree, grouped by shard. * diff --git a/src/domain/services/index/LogicalIndexBuildService.js b/src/domain/services/index/LogicalIndexBuildService.js index ac52b481..d5120c35 100644 --- a/src/domain/services/index/LogicalIndexBuildService.js +++ b/src/domain/services/index/LogicalIndexBuildService.js @@ -14,6 +14,8 @@ import PropertyIndexBuilder from './PropertyIndexBuilder.js'; import { orsetElements } from '../../crdt/ORSet.js'; import { decodeEdgeKey, decodePropKey, isEdgePropKey } from '../KeyCodec.js'; import { nodeVisibleV5, edgeVisibleV5 } from '../state/StateSerializerV5.js'; +import WarpStream from '../../stream/WarpStream.js'; +import { ReceiptShard } from '../../artifacts/IndexShard.js'; export default class LogicalIndexBuildService { /** @@ -34,10 +36,63 @@ export default class LogicalIndexBuildService { * @returns {{ tree: Record, receipt: Record }} */ build(state, options = {}) { + const { indexBuilder, propBuilder } = this._populateBuilders(state, options); + + const indexTree = indexBuilder.serialize(); + const propTree = propBuilder.serialize(); + const tree = { ...indexTree, ...propTree }; + + const receiptBlob = indexTree['receipt.cbor']; + if (!receiptBlob) { throw new Error('Missing receipt.cbor in index tree'); } + const receipt = /** @type {Record} */ (this._codec.decode(receiptBlob)); + + return { tree, receipt }; + } + + /** + * Builds a complete logical index as a WarpStream of IndexShard records. + * + * The stream yields MetaShard, LabelShard, EdgeShard, PropertyShard, + * and ReceiptShard instances in builder order. Pipe through + * IndexShardEncodeTransform → GitBlobWriteTransform → TreeAssemblerSink + * to persist. + * + * @param {import('../JoinReducer.js').WarpStateV5} state + * @param {{ existingMeta?: Record, nextLocalId: number }>, existingLabels?: Record|Array<[string, number]> }} [options] + * @returns {{ stream: WarpStream, receipt: ReceiptShard }} + */ + buildStream(state, options = {}) { + const { indexBuilder, propBuilder } = this._populateBuilders(state, options); + + // Collect shards once — generators yield fresh iterators on each call, + // so calling yieldShards() twice would re-iterate all bitmaps. + const indexShards = [...indexBuilder.yieldShards()]; + const receiptShard = indexShards.find((s) => s instanceof ReceiptShard); + if (!receiptShard) { + throw new Error('LogicalIndexBuildService: index builder did not emit a ReceiptShard'); + } + + // Merge both builders' shard streams + const stream = WarpStream.mux( + WarpStream.from(indexShards), + WarpStream.from(propBuilder.yieldShards()), + ); + + return { stream, receipt: /** @type {ReceiptShard} */ (receiptShard) }; + } + + /** + * Populates both builders from state. Shared between build() and buildStream(). + * + * @param {import('../JoinReducer.js').WarpStateV5} state + * @param {{ existingMeta?: Record, nextLocalId: number }>, existingLabels?: Record|Array<[string, number]> }} options + * @returns {{ indexBuilder: LogicalBitmapIndexBuilder, propBuilder: PropertyIndexBuilder }} + * @private + */ + _populateBuilders(state, options) { const indexBuilder = new LogicalBitmapIndexBuilder({ codec: this._codec }); const propBuilder = new PropertyIndexBuilder({ codec: this._codec }); - // Seed existing data for stability if (options.existingMeta) { for (const [shardKey, meta] of Object.entries(options.existingMeta)) { indexBuilder.loadExistingMeta(shardKey, meta); @@ -47,62 +102,51 @@ export default class LogicalIndexBuildService { indexBuilder.loadExistingLabels(options.existingLabels); } - // 1. Register and mark alive all visible nodes (sorted for deterministic ID assignment) const aliveNodes = [...orsetElements(state.nodeAlive)].sort(); for (const nodeId of aliveNodes) { indexBuilder.registerNode(nodeId); indexBuilder.markAlive(nodeId); } - // 2. Collect visible edges and register labels (sorted for deterministic ID assignment) - const visibleEdges = []; - for (const edgeKey of orsetElements(state.edgeAlive)) { - if (edgeVisibleV5(state, edgeKey)) { - visibleEdges.push(decodeEdgeKey(edgeKey)); - } - } - visibleEdges.sort((a, b) => { - if (a.from !== b.from) { - return a.from < b.from ? -1 : 1; - } - if (a.to !== b.to) { - return a.to < b.to ? -1 : 1; - } - if (a.label !== b.label) { - return a.label < b.label ? -1 : 1; - } - return 0; - }); - const uniqueLabels = [...new Set(visibleEdges.map(e => e.label))].sort(); + const visibleEdges = _collectVisibleEdges(state); + const uniqueLabels = [...new Set(visibleEdges.map((e) => e.label))].sort(); for (const label of uniqueLabels) { indexBuilder.registerLabel(label); } - - // 3. Add edges for (const { from, to, label } of visibleEdges) { indexBuilder.addEdge(from, to, label); } - // 4. Build property index from visible props for (const [propKey, register] of state.prop) { - if (isEdgePropKey(propKey)) { - continue; - } + if (isEdgePropKey(propKey)) { continue; } const { nodeId, propKey: key } = decodePropKey(propKey); if (nodeVisibleV5(state, nodeId)) { propBuilder.addProperty(nodeId, key, register.value); } } - // 5. Serialize - const indexTree = indexBuilder.serialize(); - const propTree = propBuilder.serialize(); - const tree = { ...indexTree, ...propTree }; - - const receiptBlob = indexTree['receipt.cbor']; - if (!receiptBlob) { throw new Error('Missing receipt.cbor in index tree'); } - const receipt = /** @type {Record} */ (this._codec.decode(receiptBlob)); + return { indexBuilder, propBuilder }; + } +} - return { tree, receipt }; +/** + * Collects and sorts visible edges from state. + * + * @param {import('../JoinReducer.js').WarpStateV5} state + * @returns {Array<{from: string, to: string, label: string}>} + */ +function _collectVisibleEdges(state) { + const visibleEdges = []; + for (const edgeKey of orsetElements(state.edgeAlive)) { + if (edgeVisibleV5(state, edgeKey)) { + visibleEdges.push(decodeEdgeKey(edgeKey)); + } } + visibleEdges.sort((a, b) => { + if (a.from !== b.from) { return a.from < b.from ? -1 : 1; } + if (a.to !== b.to) { return a.to < b.to ? -1 : 1; } + if (a.label !== b.label) { return a.label < b.label ? -1 : 1; } + return 0; + }); + return visibleEdges; } diff --git a/src/domain/services/index/PropertyIndexBuilder.js b/src/domain/services/index/PropertyIndexBuilder.js index 4835d16b..16624d08 100644 --- a/src/domain/services/index/PropertyIndexBuilder.js +++ b/src/domain/services/index/PropertyIndexBuilder.js @@ -9,6 +9,7 @@ import defaultCodec from '../../utils/defaultCodec.js'; import computeShardKey from '../../utils/shardKey.js'; +import { PropertyShard } from '../../artifacts/IndexShard.js'; /** * Creates a null-prototype object typed as Record. @@ -67,8 +68,6 @@ export default class PropertyIndexBuilder { /** @type {Record} */ const tree = {}; for (const [shardKey, shard] of this._shards) { - // Encode as array of [nodeId, props] pairs to avoid __proto__ key issues - // when CBOR decodes into plain objects. Sorted by nodeId for determinism. const entries = [...shard.entries()] .sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0)) .map(([nodeId, props]) => [nodeId, props]); @@ -76,4 +75,21 @@ export default class PropertyIndexBuilder { } return tree; } + + /** + * Yields PropertyShard instances without encoding. + * + * @returns {Generator} + */ + *yieldShards() { + for (const [shardKey, shard] of this._shards) { + const entries = [...shard.entries()] + .sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0)) + .map(([nodeId, props]) => [nodeId, props]); + yield new PropertyShard({ + shardKey, + entries: /** @type {Array<[string, Record]>} */ (entries), + }); + } + } } diff --git a/src/domain/services/query/Observer.js b/src/domain/services/query/Observer.js index 4cbfc1b5..b962a952 100644 --- a/src/domain/services/query/Observer.js +++ b/src/domain/services/query/Observer.js @@ -17,104 +17,20 @@ import { decodeEdgeKey } from '../KeyCodec.js'; import { matchGlob } from '../../utils/matchGlob.js'; -/** @import { WorldlineSource } from '../../../../index.js' */ +import WorldlineSelector from '../../types/WorldlineSelector.js'; +import LiveSelector from '../../types/LiveSelector.js'; + /** - * Clones an observer worldline source descriptor, producing an independent copy. - * @param {{ - * kind: 'live', - * ceiling?: number|null - * } | { - * kind: 'coordinate', - * frontier: Map|Record, - * ceiling?: number|null - * } | { - * kind: 'strand', - * strandId: string, - * ceiling?: number|null - * } | { - * kind: 'strand', - * strandId: string, - * ceiling?: number|null - * } | null | undefined} source - * @returns {{ - * kind: 'live', - * ceiling?: number|null - * } | { - * kind: 'coordinate', - * frontier: Map|Record, - * ceiling?: number|null - * } | { - * kind: 'strand', - * strandId: string, - * ceiling?: number|null - * } | null} + * Converts a raw source to a WorldlineSelector, or returns null. + * + * @param {WorldlineSelector|{ kind: string, [key: string]: unknown }|null|undefined} source + * @returns {WorldlineSelector|null} */ -function cloneObserverSource(source) { +function toSelector(source) { if (source === null || source === undefined) { return null; } - return cloneNonNullSource(source); -} - -/** - * Clones a live source descriptor. - * @param {{ ceiling?: number|null }} source - * @returns {{ kind: 'live', ceiling?: number|null }} - */ -function cloneLiveSource(source) { - return 'ceiling' in source - ? { kind: 'live', ceiling: source.ceiling ?? null } - : { kind: 'live' }; -} - -/** - * Clones a coordinate source descriptor, deep-copying the frontier. - * @param {{ frontier?: Map|Record, ceiling?: number|null }} source - * @returns {{ kind: 'coordinate', frontier: Map|Record, ceiling: number|null }} - */ -function cloneCoordinateSource(source) { - return { - kind: 'coordinate', - frontier: source.frontier instanceof Map - ? new Map(source.frontier) - : { .../** @type {Record} */ (source.frontier) }, - ceiling: source.ceiling ?? null, - }; -} - -/** - * Clones a non-null observer source descriptor. - * @param {{ - * kind: 'live' | 'coordinate' | 'strand', - * ceiling?: number|null, - * frontier?: Map|Record, - * strandId?: string - * }} source - * @returns {{ - * kind: 'live', - * ceiling?: number|null - * } | { - * kind: 'coordinate', - * frontier: Map|Record, - * ceiling?: number|null - * } | { - * kind: 'strand', - * strandId: string, - * ceiling?: number|null - * }} - */ -function cloneNonNullSource(source) { - if (source.kind === 'live') { - return cloneLiveSource(source); - } - if (source.kind === 'coordinate') { - return cloneCoordinateSource(source); - } - return { - kind: 'strand', - strandId: /** @type {string} */ (source.strandId), - ceiling: source.ceiling ?? null, - }; + return WorldlineSelector.from(source).clone(); } /** @@ -336,7 +252,7 @@ export default class Observer { * Initializes the backing graph, snapshot, and source state. * @param {import('../../WarpRuntime.js').default|undefined} graph * @param {{ state: import('../JoinReducer.js').WarpStateV5, stateHash: string }|undefined} snapshot - * @param {{ kind: 'live', ceiling?: number|null } | { kind: 'coordinate', frontier: Map|Record, ceiling?: number|null } | { kind: 'strand', strandId: string, ceiling?: number|null } | undefined} source + * @param {import('../../types/WorldlineSelector.js').default|import('../../../../index.js').WorldlineSource|undefined} source * @private */ _initBacking(graph, snapshot, source) { @@ -344,8 +260,8 @@ export default class Observer { this._graph = graph || null; /** @type {{ state: import('../JoinReducer.js').WarpStateV5, stateHash: string }|null} */ this._snapshot = snapshot || null; - /** @type {{ kind: 'live', ceiling?: number|null } | { kind: 'coordinate', frontier: Map|Record, ceiling?: number|null } | { kind: 'strand', strandId: string, ceiling?: number|null } | null} */ - this._source = cloneObserverSource(source || { kind: 'live' }); + /** @type {WorldlineSelector|null} */ + this._source = toSelector(/** @type {WorldlineSelector|{ kind: string, [key: string]: unknown }} */ (source || new LiveSelector())); /** @type {import('../../../../index.js').VisibleStateReaderV5|null} */ this._stateReader = snapshot ? createStateReaderV5(snapshot.state) : null; /** @type {{ outgoing: Map, incoming: Map }|null} */ @@ -363,10 +279,10 @@ export default class Observer { /** * Gets the effective pinned source for this observer. * - * @returns {{ kind: 'live', ceiling?: number|null } | { kind: 'coordinate', frontier: Map|Record, ceiling?: number|null } | { kind: 'strand', strandId: string, ceiling?: number|null } | null} + * @returns {import('../../../../index.js').WorldlineSource|null} */ get source() { - return cloneObserverSource(this._source); + return this._source ? /** @type {import('../../../../index.js').WorldlineSource} */ (this._source.toDTO()) : null; } /** @@ -417,16 +333,13 @@ export default class Observer { async seek(options = undefined) { const graph = this._requireGraph(); const config = this._buildConfigSnapshot(); - /** @type {WorldlineSource|null} */ + /** @type {WorldlineSelector} */ const nextSource = options?.source - ? cloneObserverSource(/** @type {WorldlineSource} */ (options.source)) - : { kind: 'live' }; - if (nextSource === null) { - throw new Error('observer seek requires a non-null source'); - } + ? WorldlineSelector.from(options.source).clone() + : new LiveSelector(); // eslint-disable-next-line @typescript-eslint/no-unsafe-return -- return through defineProperty delegation; type is declared in @returns - return await graph.observer(/** @type {string} */ (this._name), config, { source: nextSource }); + return await graph.observer(/** @type {string} */ (this._name), config, { source: /** @type {import('../../../../index.js').WorldlineSource} */ (nextSource.toDTO()) }); } // =========================================================================== diff --git a/src/domain/services/state/CheckpointService.js b/src/domain/services/state/CheckpointService.js index c00cd1fd..3a9cfe2f 100644 --- a/src/domain/services/state/CheckpointService.js +++ b/src/domain/services/state/CheckpointService.js @@ -238,16 +238,18 @@ function collectContentAnchorEntries(propMap) { * └── provenanceIndex.cbor # Optional: node-to-patchSha index (HG/IO/2) * ``` * - * @param {{ persistence: import('../../../ports/CommitPort.js').default & import('../../../ports/BlobPort.js').default & import('../../../ports/TreePort.js').default, graphName: string, state: import('../JoinReducer.js').WarpStateV5, frontier: import('../Frontier.js').Frontier, parents?: string[], compact?: boolean, provenanceIndex?: import('../provenance/ProvenanceIndex.js').ProvenanceIndex, codec?: import('../../../ports/CodecPort.js').default, crypto?: import('../../../ports/CryptoPort.js').default, indexTree?: Record }} options - Checkpoint creation options + * @param {{ persistence: import('../../../ports/CommitPort.js').default & import('../../../ports/BlobPort.js').default & import('../../../ports/TreePort.js').default, graphName: string, state: import('../JoinReducer.js').WarpStateV5, frontier: import('../Frontier.js').Frontier, parents?: string[], compact?: boolean, provenanceIndex?: import('../provenance/ProvenanceIndex.js').ProvenanceIndex, codec?: import('../../../ports/CodecPort.js').default, crypto?: import('../../../ports/CryptoPort.js').default, indexTree?: Record, checkpointStore?: import('../../../ports/CheckpointStorePort.js').default, stateHashService?: import('./StateHashService.js').default }} options - Checkpoint creation options * @returns {Promise} The checkpoint commit SHA */ -export async function create({ persistence, graphName, state, frontier, parents = [], compact = true, provenanceIndex, codec, crypto, indexTree }) { +export async function create({ persistence, graphName, state, frontier, parents = [], compact = true, provenanceIndex, codec, crypto, indexTree, checkpointStore, stateHashService }) { /** @type {Parameters[0]} */ const opts = { persistence, graphName, state, frontier, parents, compact }; if (provenanceIndex !== undefined && provenanceIndex !== null) { opts.provenanceIndex = provenanceIndex; } if (codec !== undefined && codec !== null) { opts.codec = codec; } if (crypto !== undefined && crypto !== null) { opts.crypto = crypto; } if (indexTree !== undefined && indexTree !== null) { opts.indexTree = indexTree; } + if (checkpointStore !== undefined && checkpointStore !== null) { opts.checkpointStore = checkpointStore; } + if (stateHashService !== undefined && stateHashService !== null) { opts.stateHashService = stateHashService; } return await createV5(opts); } @@ -263,7 +265,7 @@ export async function create({ persistence, graphName, state, frontier, parents * └── provenanceIndex.cbor # Optional: node-to-patchSha index (HG/IO/2) * ``` * - * @param {{ persistence: import('../../../ports/CommitPort.js').default & import('../../../ports/BlobPort.js').default & import('../../../ports/TreePort.js').default, graphName: string, state: import('../JoinReducer.js').WarpStateV5, frontier: import('../Frontier.js').Frontier, parents?: string[], compact?: boolean, provenanceIndex?: import('../provenance/ProvenanceIndex.js').ProvenanceIndex, codec?: import('../../../ports/CodecPort.js').default, crypto?: import('../../../ports/CryptoPort.js').default, indexTree?: Record }} options - Checkpoint creation options + * @param {{ persistence: import('../../../ports/CommitPort.js').default & import('../../../ports/BlobPort.js').default & import('../../../ports/TreePort.js').default, graphName: string, state: import('../JoinReducer.js').WarpStateV5, frontier: import('../Frontier.js').Frontier, parents?: string[], compact?: boolean, provenanceIndex?: import('../provenance/ProvenanceIndex.js').ProvenanceIndex, codec?: import('../../../ports/CodecPort.js').default, crypto?: import('../../../ports/CryptoPort.js').default, indexTree?: Record, checkpointStore?: import('../../../ports/CheckpointStorePort.js').default, stateHashService?: import('./StateHashService.js').default }} options - Checkpoint creation options * @returns {Promise} The checkpoint commit SHA */ export async function createV5({ @@ -277,6 +279,8 @@ export async function createV5({ codec, crypto, indexTree, + checkpointStore, + stateHashService, }) { // 1. Compute appliedVV from actual state dots const appliedVV = computeAppliedVV(state); @@ -291,27 +295,56 @@ export async function createV5({ orsetCompact(checkpointState.edgeAlive, appliedVV); } - // 3. Serialize full state (AUTHORITATIVE) + // 3–6. Serialize and write state, frontier, appliedVV. + // When checkpointStore is available, it owns serialization + blob writes. + // Otherwise fall back to the legacy serialize + writeBlob path. + // codecOpt is still needed for provenance index serialization (Slice 4 scope). const codecOpt = codec !== undefined && codec !== null ? { codec } : {}; - const stateBuffer = serializeFullStateV5(checkpointState, codecOpt); - - // 4. Compute state hash - const stateHash = await computeStateHashV5(checkpointState, { ...codecOpt, crypto: /** @type {import('../../../ports/CryptoPort.js').default} */ (crypto) }); - - // 5. Serialize frontier and appliedVV - const frontierBuffer = serializeFrontier(frontier, codecOpt); - const appliedVVBuffer = serializeAppliedVV(appliedVV, codecOpt); - - // 6. Write blobs to git - const stateBlobOid = await persistence.writeBlob(stateBuffer); - const frontierBlobOid = await persistence.writeBlob(frontierBuffer); - const appliedVVBlobOid = await persistence.writeBlob(appliedVVBuffer); - - // 6b. Optionally serialize and write provenance index + /** @type {string} */ + let stateBlobOid; + /** @type {string} */ + let stateHash; + /** @type {string} */ + let frontierBlobOid; + /** @type {string} */ + let appliedVVBlobOid; + /** @type {string|null} */ let provenanceIndexBlobOid = null; - if (provenanceIndex) { - const provenanceIndexBuffer = provenanceIndex.serialize(codecOpt); - provenanceIndexBlobOid = await persistence.writeBlob(provenanceIndexBuffer); + + if (checkpointStore !== undefined && checkpointStore !== null) { + // Compute stateHash first via StateHashService (preferred) or legacy fallback + if (stateHashService !== undefined && stateHashService !== null) { + stateHash = await stateHashService.compute(checkpointState); + } else { + stateHash = await computeStateHashV5(checkpointState, { ...codecOpt, crypto: /** @type {import('../../../ports/CryptoPort.js').default} */ (crypto) }); + } + const writeResult = await checkpointStore.writeCheckpoint({ + state: checkpointState, + frontier, + appliedVV, + stateHash, + ...(provenanceIndex ? { provenanceIndex } : {}), + }); + stateBlobOid = writeResult.stateBlobOid; + frontierBlobOid = writeResult.frontierBlobOid; + appliedVVBlobOid = writeResult.appliedVVBlobOid; + provenanceIndexBlobOid = writeResult.provenanceIndexBlobOid; + } else { + // Legacy path: serialize in-process, write raw blobs + const stateBuffer = serializeFullStateV5(checkpointState, codecOpt); + stateHash = await computeStateHashV5(checkpointState, { ...codecOpt, crypto: /** @type {import('../../../ports/CryptoPort.js').default} */ (crypto) }); + const frontierBuffer = serializeFrontier(frontier, codecOpt); + const appliedVVBuffer = serializeAppliedVV(appliedVV, codecOpt); + stateBlobOid = await persistence.writeBlob(stateBuffer); + frontierBlobOid = await persistence.writeBlob(frontierBuffer); + appliedVVBlobOid = await persistence.writeBlob(appliedVVBuffer); + + // 6b. Optionally serialize and write provenance index (legacy path only; + // when checkpointStore is used, writeCheckpoint already wrote it) + if (provenanceIndex) { + const provenanceIndexBuffer = provenanceIndex.serialize(codecOpt); + provenanceIndexBlobOid = await persistence.writeBlob(provenanceIndexBuffer); + } } // 6c. Optionally write index subtree (schema 4) @@ -391,11 +424,11 @@ export async function createV5({ * * @param {import('../../../ports/CommitPort.js').default & import('../../../ports/BlobPort.js').default & import('../../../ports/TreePort.js').default} persistence - Git persistence adapter * @param {string} checkpointSha - The checkpoint commit SHA to load - * @param {{ codec?: import('../../../ports/CodecPort.js').default }} [options] - Load options + * @param {{ codec?: import('../../../ports/CodecPort.js').default, checkpointStore?: import('../../../ports/CheckpointStorePort.js').default }} [options] - Load options * @returns {Promise<{state: import('../JoinReducer.js').WarpStateV5, frontier: import('../Frontier.js').Frontier, stateHash: string, schema: number, appliedVV: VersionVector|null, provenanceIndex?: import('../provenance/ProvenanceIndex.js').ProvenanceIndex, indexShardOids: Record|null}>} The loaded checkpoint data * @throws {Error} If checkpoint is schema:1 (migration required) */ -export async function loadCheckpoint(persistence, checkpointSha, { codec } = {}) { +export async function loadCheckpoint(persistence, checkpointSha, { codec, checkpointStore } = {}) { // 1. Read commit message and decode const message = await persistence.showNode(checkpointSha); const decoded = /** @type {{ schema: number, stateHash: string, indexOid: string }} */ (decodeCheckpointMessage(message)); @@ -417,6 +450,26 @@ export async function loadCheckpoint(persistence, checkpointSha, { codec } = {}) // 3b. Partition: entries with 'index/' prefix are bitmap index shards const { treeOids, indexShardOids } = partitionTreeOids(rawTreeOids); + if (checkpointStore !== undefined && checkpointStore !== null) { + // New collapsed API: one call reads all artifacts + const cpData = await checkpointStore.readCheckpoint(treeOids); + /** @type {{ state: import('../JoinReducer.js').WarpStateV5, frontier: import('../Frontier.js').Frontier, stateHash: string, schema: number, appliedVV: VersionVector|null, provenanceIndex?: import('../provenance/ProvenanceIndex.js').ProvenanceIndex, indexShardOids: Record|null }} */ + const result = { + state: cpData.state, + frontier: cpData.frontier, + stateHash: decoded.stateHash, // Authoritative: from commit message, not adapter + schema: decoded.schema, // Authoritative: from commit message + appliedVV: cpData.appliedVV, + indexShardOids: Object.keys(indexShardOids).length > 0 ? indexShardOids : cpData.indexShardOids, + }; + if (cpData.provenanceIndex !== null && cpData.provenanceIndex !== undefined) { + result.provenanceIndex = cpData.provenanceIndex; + } + return result; + } + + // Legacy path: read each blob individually + // 4. Read frontier.cbor blob const frontierOid = treeOids['frontier.cbor']; if (frontierOid === undefined) { @@ -431,7 +484,6 @@ export async function loadCheckpoint(persistence, checkpointSha, { codec } = {}) throw new Error(`Checkpoint ${checkpointSha} missing state.cbor in tree`); } const stateBuffer = await persistence.readBlob(stateOid); - // V5: Load AUTHORITATIVE full state from state.cbor (NEVER use visible.cbor for resume) const state = deserializeFullStateV5(stateBuffer, loadCodecOpt); diff --git a/src/domain/services/state/StateHashService.js b/src/domain/services/state/StateHashService.js new file mode 100644 index 00000000..f412996a --- /dev/null +++ b/src/domain/services/state/StateHashService.js @@ -0,0 +1,48 @@ +import { projectStateV5 } from './StateSerializerV5.js'; +import WarpError from '../../errors/WarpError.js'; + +/** + * Computes canonical state hashes for verification, comparison, + * and checkpoint creation. + * + * The hash is SHA-256 of the CBOR-encoded visible state projection. + * This service owns the hash computation — it is NOT buried inside + * any single adapter or write path. + * + * Consumers: checkpoint creation, comparison, detached integrity + * checks, materialization verification. + */ +export default class StateHashService { + /** + * Creates a StateHashService. + * + * @param {{ + * codec: import('../../../ports/CodecPort.js').default, + * crypto: import('../../../ports/CryptoPort.js').default, + * }} deps + */ + constructor({ codec, crypto }) { + if (codec === undefined || codec === null) { + throw new WarpError('StateHashService requires a codec', 'E_MISSING_DEPENDENCY'); + } + if (crypto === undefined || crypto === null) { + throw new WarpError('StateHashService requires a crypto adapter', 'E_MISSING_DEPENDENCY'); + } + /** @type {import('../../../ports/CodecPort.js').default} */ + this._codec = codec; + /** @type {import('../../../ports/CryptoPort.js').default} */ + this._crypto = crypto; + } + + /** + * Computes the SHA-256 hash of the canonical visible state projection. + * + * @param {import('../../services/JoinReducer.js').WarpStateV5} state + * @returns {Promise} Hex-encoded SHA-256 hash + */ + async compute(state) { + const projection = projectStateV5(state); + const bytes = this._codec.encode(projection); + return await this._crypto.hash('sha256', bytes); + } +} diff --git a/src/domain/services/strand/StrandService.js b/src/domain/services/strand/StrandService.js index 078c8e0e..b3d1e091 100644 --- a/src/domain/services/strand/StrandService.js +++ b/src/domain/services/strand/StrandService.js @@ -797,9 +797,11 @@ async function openDetachedReadGraph(graph) { if (graph._logger !== undefined && graph._logger !== null) { opts.logger = graph._logger; } if (graph._crypto !== undefined && graph._crypto !== null) { opts.crypto = graph._crypto; } if (graph._codec !== undefined && graph._codec !== null) { opts.codec = graph._codec; } + if (graph._patchJournal !== undefined && graph._patchJournal !== null) { opts.patchJournal = /** @type {import('../../../ports/PatchJournalPort.js').default} */ (graph._patchJournal); } if (graph._seekCache !== undefined && graph._seekCache !== null) { opts.seekCache = graph._seekCache; } if (graph._blobStorage !== undefined && graph._blobStorage !== null) { opts.blobStorage = graph._blobStorage; } if (graph._patchBlobStorage !== undefined && graph._patchBlobStorage !== null) { opts.patchBlobStorage = graph._patchBlobStorage; } + if (graph._checkpointStore !== undefined && graph._checkpointStore !== null) { opts.checkpointStore = graph._checkpointStore; } return await GraphClass.open(opts); } @@ -1117,11 +1119,10 @@ export default class StrandService { onCommitSuccess: async (/** @type {{ patch: import('../../types/WarpTypesV2.js').PatchV2, sha: string }} */ { patch, sha }) => { await this._syncOverlayDescriptor(descriptor, { patch, sha }); }, - codec: this._graph._codec, }; - if (this._graph._logger) { pbOpts.logger = this._graph._logger; } - if (this._graph._blobStorage) { pbOpts.blobStorage = this._graph._blobStorage; } - if (this._graph._patchBlobStorage) { pbOpts.patchBlobStorage = this._graph._patchBlobStorage; } + if (this._graph._patchJournal !== null && this._graph._patchJournal !== undefined) { pbOpts.patchJournal = this._graph._patchJournal; } + if (this._graph._logger !== null && this._graph._logger !== undefined) { pbOpts.logger = this._graph._logger; } + if (this._graph._blobStorage !== null && this._graph._blobStorage !== undefined) { pbOpts.blobStorage = this._graph._blobStorage; } return new PatchBuilderV2(pbOpts); } @@ -1316,11 +1317,10 @@ export default class StrandService { getCurrentState: () => state, expectedParentSha: descriptor.overlay.headPatchSha ?? null, onDeleteWithData: this._graph._onDeleteWithData, - codec: this._graph._codec, }; - if (this._graph._logger) { intentPbOpts.logger = this._graph._logger; } - if (this._graph._blobStorage) { intentPbOpts.blobStorage = this._graph._blobStorage; } - if (this._graph._patchBlobStorage) { intentPbOpts.patchBlobStorage = this._graph._patchBlobStorage; } + if (this._graph._patchJournal !== null && this._graph._patchJournal !== undefined) { intentPbOpts.patchJournal = this._graph._patchJournal; } + if (this._graph._logger !== null && this._graph._logger !== undefined) { intentPbOpts.logger = this._graph._logger; } + if (this._graph._blobStorage !== null && this._graph._blobStorage !== undefined) { intentPbOpts.blobStorage = this._graph._blobStorage; } const builder = new PatchBuilderV2(intentPbOpts); await build(builder); const patch = builder.build(); @@ -1977,12 +1977,23 @@ export default class StrandService { writer: overlayId, lamport, }; - const patchCbor = this._graph._codec.encode(committedPatch); - const patchBlobOid = this._graph._patchBlobStorage - ? await this._graph._patchBlobStorage.store(patchCbor, { - slug: `${this._graph._graphName}/${overlayId}/patch`, - }) - : await this._graph._persistence.writeBlob(patchCbor); + /** @type {string} */ + let patchBlobOid; + /** @type {import('../../../ports/PatchJournalPort.js').default | null | undefined} */ + const journal = this._graph._patchJournal; + if (journal !== undefined && journal !== null) { + patchBlobOid = await journal.writePatch( + /** @type {import('../../types/WarpTypesV2.js').PatchV2} */ (committedPatch), + ); + } else { + // Legacy fallback: encode + write blob directly + const patchCbor = this._graph._codec.encode(committedPatch); + patchBlobOid = this._graph._patchBlobStorage + ? await this._graph._patchBlobStorage.store(patchCbor, { + slug: `${this._graph._graphName}/${overlayId}/patch`, + }) + : await this._graph._persistence.writeBlob(patchCbor); + } const treeEntries = [`100644 blob ${patchBlobOid}\tpatch.cbor`]; const uniqueBlobOids = [...new Set(contentBlobOids)]; diff --git a/src/domain/services/sync/SyncProtocol.js b/src/domain/services/sync/SyncProtocol.js index 7bc9da11..1d0a8c8f 100644 --- a/src/domain/services/sync/SyncProtocol.js +++ b/src/domain/services/sync/SyncProtocol.js @@ -36,13 +36,11 @@ * @see Frontier - Frontier manipulation utilities */ -import defaultCodec from '../../utils/defaultCodec.js'; import nullLogger from '../../utils/nullLogger.js'; import { decodePatchMessage, assertOpsCompatible, SCHEMA_V3 } from '../codec/WarpMessageCodec.js'; import { join, cloneStateV5, isKnownRawOp } from '../JoinReducer.js'; import SchemaUnsupportedError from '../../errors/SchemaUnsupportedError.js'; import SyncError from '../../errors/SyncError.js'; -import EncryptionError from '../../errors/EncryptionError.js'; import PersistenceError from '../../errors/PersistenceError.js'; import { cloneFrontier, updateFrontier } from '../Frontier.js'; import VersionVector from '../../crdt/VersionVector.js'; @@ -125,10 +123,10 @@ function objectToFrontier(obj) { * **Commit message format**: The message is encoded using WarpMessageCodec * and contains metadata (schema version, writer info) plus the patch OID. * - * @param {import('../../../ports/CommitPort.js').default & import('../../../ports/BlobPort.js').default} persistence - Git persistence layer - * (uses CommitPort.showNode() + BlobPort.readBlob() methods) + * @param {import('../../../ports/CommitPort.js').default} persistence - Git persistence layer + * (uses CommitPort.showNode() for commit message reading) * @param {string} sha - The 40-character commit SHA to load the patch from - * @param {{ codec?: import('../../../ports/CodecPort.js').default, patchBlobStorage?: import('../../../ports/BlobStoragePort.js').default }} [options] + * @param {{ patchJournal?: import('../../../ports/PatchJournalPort.js').default }} [options] * @returns {Promise} The decoded and normalized patch object containing: * - `ops`: Array of patch operations * - `context`: VersionVector (Map) of causal dependencies @@ -137,37 +135,26 @@ function objectToFrontier(obj) { * @throws {Error} If the commit cannot be read (invalid SHA, not found) * @throws {Error} If the commit message cannot be decoded (malformed, wrong schema) * @throws {Error} If the patch blob cannot be read (blob not found, I/O error) - * @throws {Error} If the patch blob cannot be CBOR-decoded (corrupted data) - * @throws {EncryptionError} If the patch is encrypted but no patchBlobStorage is provided + * @throws {Error} If the patch blob cannot be decoded (corrupted data) + * @throws {Error} If patchJournal is not provided * @private */ -async function loadPatchFromCommit(persistence, sha, { codec: codecOpt, patchBlobStorage } = /** @type {{ codec?: import('../../../ports/CodecPort.js').default, patchBlobStorage?: import('../../../ports/BlobStoragePort.js').default }} */ ({})) { - const codec = codecOpt || defaultCodec; - // Read commit message to extract patch OID - const message = await persistence.showNode(sha); - const decoded = decodePatchMessage(message); - - // Read the patch blob (encrypted or plain) - /** @type {Uint8Array} */ - let patchBuffer; - if (decoded.encrypted) { - if (!patchBlobStorage) { - throw new EncryptionError( - 'This graph contains encrypted patches; provide patchBlobStorage with an encryption key', - ); - } - patchBuffer = await patchBlobStorage.retrieve(decoded.patchOid); - } else { - patchBuffer = await persistence.readBlob(decoded.patchOid); - } - if (patchBuffer === null || patchBuffer === undefined) { +async function loadPatchFromCommit(persistence, sha, { patchJournal } = /** @type {{ patchJournal?: import('../../../ports/PatchJournalPort.js').default }} */ ({})) { + if (!patchJournal) { throw new PersistenceError( - `Patch blob not found: ${decoded.patchOid}`, + 'patchJournal is required for loading patches', PersistenceError.E_MISSING_OBJECT, - { context: { oid: decoded.patchOid } }, + { context: { sha } }, ); } - const patch = /** @type {DecodedPatch} */ (codec.decode(patchBuffer)); + // Read commit message to extract patch OID and encrypted flag + const message = await persistence.showNode(sha); + const decoded = decodePatchMessage(message); + + // Read and decode the patch blob via PatchJournalPort (adapter owns the codec) + const patch = /** @type {DecodedPatch} */ ( + await patchJournal.readPatch(decoded.patchOid, { encrypted: decoded.encrypted }) + ); // Normalize the patch (convert context from object to Map) return normalizePatch(patch); @@ -194,7 +181,7 @@ async function loadPatchFromCommit(persistence, sha, { codec: codecOpt, patchBlo * @param {string|null} fromSha - Start SHA (exclusive). Pass null to load ALL patches * for this writer from the beginning of their chain. * @param {string} toSha - End SHA (inclusive). This is typically the writer's current tip. - * @param {{ codec?: import('../../../ports/CodecPort.js').default, patchBlobStorage?: import('../../../ports/BlobStoragePort.js').default }} [options] + * @param {{ patchJournal?: import('../../../ports/PatchJournalPort.js').default }} [options] * @returns {Promise>} Array of patch objects in * chronological order (oldest first). Each entry contains: * - `patch`: The decoded patch object @@ -213,7 +200,7 @@ async function loadPatchFromCommit(persistence, sha, { codec: codecOpt, patchBlo * // Load ALL patches for a new writer * const patches = await loadPatchRange(persistence, 'events', 'new-writer', null, tipSha); */ -export async function loadPatchRange(persistence, _graphName, writerId, fromSha, toSha, { codec, patchBlobStorage } = /** @type {{ codec?: import('../../../ports/CodecPort.js').default, patchBlobStorage?: import('../../../ports/BlobStoragePort.js').default }} */ ({})) { +export async function loadPatchRange(persistence, _graphName, writerId, fromSha, toSha, { patchJournal } = /** @type {{ patchJournal?: import('../../../ports/PatchJournalPort.js').default }} */ ({})) { const patches = []; /** @type {string | null} */ let cur = toSha; @@ -223,7 +210,7 @@ export async function loadPatchRange(persistence, _graphName, writerId, fromSha, const commitInfo = await persistence.getNodeInfo(cur); // Load patch from commit - const patch = await loadPatchFromCommit(persistence, cur, { ...(codec !== undefined ? { codec } : {}), ...(patchBlobStorage !== undefined ? { patchBlobStorage } : {}) }); + const patch = await loadPatchFromCommit(persistence, cur, patchJournal !== undefined ? { patchJournal } : {}); patches.unshift({ patch, sha: cur }); // Prepend for chronological order // Move to parent (first parent in linear chain) @@ -420,7 +407,7 @@ export function createSyncRequest(frontier) { * @param {import('../../../ports/CommitPort.js').default & import('../../../ports/BlobPort.js').default} persistence - Git persistence * layer for loading patches (uses CommitPort + BlobPort methods) * @param {string} graphName - Graph name for error messages and logging - * @param {{ codec?: import('../../../ports/CodecPort.js').default, logger?: import('../../../ports/LoggerPort.js').default, patchBlobStorage?: import('../../../ports/BlobStoragePort.js').default }} [options] + * @param {{ patchJournal?: import('../../../ports/PatchJournalPort.js').default, logger?: import('../../../ports/LoggerPort.js').default }} [options] * @returns {Promise} Response containing local frontier and patches. * Patches are ordered chronologically within each writer. * @throws {Error} If patch loading fails for reasons other than divergence @@ -434,7 +421,7 @@ export function createSyncRequest(frontier) { * res.json(response); * }); */ -export async function processSyncRequest(request, localFrontier, persistence, graphName, { codec, logger, patchBlobStorage } = /** @type {{ codec?: import('../../../ports/CodecPort.js').default, logger?: import('../../../ports/LoggerPort.js').default, patchBlobStorage?: import('../../../ports/BlobStoragePort.js').default }} */ ({})) { +export async function processSyncRequest(request, localFrontier, persistence, graphName, { patchJournal, logger } = /** @type {{ patchJournal?: import('../../../ports/PatchJournalPort.js').default, logger?: import('../../../ports/LoggerPort.js').default }} */ ({})) { const log = logger || nullLogger; const remoteFrontier = objectToFrontier(request.frontier); @@ -472,17 +459,19 @@ export async function processSyncRequest(request, localFrontier, persistence, gr } } - const writerPatches = await loadPatchRange( - persistence, - graphName, - writerId, - range.from, - range.to, - { ...(codec !== undefined ? { codec } : {}), ...(patchBlobStorage !== undefined ? { patchBlobStorage } : {}) } - ); - - for (const { patch, sha } of writerPatches) { - patches.push({ writerId, sha, patch }); + // Prefer streaming scan when patchJournal supports it; fall back to legacy array load. + if (patchJournal !== undefined && patchJournal !== null && typeof patchJournal.scanPatchRange === 'function') { + const stream = patchJournal.scanPatchRange(writerId, range.from, range.to); + for await (const entry of stream) { + patches.push({ writerId, sha: entry.sha, patch: entry.patch }); + } + } else { + const writerPatches = await loadPatchRange( + persistence, graphName, writerId, range.from, range.to, patchJournal !== undefined ? { patchJournal } : {}, + ); + for (const { patch, sha } of writerPatches) { + patches.push({ writerId, sha, patch }); + } } } catch (err) { // If we detect divergence, log and skip this writer (B65). diff --git a/src/domain/stream/Sink.js b/src/domain/stream/Sink.js new file mode 100644 index 00000000..7fd5be32 --- /dev/null +++ b/src/domain/stream/Sink.js @@ -0,0 +1,59 @@ +import WarpError from '../errors/WarpError.js'; + +/** + * Base class for stream sinks. + * + * A Sink is a terminal consumer of an async iterable. It accepts + * elements via `_accept()`, then produces a final accumulated result + * via `_finalize()`. Sinks do not yield values — they end the pipeline. + * + * Examples: TreeAssemblerSink accumulates [path, oid] entries and + * calls writeTree() in finalize(). ArraySink collects all items. + * + * @template T - The type of elements consumed + * @template R - The type of the accumulated result + */ +export default class Sink { + /** + * Consumes an entire async iterable and returns the accumulated result. + * + * Subclasses implement `_accept(item)` for per-element processing and + * `_finalize()` for the terminal result. The default `consume()` loop + * handles iteration, error propagation, and finalization. + * + * @param {AsyncIterable} source - The upstream async iterable + * @returns {Promise} The accumulated result + */ + async consume(source) { + if (source === null || source === undefined) { + throw new WarpError('Sink.consume() requires a source', 'E_INVALID_SOURCE'); + } + for await (const item of source) { + await this._accept(item); + } + return await this._finalize(); + } + + /** + * Accepts a single element from the stream. + * + * Override this in subclasses to process each element. + * + * @param {T} _item - The element to accept + * @returns {void | Promise} + */ + _accept(_item) { + throw new WarpError('Sink._accept() not implemented', 'E_NOT_IMPLEMENTED'); + } + + /** + * Produces the final accumulated result after all elements are consumed. + * + * Override this in subclasses to return the terminal value. + * + * @returns {R | Promise} + */ + _finalize() { + throw new WarpError('Sink._finalize() not implemented', 'E_NOT_IMPLEMENTED'); + } +} diff --git a/src/domain/stream/Transform.js b/src/domain/stream/Transform.js new file mode 100644 index 00000000..b213e2eb --- /dev/null +++ b/src/domain/stream/Transform.js @@ -0,0 +1,53 @@ +import WarpError from '../errors/WarpError.js'; + +/** + * Base class for stream transforms. + * + * A Transform maps each element of an async iterable source to a new + * value. Simple transforms pass a function to the constructor. Complex + * transforms (batching, splitting, stateful) override `apply()`. + * + * Transforms are the composition unit for WarpStream pipelines. The + * codec, the compressor, the encryptor — all Transforms. + * + * @template T + * @template U + */ +export default class Transform { + /** + * Creates a new Transform. + * + * @param {(item: T) => U | Promise} [fn] - Per-element mapping function. + * Optional — subclasses that override apply() don't need it. + */ + constructor(fn) { + if (fn !== undefined && typeof fn !== 'function') { + throw new WarpError('Transform requires a function or subclass override', 'E_INVALID_TRANSFORM'); + } + /** @type {((item: T) => U | Promise) | undefined} */ + this._fn = fn; + } + + /** + * Applies this transform to a source async iterable. + * + * The default implementation maps each element through the constructor + * function. Subclasses override this for complex transforms (batching, + * splitting, stateful accumulation). + * + * @param {AsyncIterable} source - The upstream async iterable + * @returns {AsyncIterable} A new async iterable of transformed values + */ + async *apply(source) { + if (this._fn === undefined) { + throw new WarpError( + 'Transform.apply() must be overridden or a function must be provided to the constructor', + 'E_NOT_IMPLEMENTED', + ); + } + const fn = this._fn; + for await (const item of source) { + yield await fn(item); + } + } +} diff --git a/src/domain/stream/WarpStream.js b/src/domain/stream/WarpStream.js new file mode 100644 index 00000000..647e8399 --- /dev/null +++ b/src/domain/stream/WarpStream.js @@ -0,0 +1,536 @@ +import WarpError from '../errors/WarpError.js'; +import { checkAborted } from '../utils/cancellation.js'; + +/** + * Validates that a source is a valid iterable. + * @param {unknown} source + */ +function _validateSource(source) { + if (source === null || source === undefined) { + throw new WarpError('WarpStream requires an async iterable source', 'E_INVALID_SOURCE'); + } + const s = /** @type {Record} */ (source); + const hasAsync = typeof s[Symbol.asyncIterator] === 'function'; + const hasSync = typeof s[Symbol.iterator] === 'function'; + if (!hasAsync && !hasSync) { + throw new WarpError('WarpStream source must implement Symbol.asyncIterator or Symbol.iterator', 'E_INVALID_SOURCE'); + } +} + +/** + * Composable async stream built on AsyncIterable. + * + * WarpStream is the domain concept for "data flow over time." It wraps + * an AsyncIterable and provides composable operations: pipe, tee, + * mux, demux, drain. + * + * Backpressure is natural via `for await` (pull-based). Error propagation + * uses the async iterator protocol: downstream throws trigger upstream + * `return()` for cleanup. Cooperative cancellation via AbortSignal. + * + * When the dataset is unbounded, stream-first is not an optimization — + * it is the honest API. + * + * @template T + */ +export default class WarpStream { + /** + * Creates a WarpStream wrapping an async iterable source. + * + * @param {AsyncIterable} source - The underlying async iterable + * @param {{ signal?: AbortSignal }} [options] + */ + constructor(source, { signal } = {}) { + _validateSource(source); + /** @type {AsyncIterable} */ + this._source = source; + /** @type {AbortSignal | undefined} */ + this._signal = signal; + } + + // ── Factories ───────────────────────────────────────────────────── + + /** + * Creates a WarpStream from any iterable or async iterable. + * + * @template V + * @param {AsyncIterable | Iterable} iterable + * @param {{ signal?: AbortSignal }} [options] + * @returns {WarpStream} + */ + static from(iterable, options) { + if (iterable instanceof WarpStream) { + // eslint-disable-next-line @typescript-eslint/no-unsafe-return -- instanceof narrows; cast is correct + return /** @type {WarpStream} */ (iterable); + } + // Wrap sync iterables as async + const src = /** @type {Record} */ (/** @type {unknown} */ (iterable)); + if (typeof src[Symbol.asyncIterator] === 'function') { + return new WarpStream(/** @type {AsyncIterable} */ (iterable), options); + } + if (typeof src[Symbol.iterator] === 'function') { + return new WarpStream(_syncToAsync(/** @type {Iterable} */ (iterable)), options); + } + throw new WarpError('WarpStream.from() requires an iterable or async iterable', 'E_INVALID_SOURCE'); + } + + /** + * Creates a WarpStream from explicit values. + * + * @template V + * @param {...V} items + * @returns {WarpStream} + */ + static of(...items) { + return WarpStream.from(items); + } + + /** + * Merges multiple streams into one (fan-in). + * + * Elements are interleaved in arrival order — whichever source yields + * next gets emitted next. All sources are consumed concurrently. + * + * @template V + * @param {...WarpStream} streams + * @returns {WarpStream} + */ + static mux(...streams) { + if (streams.length === 0) { + return WarpStream.from(/** @type {AsyncIterable} */ (_empty())); + } + if (streams.length === 1) { + return /** @type {WarpStream} */ (streams[0]); + } + return new WarpStream(_muxImpl(streams)); + } + + // ── Composition ─────────────────────────────────────────────────── + + /** + * Pipes this stream through a Transform, producing a new WarpStream. + * + * @template U + * @param {import('./Transform.js').default} transform + * @returns {WarpStream} + */ + pipe(transform) { + if (transform === null || transform === undefined || typeof transform.apply !== 'function') { + throw new WarpError('pipe() requires a Transform with an apply() method', 'E_INVALID_TRANSFORM'); + } + const source = this._withCancellation(); + return new WarpStream(transform.apply(source), this._signal !== undefined ? { signal: this._signal } : {}); + } + + /** + * Splits this stream into two independent branches. + * + * Both branches receive all elements. Elements are buffered for the + * slower branch (one element at a time via the pull protocol). + * + * @returns {[WarpStream, WarpStream]} + */ + tee() { + const source = this._withCancellation(); + const [a, b] = _teeImpl(source); + /** @type {{ signal?: AbortSignal }} */ + const opts = this._signal !== undefined ? { signal: this._signal } : {}; + return [ + new WarpStream(a, opts), + new WarpStream(b, opts), + ]; + } + + /** + * Splits this stream into named branches by a classifier function. + * + * Elements are routed to the branch whose key matches the classifier + * result. Unknown keys are dropped. + * + * Note: demux eagerly consumes the source. All branches must be + * consumed to avoid deadlock. Use for bounded fan-out where you know + * the key space upfront. + * + * @param {(item: T) => string} classify - Returns the branch key for each element + * @param {string[]} keys - The expected branch keys (must be known upfront) + * @returns {Map>} + */ + demux(classify, keys) { + if (!Array.isArray(keys) || keys.length === 0) { + throw new WarpError('demux() requires a non-empty keys array', 'E_INVALID_DEMUX'); + } + const source = this._withCancellation(); + const branches = _demuxImpl(source, classify, keys); + /** @type {{ signal?: AbortSignal }} */ + const demuxOpts = this._signal !== undefined ? { signal: this._signal } : {}; + /** @type {Map>} */ + const result = new Map(); + for (const [key, iter] of branches) { + result.set(key, new WarpStream(iter, demuxOpts)); + } + return result; + } + + // ── Terminal Operations ─────────────────────────────────────────── + + /** + * Drains this stream into a Sink and returns the accumulated result. + * + * @template R + * @param {import('./Sink.js').default} sink + * @returns {Promise} + */ + async drain(sink) { + if (sink === null || sink === undefined || typeof sink.consume !== 'function') { + throw new WarpError('drain() requires a Sink with a consume() method', 'E_INVALID_SINK'); + } + return await sink.consume(this._withCancellation()); + } + + /** + * Reduces the stream to a single accumulated value. + * + * @template R + * @param {(acc: R, item: T) => R | Promise} fn - Reducer function + * @param {R} init - Initial accumulator value + * @returns {Promise} + */ + async reduce(fn, init) { + let acc = init; + for await (const item of this._withCancellation()) { + acc = await fn(acc, item); + } + return acc; + } + + /** + * Executes a function for each element. Returns when the stream ends. + * + * @param {(item: T) => void | Promise} fn + * @returns {Promise} + */ + async forEach(fn) { + for await (const item of this._withCancellation()) { + await fn(item); + } + } + + /** + * Collects all elements into an array. + * + * **DANGER**: This materializes the entire stream. Use only when the + * result is known to be bounded. For unbounded streams, use forEach(), + * reduce(), or drain() instead. + * + * @returns {Promise} + */ + async collect() { + /** @type {T[]} */ + const items = []; + for await (const item of this._withCancellation()) { + items.push(item); + } + return items; + } + + // ── Interop ─────────────────────────────────────────────────────── + + /** + * Makes WarpStream directly usable in `for await` loops. + * + * @returns {AsyncIterator} + */ + [Symbol.asyncIterator]() { + return this._withCancellation()[Symbol.asyncIterator](); + } + + // ── Internal ────────────────────────────────────────────────────── + + /** + * Wraps the source with AbortSignal checking if a signal is set. + * + * @returns {AsyncIterable} + * @private + */ + _withCancellation() { + if (this._signal === undefined) { + return this._source; + } + return _cancelable(this._source, this._signal); + } +} + +// ── Private Helpers ─────────────────────────────────────────────────── + +/** + * Wraps a sync iterable as an async iterable. + * + * @template T + * @param {Iterable} iterable + * @returns {AsyncIterable} + */ +function _syncToAsync(iterable) { + return { + [Symbol.asyncIterator]() { + const iter = iterable[Symbol.iterator](); + return { + next() { + return Promise.resolve(iter.next()); + }, + }; + }, + }; +} + +/** + * An empty async iterable. + * + * @template T + * @returns {AsyncIterable} + */ +async function* _empty() { + // yields nothing +} + +/** + * Wraps an async iterable with AbortSignal cancellation. + * + * @template T + * @param {AsyncIterable} source + * @param {AbortSignal} signal + * @returns {AsyncIterable} + */ +async function* _cancelable(source, signal) { + checkAborted(signal); + for await (const item of source) { + checkAborted(signal); + yield item; + } +} + +/** + * Merges multiple async iterables into one, interleaving by arrival order. + * + * @template T + * @param {WarpStream[]} streams + * @returns {AsyncIterable} + */ +async function* _muxImpl(streams) { + const iterators = streams.map((s) => s[Symbol.asyncIterator]()); + /** @type {Map}>>} */ + const pending = new Map(); + + // Start initial pull for each iterator + for (let i = 0; i < iterators.length; i++) { + const iter = /** @type {AsyncIterator} */ (iterators[i]); + pending.set(i, iter.next().then((result) => ({ index: i, result }))); + } + + while (pending.size > 0) { + const { index, result } = await Promise.race(pending.values()); + if (result.done === true) { + pending.delete(index); + } else { + yield result.value; + // Re-arm this iterator for its next value + const iter = /** @type {AsyncIterator} */ (iterators[index]); + pending.set(index, iter.next().then((r) => ({ index, result: r }))); + } + } +} + +/** + * Tees an async iterable into two independent branches. + * + * @template T + * @param {AsyncIterable} source + * @returns {[AsyncIterable, AsyncIterable]} + */ +function _teeImpl(source) { + const iterator = source[Symbol.asyncIterator](); + /** @type {T[]} Shared cache of items pulled from source, trimmed from the front. */ + const cache = []; + /** @type {number} Offset: the absolute index of cache[0]. */ + let cacheOffset = 0; + let finished = false; + /** @type {Error | null} */ + let error = null; + /** @type {Promise> | null} In-flight pull to prevent concurrent pulls. */ + let inflight = null; + /** @type {[number, number]} Absolute consumed index per branch. */ + const consumed = [0, 0]; + + /** + * Trims cache entries that both branches have consumed. + */ + function trimCache() { + const minConsumed = Math.min(consumed[0], consumed[1]); + const trimCount = minConsumed - cacheOffset; + if (trimCount > 0) { + cache.splice(0, trimCount); + cacheOffset += trimCount; + } + } + + /** + * Ensures the cache covers absolute index `needed - 1`, or source is done. + * Serializes concurrent pulls via the inflight promise. + * @param {number} needed - Absolute index count needed. + * @returns {Promise} + */ + async function ensureCached(needed) { + while (cacheOffset + cache.length < needed && !finished && error === null) { + if (inflight !== null) { + await inflight; + continue; + } + inflight = iterator.next(); + try { + const result = await inflight; + if (result.done === true) { + finished = true; + } else { + cache.push(result.value); + } + } catch (err) { + error = /** @type {Error} */ (err); + finished = true; + } finally { + inflight = null; + } + } + } + + /** + * Creates a branch that reads from the shared cache by absolute index. + * @param {number} branchId - 0 or 1, identifying this branch for trim tracking. + * @returns {AsyncIterable} + */ + function makeBranch(branchId) { + let index = 0; + return { + [Symbol.asyncIterator]() { + return /** @type {AsyncIterator} */ ({ + async next() { + await ensureCached(index + 1); + if (error !== null) { throw error; } + if (index >= cacheOffset + cache.length) { + return { value: /** @type {T} */ (undefined), done: true }; + } + const value = cache[index - cacheOffset]; + index++; + consumed[branchId] = index; + trimCache(); + return { value, done: false }; + }, + }); + }, + }; + } + + return [makeBranch(0), makeBranch(1)]; +} + +/** + * Demuxes an async iterable into named branches. + * + * @template T + * @param {AsyncIterable} source + * @param {(item: T) => string} classify + * @param {string[]} keys + * @returns {Map>} + */ +function _demuxImpl(source, classify, keys) { + /** @type {Map) => void, reject: (err: Error) => void}>>} */ + const waiters = new Map(); + /** @type {Map>} */ + const buffers = new Map(); + let pumpStarted = false; + let pumpDone = false; + /** @type {Error | null} */ + let pumpError = null; + + for (const key of keys) { + waiters.set(key, []); + buffers.set(key, []); + } + + /** + * Pumps the source and routes elements to branch buffers/waiters. + * @returns {Promise} + */ + async function pump() { + try { + for await (const item of source) { + const key = classify(item); + const keyWaiters = waiters.get(key); + if (keyWaiters === undefined) { + continue; // unknown key — drop + } + if (keyWaiters.length > 0) { + const waiter = /** @type {{resolve: (result: IteratorResult) => void}} */ (keyWaiters.shift()); + waiter.resolve({ value: item, done: false }); + } else { + /** @type {T[]} */ (buffers.get(key)).push(item); + } + } + } catch (err) { + pumpError = /** @type {Error} */ (err); + } finally { + pumpDone = true; + // Signal all waiting branches that the source is done + for (const [, keyWaiters] of waiters) { + for (const waiter of keyWaiters) { + if (pumpError !== null) { + waiter.reject(pumpError); + } else { + waiter.resolve({ value: /** @type {T} */ (undefined), done: true }); + } + } + keyWaiters.length = 0; + } + } + } + + /** + * Creates a branch async iterable for a given key. + * @param {string} key + * @returns {AsyncIterable} + */ + function makeBranch(key) { + return { + [Symbol.asyncIterator]() { + return { + next() { + // Start pump on first pull from any branch + if (!pumpStarted) { + pumpStarted = true; + void pump(); + } + if (pumpError !== null) { + return Promise.reject(pumpError); + } + const buffer = /** @type {T[]} */ (buffers.get(key)); + if (buffer.length > 0) { + return Promise.resolve({ value: /** @type {T} */ (buffer.shift()), done: false }); + } + if (pumpDone) { + return Promise.resolve({ value: /** @type {T} */ (undefined), done: true }); + } + // Wait for next item routed to this branch + return new Promise((resolve, reject) => { + /** @type {Array<{resolve: (result: IteratorResult) => void, reject: (err: Error) => void}>} */ (waiters.get(key)).push({ resolve, reject }); + }); + }, + }; + }, + }; + } + + /** @type {Map>} */ + const result = new Map(); + for (const key of keys) { + result.set(key, makeBranch(key)); + } + return result; +} diff --git a/src/domain/types/CoordinateSelector.js b/src/domain/types/CoordinateSelector.js new file mode 100644 index 00000000..1aa99f93 --- /dev/null +++ b/src/domain/types/CoordinateSelector.js @@ -0,0 +1,72 @@ +/** + * CoordinateSelector — observe a hypothetical worldline at specific writer tips. + * + * @module domain/types/CoordinateSelector + */ + +import WorldlineSelector, { validateCeiling } from './WorldlineSelector.js'; + +/** + * Worldline selector pinned to an explicit writer-tip coordinate. + * + * The coordinate specifies a hypothetical worldline that would result + * from merging only these writers at these commit SHAs. The frontier + * may be empty (produces empty materialized state). + */ +class CoordinateSelector extends WorldlineSelector { + /** @type {Map} */ + #frontier; + + /** + * Creates a CoordinateSelector. + * + * @param {Map|Record} frontier - Writer-tip frontier. May be empty. + * @param {number|null} [ceiling] - Lamport ceiling for time-travel. + */ + constructor(frontier, ceiling) { + super(); + + if (frontier === null || frontier === undefined || typeof frontier !== 'object') { + throw new TypeError('frontier must be a Map or plain object'); + } + + this.#frontier = frontier instanceof Map + ? new Map(frontier) + : new Map(Object.entries(frontier)); + + /** @type {number|null} */ + this.ceiling = validateCeiling(ceiling); + Object.freeze(this); + } + + /** + * Returns a defensive copy of the frontier. + * + * @returns {Map} + */ + get frontier() { + return new Map(this.#frontier); + } + + /** + * Deep-clone this selector, copying the frontier. + * + * @returns {CoordinateSelector} + */ + clone() { + return new CoordinateSelector(new Map(this.#frontier), this.ceiling); + } + + /** + * Convert to a plain DTO for the public API. + * + * @returns {{ kind: 'coordinate', frontier: Map, ceiling: number|null }} + */ + toDTO() { + return { kind: 'coordinate', frontier: new Map(this.#frontier), ceiling: this.ceiling }; + } +} + +WorldlineSelector._register('coordinate', CoordinateSelector); + +export default CoordinateSelector; diff --git a/src/domain/types/LiveSelector.js b/src/domain/types/LiveSelector.js new file mode 100644 index 00000000..23cdd5ce --- /dev/null +++ b/src/domain/types/LiveSelector.js @@ -0,0 +1,54 @@ +/** + * LiveSelector — observe the canonical worldline (all writers merged). + * + * @module domain/types/LiveSelector + */ + +import WorldlineSelector, { validateCeiling } from './WorldlineSelector.js'; + +/** + * Worldline selector pinned to the canonical (live) worldline. + * + * The canonical worldline merges every writer's patches via CRDT join. + * The optional ceiling selects a tick: "observe at tick N." + */ +class LiveSelector extends WorldlineSelector { + /** + * Creates a LiveSelector. + * + * @param {number|null} [ceiling] - Lamport ceiling for time-travel. Null or non-negative integer. + */ + constructor(ceiling) { + super(); + /** @type {number|null} */ + this.ceiling = validateCeiling(ceiling); + Object.freeze(this); + } + + /** + * Deep-clone this selector. + * + * @returns {LiveSelector} + */ + clone() { + return new LiveSelector(this.ceiling); + } + + /** + * Convert to a plain DTO for the public API. + * + * Omits ceiling when null to match the legacy WorldlineSource shape + * (consumers may check `'ceiling' in dto`). + * + * @returns {{ kind: 'live', ceiling?: number|null }} + */ + toDTO() { + return this.ceiling !== null + ? { kind: 'live', ceiling: this.ceiling } + : { kind: 'live' }; + } +} + +WorldlineSelector._register('live', LiveSelector); + +export default LiveSelector; diff --git a/src/domain/types/StrandSelector.js b/src/domain/types/StrandSelector.js new file mode 100644 index 00000000..b85d00b0 --- /dev/null +++ b/src/domain/types/StrandSelector.js @@ -0,0 +1,57 @@ +/** + * StrandSelector — observe one writer's isolated worldline. + * + * @module domain/types/StrandSelector + */ + +import WorldlineSelector, { validateCeiling } from './WorldlineSelector.js'; + +/** + * Worldline selector pinned to a single strand's visible patch universe. + * + * Used for branch-and-compare workflows where you want one writer's + * isolated perspective. + */ +class StrandSelector extends WorldlineSelector { + /** + * Creates a StrandSelector. + * + * @param {string} strandId - The strand identifier. Must be a non-empty string. + * @param {number|null} [ceiling] - Lamport ceiling for time-travel. + */ + constructor(strandId, ceiling) { + super(); + + if (typeof strandId !== 'string' || strandId.length === 0) { + throw new TypeError('strandId must be a non-empty string'); + } + + /** @type {string} */ + this.strandId = strandId; + /** @type {number|null} */ + this.ceiling = validateCeiling(ceiling); + Object.freeze(this); + } + + /** + * Deep-clone this selector. + * + * @returns {StrandSelector} + */ + clone() { + return new StrandSelector(this.strandId, this.ceiling); + } + + /** + * Convert to a plain DTO for the public API. + * + * @returns {{ kind: 'strand', strandId: string, ceiling: number|null }} + */ + toDTO() { + return { kind: 'strand', strandId: this.strandId, ceiling: this.ceiling }; + } +} + +WorldlineSelector._register('strand', StrandSelector); + +export default StrandSelector; diff --git a/src/domain/types/WorldlineSelector.js b/src/domain/types/WorldlineSelector.js new file mode 100644 index 00000000..2e592f33 --- /dev/null +++ b/src/domain/types/WorldlineSelector.js @@ -0,0 +1,126 @@ +/** + * WorldlineSelector — abstract base for worldline selector descriptors. + * + * A worldline selector specifies which worldline an observer projects. + * Three variants: LiveSelector (canonical worldline), + * CoordinateSelector (hypothetical worldline at specific writer tips), + * StrandSelector (single writer's isolated worldline). + * + * @module domain/types/WorldlineSelector + */ + +/** + * Validates a ceiling value. Must be null, undefined, or a non-negative integer. + * + * @param {unknown} ceiling + * @returns {number|null} normalized ceiling + */ +function validateCeiling(ceiling) { + if (ceiling === undefined || ceiling === null) { + return null; + } + if (typeof ceiling !== 'number' || !Number.isInteger(ceiling) || ceiling < 0) { + throw new TypeError(`ceiling must be null or a non-negative integer, got ${typeof ceiling === 'number' ? ceiling : typeof ceiling}`); + } + return ceiling; +} + +/** + * Subclass registry — populated by each subclass module on import. + * + * @type {Record} + */ +const registry = {}; + +/** + * Abstract base for worldline selectors. + * + * Subclasses: LiveSelector, CoordinateSelector, StrandSelector. + * Use WorldlineSelector.from() to convert plain { kind } objects + * at API boundaries. + */ +class WorldlineSelector { + /** + * Deep-clone this selector. + * + * @abstract + * @returns {WorldlineSelector} + */ + clone() { + throw new Error('WorldlineSelector.clone() is abstract'); + } + + /** + * Convert this selector to a plain DTO matching the WorldlineSource shape. + * + * @abstract + * @returns {{ kind: string, [key: string]: unknown }} + */ + toDTO() { + throw new Error('WorldlineSelector.toDTO() is abstract'); + } + + /** + * Register a subclass for use in from(). + * + * @param {string} kind + * @param {Function} ctor + */ + static _register(kind, ctor) { + if (Object.isFrozen(registry)) { + throw new Error('WorldlineSelector registry is frozen — cannot register after first use'); + } + registry[kind] = ctor; + } + + /** + * Normalize a raw source descriptor into a WorldlineSelector instance. + * + * Accepts class instances (returned as-is), plain { kind } objects + * (converted to the appropriate subclass), and null/undefined + * (defaults to LiveSelector). + * + * @param {WorldlineSelector|{ kind: string, [key: string]: unknown }|null|undefined} raw + * @returns {WorldlineSelector} + */ + static from(raw) { + if (raw instanceof WorldlineSelector) { + return raw; + } + // Freeze registry on first use — prevents post-init hijacking + if (!Object.isFrozen(registry)) { + Object.freeze(registry); + } + return fromPlainObject(raw); + } +} + +/** + * Builds a WorldlineSelector from a plain object or null/undefined. + * + * Note: the kind→constructor-args mapping is hardcoded for the three + * known selector kinds. Adding a new kind requires editing this function. + * + * Kept separate from the class to reduce static from() complexity. + * + * @param {{ kind: string, [key: string]: unknown }|null|undefined} raw + * @returns {WorldlineSelector} + */ +function fromPlainObject(raw) { + const value = raw ?? { kind: 'live' }; + const { kind } = value; + if (!(kind in registry)) { + throw new TypeError(`unknown worldline selector kind: ${String(kind)}`); + } + const Ctor = /** @type {new (...args: unknown[]) => WorldlineSelector} */ (registry[kind]); + if (kind === 'live') { + return new Ctor(value['ceiling']); + } + if (kind === 'coordinate') { + return new Ctor(value['frontier'], value['ceiling']); + } + return new Ctor(value['strandId'], value['ceiling']); +} + +export { validateCeiling }; +export default WorldlineSelector; diff --git a/src/domain/utils/canonicalCbor.js b/src/domain/utils/canonicalCbor.js deleted file mode 100644 index 75ee22cf..00000000 --- a/src/domain/utils/canonicalCbor.js +++ /dev/null @@ -1,36 +0,0 @@ -/** - * Canonical CBOR encoding/decoding. - * - * Delegates to defaultCodec which already sorts keys recursively - * and handles Maps, null-prototype objects, and arrays. - * - * Deterministic output relies on cbor-x's key-sorting behaviour, - * which approximates RFC 7049 Section 3.9 (Canonical CBOR) by sorting - * map keys in length-first lexicographic order. This is sufficient for - * content-addressed equality within the WARP system but should not be - * assumed to match other canonical CBOR implementations byte-for-byte. - * - * @module domain/utils/canonicalCbor - */ - -import defaultCodec from './defaultCodec.js'; - -/** - * Encodes a value to canonical CBOR bytes with sorted keys. - * - * @param {unknown} value - The value to encode - * @returns {Uint8Array} CBOR-encoded bytes - */ -export function encodeCanonicalCbor(value) { - return defaultCodec.encode(value); -} - -/** - * Decodes CBOR bytes to a value. - * - * @param {Uint8Array} buffer - CBOR bytes - * @returns {unknown} Decoded value - */ -export function decodeCanonicalCbor(buffer) { - return defaultCodec.decode(buffer); -} diff --git a/src/domain/warp/Writer.js b/src/domain/warp/Writer.js index 56e6e6f9..60148b56 100644 --- a/src/domain/warp/Writer.js +++ b/src/domain/warp/Writer.js @@ -14,7 +14,6 @@ * @see WARP Writer Spec v1 */ -import defaultCodec from '../utils/defaultCodec.js'; import nullLogger from '../utils/nullLogger.js'; import { validateWriterId, buildWriterRef } from '../utils/RefLayout.js'; import { PatchSession } from './PatchSession.js'; @@ -47,11 +46,10 @@ function _assertValidLamport(lamport, commitSha) { * Maps private Writer fields to PatchBuilderV2 option keys. */ const _WRITER_OPTIONAL_KEYS = /** @type {const} */ ([ - ['_codec', 'codec'], + ['_patchJournal', 'patchJournal'], ['_logger', 'logger'], ['_onCommitSuccess', 'onCommitSuccess'], ['_blobStorage', 'blobStorage'], - ['_patchBlobStorage', 'patchBlobStorage'], ]); /** @@ -77,20 +75,26 @@ export class Writer { /** * Creates a new Writer instance. * - * @param {{ persistence: import('../../ports/CommitPort.js').default & import('../../ports/BlobPort.js').default & import('../../ports/TreePort.js').default & import('../../ports/RefPort.js').default, graphName: string, writerId: string, versionVector: import('../crdt/VersionVector.js').default, getCurrentState: () => import('../services/JoinReducer.js').WarpStateV5 | null, onCommitSuccess?: (result: {patch: import('../types/WarpTypesV2.js').PatchV2, sha: string}) => void | Promise, onDeleteWithData?: 'reject'|'cascade'|'warn', codec?: import('../../ports/CodecPort.js').default, logger?: import('../../ports/LoggerPort.js').default, blobStorage?: import('../../ports/BlobStoragePort.js').default, patchBlobStorage?: import('../../ports/BlobStoragePort.js').default }} options + * @param {{ persistence: import('../../ports/CommitPort.js').default & import('../../ports/BlobPort.js').default & import('../../ports/TreePort.js').default & import('../../ports/RefPort.js').default, graphName: string, writerId: string, versionVector: import('../crdt/VersionVector.js').default, getCurrentState: () => import('../services/JoinReducer.js').WarpStateV5 | null, onCommitSuccess?: (result: {patch: import('../types/WarpTypesV2.js').PatchV2, sha: string}) => void | Promise, onDeleteWithData?: 'reject'|'cascade'|'warn', patchJournal: import('../../ports/PatchJournalPort.js').default, logger?: import('../../ports/LoggerPort.js').default, blobStorage?: import('../../ports/BlobStoragePort.js').default }} options */ - constructor({ persistence, graphName, writerId, versionVector, getCurrentState, onCommitSuccess, onDeleteWithData = 'warn', codec, logger, blobStorage, patchBlobStorage }) { + constructor({ persistence, graphName, writerId, versionVector, getCurrentState, onCommitSuccess, onDeleteWithData = 'warn', patchJournal, logger, blobStorage }) { validateWriterId(writerId); + if (patchJournal === null || patchJournal === undefined) { + throw new WriterError( + 'E_MISSING_JOURNAL', + 'patchJournal is required — Writer.beginPatch() produces patches that must be persisted via a PatchJournalPort.', + ); + } this._initFields(/** @type {Parameters[0]} */ ({ persistence, graphName, writerId, versionVector, getCurrentState, onCommitSuccess, onDeleteWithData, - codec, logger, blobStorage, patchBlobStorage, + patchJournal, logger, blobStorage, })); } /** * Assigns all Writer instance fields from the validated constructor options. - * @param {{ persistence: import('../../ports/CommitPort.js').default & import('../../ports/BlobPort.js').default & import('../../ports/TreePort.js').default & import('../../ports/RefPort.js').default, graphName: string, writerId: string, versionVector: import('../crdt/VersionVector.js').default, getCurrentState: () => import('../services/JoinReducer.js').WarpStateV5 | null, onCommitSuccess?: (result: {patch: import('../types/WarpTypesV2.js').PatchV2, sha: string}) => void | Promise, onDeleteWithData: 'reject'|'cascade'|'warn', codec?: import('../../ports/CodecPort.js').default, logger?: import('../../ports/LoggerPort.js').default, blobStorage?: import('../../ports/BlobStoragePort.js').default, patchBlobStorage?: import('../../ports/BlobStoragePort.js').default }} opts + * @param {{ persistence: import('../../ports/CommitPort.js').default & import('../../ports/BlobPort.js').default & import('../../ports/TreePort.js').default & import('../../ports/RefPort.js').default, graphName: string, writerId: string, versionVector: import('../crdt/VersionVector.js').default, getCurrentState: () => import('../services/JoinReducer.js').WarpStateV5 | null, onCommitSuccess?: (result: {patch: import('../types/WarpTypesV2.js').PatchV2, sha: string}) => void | Promise, onDeleteWithData: 'reject'|'cascade'|'warn', patchJournal: import('../../ports/PatchJournalPort.js').default, logger?: import('../../ports/LoggerPort.js').default, blobStorage?: import('../../ports/BlobStoragePort.js').default }} opts * @private */ _initFields(opts) { @@ -108,14 +112,12 @@ export class Writer { this._onCommitSuccess = opts.onCommitSuccess; /** @type {'reject'|'cascade'|'warn'} */ this._onDeleteWithData = opts.onDeleteWithData; - /** @type {import('../../ports/CodecPort.js').default|undefined} */ - this._codec = opts.codec ?? defaultCodec; + /** @type {import('../../ports/PatchJournalPort.js').default} */ + this._patchJournal = opts.patchJournal; /** @type {import('../../ports/LoggerPort.js').default} */ this._logger = opts.logger ?? nullLogger; /** @type {import('../../ports/BlobStoragePort.js').default|null} */ this._blobStorage = opts.blobStorage ?? null; - /** @type {import('../../ports/BlobStoragePort.js').default|null} */ - this._patchBlobStorage = opts.patchBlobStorage ?? null; /** @type {boolean} */ this._commitInProgress = false; } diff --git a/src/infrastructure/adapters/CborCheckpointStoreAdapter.js b/src/infrastructure/adapters/CborCheckpointStoreAdapter.js new file mode 100644 index 00000000..cd6bd8e2 --- /dev/null +++ b/src/infrastructure/adapters/CborCheckpointStoreAdapter.js @@ -0,0 +1,365 @@ +import CheckpointStorePort from '../../ports/CheckpointStorePort.js'; +import WarpError from '../../domain/errors/WarpError.js'; +import { orsetSerialize, orsetDeserialize } from '../../domain/crdt/ORSet.js'; +import VersionVector, { vvSerialize } from '../../domain/crdt/VersionVector.js'; +import { createEmptyStateV5 } from '../../domain/services/JoinReducer.js'; +import WarpStateV5 from '../../domain/services/state/WarpStateV5.js'; +import { ProvenanceIndex } from '../../domain/services/provenance/ProvenanceIndex.js'; + +/** + * CBOR-backed implementation of CheckpointStorePort. + * + * Owns the codec and raw blob persistence. Domain services call + * writeCheckpoint(record) with domain objects; the adapter internally + * encodes each artifact and writes blobs. + * + * @extends CheckpointStorePort + */ +export class CborCheckpointStoreAdapter extends CheckpointStorePort { + /** + * Creates a new CborCheckpointStoreAdapter. + * + * @param {{ + * codec: import('../../ports/CodecPort.js').default, + * blobPort: import('../../ports/BlobPort.js').default, + * }} options + */ + constructor({ codec, blobPort }) { + super(); + if (codec === null || codec === undefined) { + throw new WarpError('CborCheckpointStoreAdapter requires a codec', 'E_INVALID_DEPENDENCY'); + } + if (blobPort === null || blobPort === undefined) { + throw new WarpError('CborCheckpointStoreAdapter requires a blobPort', 'E_INVALID_DEPENDENCY'); + } + /** @type {import('../../ports/CodecPort.js').default} */ + this._codec = codec; + /** @type {import('../../ports/BlobPort.js').default} */ + this._blobPort = blobPort; + } + + /** + * Persists a complete checkpoint: encodes and writes all artifacts + * as blobs, returns the OIDs for tree assembly. + * + * @param {import('../../ports/CheckpointStorePort.js').CheckpointRecord} record + * @returns {Promise} + */ + async writeCheckpoint(record) { + // Encode all artifacts in parallel + const stateBytes = this._encodeFullState(record.state); + const frontierBytes = this._encodeFrontier(record.frontier); + const appliedVVBytes = this._encodeAppliedVV(record.appliedVV); + + /** @type {Uint8Array | null} */ + let provenanceBytes = null; + if (record.provenanceIndex !== null && record.provenanceIndex !== undefined) { + provenanceBytes = record.provenanceIndex.serialize({ codec: this._codec }); + } + + // Write blobs in parallel + const writes = [ + this._blobPort.writeBlob(stateBytes), + this._blobPort.writeBlob(frontierBytes), + this._blobPort.writeBlob(appliedVVBytes), + ]; + if (provenanceBytes !== null) { + writes.push(this._blobPort.writeBlob(provenanceBytes)); + } + + const oids = await Promise.all(writes); + return { + stateBlobOid: /** @type {string} */ (oids[0]), + frontierBlobOid: /** @type {string} */ (oids[1]), + appliedVVBlobOid: /** @type {string} */ (oids[2]), + provenanceIndexBlobOid: oids.length > 3 ? /** @type {string} */ (oids[3]) : null, + }; + } + + /** + * Reads checkpoint artifacts from a tree of OIDs. + * + * @param {Record} treeOids - Map of path → blob OID + * @returns {Promise} + */ + async readCheckpoint(treeOids) { + const stateOid = treeOids['state.cbor']; + const frontierOid = treeOids['frontier.cbor']; + const appliedVVOid = treeOids['appliedVV.cbor']; + const provenanceOid = treeOids['provenanceIndex.cbor']; + + if (stateOid === undefined) { + throw new WarpError('Checkpoint missing state.cbor', 'E_MISSING_ARTIFACT'); + } + if (frontierOid === undefined) { + throw new WarpError('Checkpoint missing frontier.cbor', 'E_MISSING_ARTIFACT'); + } + + // Read blobs in parallel + /** @type {Array>} */ + const reads = [ + this._blobPort.readBlob(stateOid), + this._blobPort.readBlob(frontierOid), + ]; + if (appliedVVOid !== undefined) { + reads.push(this._blobPort.readBlob(appliedVVOid)); + } + if (provenanceOid !== undefined) { + reads.push(this._blobPort.readBlob(provenanceOid)); + } + + const buffers = await Promise.all(reads); + let idx = 0; + const state = this._decodeFullState(/** @type {Uint8Array} */ (buffers[idx++])); + const frontier = this._decodeFrontier(/** @type {Uint8Array} */ (buffers[idx++])); + + /** @type {VersionVector | null} */ + let appliedVV = null; + if (appliedVVOid !== undefined) { + appliedVV = this._decodeAppliedVV(/** @type {Uint8Array} */ (buffers[idx++])); + } + + /** @type {ProvenanceIndex | null} */ + let provenanceIndex = null; + if (provenanceOid !== undefined) { + provenanceIndex = ProvenanceIndex.deserialize(/** @type {Uint8Array} */ (buffers[idx++]), { codec: this._codec }); + } + + // Partition index shard OIDs (entries with 'index/' prefix) + /** @type {Record | null} */ + let indexShardOids = null; + const shardEntries = Object.entries(treeOids).filter(([p]) => p.startsWith('index/')); + if (shardEntries.length > 0) { + indexShardOids = Object.fromEntries(shardEntries.map(([p, o]) => [p.slice('index/'.length), o])); + } + + return { + state, + frontier, + appliedVV, + stateHash: '', // Caller reads from commit message + schema: 2, + ...(provenanceIndex !== null ? { provenanceIndex } : {}), + indexShardOids, + }; + } + + // ── Encode Helpers ────────────────────────────────────────────────── + + /** + * Encodes full V5 state to CBOR bytes. + * + * @param {import('../../domain/services/JoinReducer.js').WarpStateV5} state + * @returns {Uint8Array} + */ + _encodeFullState(state) { + return this._codec.encode({ + version: 'full-v5', + nodeAlive: orsetSerialize(state.nodeAlive), + edgeAlive: orsetSerialize(state.edgeAlive), + prop: _serializePropsArray(state.prop), + observedFrontier: vvSerialize(state.observedFrontier), + edgeBirthEvent: _serializeEdgeBirthArray(state.edgeBirthEvent), + }); + } + + /** + * Encodes a frontier Map to CBOR bytes. + * + * @param {Map} frontier + * @returns {Uint8Array} + */ + _encodeFrontier(frontier) { + /** @type {Record} */ + const obj = {}; + for (const key of Array.from(frontier.keys()).sort()) { + obj[key] = frontier.get(key); + } + return this._codec.encode(obj); + } + + /** + * Encodes an applied VersionVector to CBOR bytes. + * + * @param {VersionVector} vv + * @returns {Uint8Array} + */ + _encodeAppliedVV(vv) { + return this._codec.encode(vvSerialize(vv)); + } + + // ── Decode Helpers ────────────────────────────────────────────────── + + /** + * Decodes CBOR bytes to full V5 state. + * + * @param {Uint8Array} buffer + * @returns {import('../../domain/services/JoinReducer.js').WarpStateV5} + */ + _decodeFullState(buffer) { + if (buffer === null || buffer === undefined) { + return createEmptyStateV5(); + } + const obj = /** @type {Record} */ (this._codec.decode(buffer)); + if (obj === null || obj === undefined) { + return createEmptyStateV5(); + } + if (obj['version'] !== undefined && obj['version'] !== 'full-v5') { + throw new WarpError( + `Unsupported full state version: expected 'full-v5', got '${JSON.stringify(obj['version'])}'`, + 'E_UNSUPPORTED_VERSION', + ); + } + return new WarpStateV5({ + nodeAlive: orsetDeserialize(obj['nodeAlive'] ?? {}), + edgeAlive: orsetDeserialize(obj['edgeAlive'] ?? {}), + prop: _deserializeProps(/** @type {[string, unknown][]} */ (obj['prop'])), + observedFrontier: VersionVector.from( + /** @type {{ [x: string]: number }} */ (obj['observedFrontier'] ?? {}), + ), + edgeBirthEvent: _deserializeEdgeBirthEvent(obj), + }); + } + + /** + * Decodes CBOR bytes to a frontier Map. + * + * @param {Uint8Array} buffer + * @returns {Map} + */ + _decodeFrontier(buffer) { + const obj = /** @type {Record} */ (this._codec.decode(buffer)); + /** @type {Map} */ + const frontier = new Map(); + for (const [k, v] of Object.entries(obj)) { + frontier.set(k, v); + } + return frontier; + } + + /** + * Decodes CBOR bytes to a VersionVector. + * + * @param {Uint8Array} buffer + * @returns {VersionVector} + */ + _decodeAppliedVV(buffer) { + const obj = /** @type {{ [x: string]: number }} */ (this._codec.decode(buffer)); + return VersionVector.from(obj); + } +} + +// ── Private Helpers ─────────────────────────────────────────────────── + +/** + * Serializes the props Map into a sorted array. + * + * @param {Map>} propMap + * @returns {Array<[string, unknown]>} + */ +function _serializePropsArray(propMap) { + /** @type {Array<[string, unknown]>} */ + const arr = []; + for (const [key, register] of propMap) { + arr.push([key, _serializeLWWRegister(register)]); + } + arr.sort((a, b) => (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0)); + return arr; +} + +/** + * Serializes the edgeBirthEvent Map. + * + * @param {Map | undefined} edgeBirthEvent + * @returns {Array<[string, {lamport: number, writerId: string, patchSha: string, opIndex: number}]>} + */ +function _serializeEdgeBirthArray(edgeBirthEvent) { + /** @type {Array<[string, {lamport: number, writerId: string, patchSha: string, opIndex: number}]>} */ + const result = []; + if (edgeBirthEvent !== undefined && edgeBirthEvent !== null) { + for (const [key, eventId] of edgeBirthEvent) { + result.push([key, { + lamport: eventId.lamport, writerId: eventId.writerId, + patchSha: eventId.patchSha, opIndex: eventId.opIndex, + }]); + } + result.sort((a, b) => (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0)); + } + return result; +} + +/** + * Deserializes props array. + * + * @param {Array<[string, unknown]>} propArray + * @returns {Map>} + */ +function _deserializeProps(propArray) { + /** @type {Map>} */ + const prop = new Map(); + if (!Array.isArray(propArray)) { return prop; } + for (const [key, registerObj] of propArray) { + const register = _deserializeLWWRegister( + /** @type {{ eventId: { lamport: number, writerId: string, patchSha: string, opIndex: number }, value: unknown } | null} */ (registerObj), + ); + if (register !== null) { prop.set(key, register); } + } + return prop; +} + +/** + * Deserializes edge birth events. + * + * @param {Record} obj + * @returns {Map} + */ +function _deserializeEdgeBirthEvent(obj) { + /** @type {Map} */ + const result = new Map(); + const birthData = obj['edgeBirthEvent'] ?? obj['edgeBirthLamport']; + if (!Array.isArray(birthData)) { return result; } + const typedData = /** @type {Array<[string, unknown]>} */ (birthData); + for (const [key, val] of typedData) { + if (typeof val === 'number') { + result.set(key, { lamport: val, writerId: '', patchSha: '0000', opIndex: 0 }); + } else { + const ev = /** @type {{lamport: number, writerId: string, patchSha: string, opIndex: number}} */ (val); + result.set(key, { lamport: ev.lamport, writerId: ev.writerId, patchSha: ev.patchSha, opIndex: ev.opIndex }); + } + } + return result; +} + +/** + * Serializes an LWW register. + * + * @param {import('../../domain/crdt/LWW.js').LWWRegister} register + * @returns {{ eventId: { lamport: number, opIndex: number, patchSha: string, writerId: string }, value: unknown } | null} + */ +function _serializeLWWRegister(register) { + if (register === null || register === undefined) { return null; } + return { + eventId: { + lamport: register.eventId.lamport, opIndex: register.eventId.opIndex, + patchSha: register.eventId.patchSha, writerId: register.eventId.writerId, + }, + value: register.value, + }; +} + +/** + * Deserializes an LWW register. + * + * @param {{ eventId: { lamport: number, writerId: string, patchSha: string, opIndex: number }, value: unknown } | null} obj + * @returns {import('../../domain/crdt/LWW.js').LWWRegister | null} + */ +function _deserializeLWWRegister(obj) { + if (obj === null || obj === undefined) { return null; } + return { + eventId: { + lamport: obj.eventId.lamport, writerId: obj.eventId.writerId, + patchSha: obj.eventId.patchSha, opIndex: obj.eventId.opIndex, + }, + value: obj.value, + }; +} diff --git a/src/infrastructure/adapters/CborDecodeTransform.js b/src/infrastructure/adapters/CborDecodeTransform.js new file mode 100644 index 00000000..21a08171 --- /dev/null +++ b/src/infrastructure/adapters/CborDecodeTransform.js @@ -0,0 +1,38 @@ +import Transform from '../../domain/stream/Transform.js'; +import WarpError from '../../domain/errors/WarpError.js'; + +/** + * Stream transform that CBOR-decodes the value component of [path, bytes] entries. + * + * Input: `[string, Uint8Array]` — path + CBOR bytes + * Output: `[string, unknown]` — path + decoded domain object + * + * @extends {Transform<[string, Uint8Array], [string, unknown]>} + */ +export class CborDecodeTransform extends Transform { + /** + * Creates a CborDecodeTransform. + * + * @param {import('../../ports/CodecPort.js').default} codec + */ + constructor(codec) { + super(); + if (codec === null || codec === undefined) { + throw new WarpError('CborDecodeTransform requires a codec', 'E_INVALID_DEPENDENCY'); + } + /** @type {import('../../ports/CodecPort.js').default} */ + this._codec = codec; + } + + /** + * Decodes each [path, bytes] entry to [path, data]. + * + * @param {AsyncIterable<[string, Uint8Array]>} source + * @returns {AsyncIterable<[string, unknown]>} + */ + async *apply(source) { + for await (const [path, bytes] of source) { + yield [path, this._codec.decode(bytes)]; + } + } +} diff --git a/src/infrastructure/adapters/CborEncodeTransform.js b/src/infrastructure/adapters/CborEncodeTransform.js new file mode 100644 index 00000000..eb8df628 --- /dev/null +++ b/src/infrastructure/adapters/CborEncodeTransform.js @@ -0,0 +1,38 @@ +import Transform from '../../domain/stream/Transform.js'; +import WarpError from '../../domain/errors/WarpError.js'; + +/** + * Stream transform that CBOR-encodes the value component of [path, data] entries. + * + * Input: `[string, unknown]` — path + domain object + * Output: `[string, Uint8Array]` — path + CBOR bytes + * + * @extends {Transform<[string, unknown], [string, Uint8Array]>} + */ +export class CborEncodeTransform extends Transform { + /** + * Creates a CborEncodeTransform. + * + * @param {import('../../ports/CodecPort.js').default} codec + */ + constructor(codec) { + super(); + if (codec === null || codec === undefined) { + throw new WarpError('CborEncodeTransform requires a codec', 'E_INVALID_DEPENDENCY'); + } + /** @type {import('../../ports/CodecPort.js').default} */ + this._codec = codec; + } + + /** + * Encodes each [path, data] entry to [path, bytes]. + * + * @param {AsyncIterable<[string, unknown]>} source + * @returns {AsyncIterable<[string, Uint8Array]>} + */ + async *apply(source) { + for await (const [path, data] of source) { + yield [path, this._codec.encode(data)]; + } + } +} diff --git a/src/infrastructure/adapters/CborPatchJournalAdapter.js b/src/infrastructure/adapters/CborPatchJournalAdapter.js new file mode 100644 index 00000000..b4b22c41 --- /dev/null +++ b/src/infrastructure/adapters/CborPatchJournalAdapter.js @@ -0,0 +1,183 @@ +import PatchJournalPort from '../../ports/PatchJournalPort.js'; +import WarpError from '../../domain/errors/WarpError.js'; +import WarpStream from '../../domain/stream/WarpStream.js'; +import PatchEntry from '../../domain/artifacts/PatchEntry.js'; +import { decodePatchMessage, detectMessageKind } from '../../domain/services/codec/WarpMessageCodec.js'; +import SyncError from '../../domain/errors/SyncError.js'; +import EncryptionError from '../../domain/errors/EncryptionError.js'; +import VersionVector from '../../domain/crdt/VersionVector.js'; + +/** + * CBOR-backed implementation of PatchJournalPort. + * + * Owns the codec and raw blob persistence. Domain services pass PatchV2 + * objects in and get PatchV2 objects back — no bytes leak across the + * port boundary. + * + * Supports both plain Git blob storage (BlobPort) and encrypted external + * storage (BlobStoragePort) via the optional `patchBlobStorage` parameter. + * + * @extends PatchJournalPort + */ +export class CborPatchJournalAdapter extends PatchJournalPort { + /** + * Creates a new CborPatchJournalAdapter. + * + * @param {{ + * codec: import('../../ports/CodecPort.js').default, + * blobPort: import('../../ports/BlobPort.js').default, + * commitPort?: import('../../ports/CommitPort.js').default, + * patchBlobStorage?: import('../../ports/BlobStoragePort.js').default | null, + * }} options + */ + constructor({ codec, blobPort, commitPort, patchBlobStorage }) { + super(); + if (codec === null || codec === undefined) { + throw new WarpError('CborPatchJournalAdapter requires a codec', 'E_INVALID_DEPENDENCY'); + } + if (blobPort === null || blobPort === undefined) { + throw new WarpError('CborPatchJournalAdapter requires a blobPort', 'E_INVALID_DEPENDENCY'); + } + /** @type {import('../../ports/CodecPort.js').default} */ + this._codec = codec; + /** @type {import('../../ports/BlobPort.js').default} */ + this._blobPort = blobPort; + /** @type {import('../../ports/CommitPort.js').default | null} */ + this._commitPort = commitPort ?? null; + /** @type {import('../../ports/BlobStoragePort.js').default | null} */ + this._patchBlobStorage = patchBlobStorage ?? null; + } + + /** + * Encodes a PatchV2 to CBOR and persists it as a blob. + * + * @param {import('../../domain/types/WarpTypesV2.js').PatchV2} patch + * @returns {Promise} The blob OID + */ + async writePatch(patch) { + const bytes = this._codec.encode(patch); + if (this._patchBlobStorage) { + return await this._patchBlobStorage.store(bytes); + } + return await this._blobPort.writeBlob(bytes); + } + + /** + * Reads a blob by OID and decodes the CBOR bytes to a PatchV2. + * + * @param {string} patchOid + * @param {{ encrypted?: boolean }} [options] + * @returns {Promise} + */ + async readPatch(patchOid, { encrypted = false } = {}) { + /** @type {Uint8Array} */ + let bytes; + if (encrypted && this._patchBlobStorage) { + bytes = await this._patchBlobStorage.retrieve(patchOid); + } else if (encrypted) { + throw new EncryptionError( + `Patch ${patchOid} is encrypted but no patchBlobStorage is configured`, + ); + } else { + bytes = await this._blobPort.readBlob(patchOid); + } + return /** @type {import('../../domain/types/WarpTypesV2.js').PatchV2} */ ( + this._codec.decode(bytes) + ); + } + + /** + * Whether this journal uses external blob storage. + * + * @returns {boolean} + */ + get usesExternalStorage() { + return this._patchBlobStorage !== null; + } + + /** + * Scans patches in a writer's chain between two SHAs, yielding + * PatchEntry instances in chronological order (oldest first). + * + * Walks the commit DAG backwards from toSha to fromSha, decodes + * each patch, and yields PatchEntry. The walk is streamed — patches + * are yielded as they're decoded, not accumulated into an array. + * + * @param {string} writerId - The writer whose chain to scan + * @param {string|null} fromSha - Start SHA (exclusive), null for all + * @param {string} toSha - End SHA (inclusive) + * @returns {WarpStream} + */ + scanPatchRange(writerId, fromSha, toSha) { + const adapter = this; + return WarpStream.from( + /** Walks commit chain and yields PatchEntry instances. @returns {AsyncGenerator} */ + (async function* () { + if (adapter._commitPort === null) { + throw new SyncError('scanPatchRange requires commitPort on the adapter', { + code: 'E_MISSING_COMMIT_PORT', + context: { writerId }, + }); + } + const commitPort = adapter._commitPort; + + // Walk backwards, collect into stack for chronological order + /** @type {Array<{sha: string, patchOid: string, encrypted: boolean}>} */ + const stack = []; + /** @type {string | null} */ + let cur = toSha; + + while (cur !== null && cur !== fromSha) { + const nodeInfo = await commitPort.getNodeInfo(cur); + const kind = detectMessageKind(nodeInfo.message); + if (kind !== 'patch') { + break; + } + const meta = decodePatchMessage(nodeInfo.message); + stack.push({ sha: cur, patchOid: meta.patchOid, encrypted: meta.encrypted }); + + /** @type {string | null} */ + const parent = (Array.isArray(nodeInfo.parents) && nodeInfo.parents.length > 0) + ? /** @type {string} */ (nodeInfo.parents[0]) + : null; + cur = parent; + } + + // Divergence check + if (fromSha !== null && fromSha !== undefined && fromSha.length > 0 && cur === null) { + throw new SyncError( + `Divergence detected: ${toSha} does not descend from ${fromSha} for writer ${writerId}`, + { code: 'E_SYNC_DIVERGENCE', context: { writerId, fromSha, toSha } }, + ); + } + + // Yield in chronological order (oldest first) + for (let i = stack.length - 1; i >= 0; i--) { + const { sha, patchOid, encrypted } = /** @type {{ sha: string, patchOid: string, encrypted: boolean }} */ (stack[i]); + const raw = await adapter.readPatch(patchOid, { encrypted }); + const patch = _normalizePatch(raw); + yield new PatchEntry({ patch, sha }); + } + })(), + ); + } +} + +/** + * Normalizes a decoded patch (converts context from plain object to Map). + * + * @param {import('../../domain/types/WarpTypesV2.js').PatchV2} patch + * @returns {import('../../domain/types/WarpTypesV2.js').PatchV2} + */ +function _normalizePatch(patch) { + if (patch.context !== null && patch.context !== undefined && !(patch.context instanceof Map)) { + const ctx = patch.context; + if (ctx instanceof VersionVector) { + return patch; + } + /** @type {Record} */ + const record = Object.fromEntries(Object.entries(/** @type {Record} */ (ctx))); + return { ...patch, context: record }; + } + return patch; +} diff --git a/src/infrastructure/adapters/GitBlobWriteTransform.js b/src/infrastructure/adapters/GitBlobWriteTransform.js new file mode 100644 index 00000000..54590d29 --- /dev/null +++ b/src/infrastructure/adapters/GitBlobWriteTransform.js @@ -0,0 +1,40 @@ +import Transform from '../../domain/stream/Transform.js'; +import WarpError from '../../domain/errors/WarpError.js'; + +/** + * Stream transform that writes the bytes component of [path, bytes] entries + * as Git blobs and yields [path, oid]. + * + * Input: `[string, Uint8Array]` — path + blob content + * Output: `[string, string]` — path + Git blob OID + * + * @extends {Transform<[string, Uint8Array], [string, string]>} + */ +export class GitBlobWriteTransform extends Transform { + /** + * Creates a GitBlobWriteTransform. + * + * @param {import('../../ports/BlobPort.js').default} blobPort + */ + constructor(blobPort) { + super(); + if (blobPort === null || blobPort === undefined) { + throw new WarpError('GitBlobWriteTransform requires a blobPort', 'E_INVALID_DEPENDENCY'); + } + /** @type {import('../../ports/BlobPort.js').default} */ + this._blobPort = blobPort; + } + + /** + * Writes each [path, bytes] entry as a blob, yielding [path, oid]. + * + * @param {AsyncIterable<[string, Uint8Array]>} source + * @returns {AsyncIterable<[string, string]>} + */ + async *apply(source) { + for await (const [path, bytes] of source) { + const oid = await this._blobPort.writeBlob(bytes); + yield [path, oid]; + } + } +} diff --git a/src/infrastructure/adapters/IndexShardEncodeTransform.js b/src/infrastructure/adapters/IndexShardEncodeTransform.js new file mode 100644 index 00000000..944f6b92 --- /dev/null +++ b/src/infrastructure/adapters/IndexShardEncodeTransform.js @@ -0,0 +1,101 @@ +import Transform from '../../domain/stream/Transform.js'; +import { + MetaShard, + EdgeShard, + LabelShard, + PropertyShard, + ReceiptShard, +} from '../../domain/artifacts/IndexShard.js'; + +/** @typedef {import('../../domain/artifacts/IndexShard.js').IndexShard} IndexShard */ +import WarpError from '../../domain/errors/WarpError.js'; + +/** + * Stream transform that maps IndexShard instances to [path, bytes] entries. + * + * Owns path mapping (domain → Git tree path) AND CBOR encoding. + * The adapter knows which IndexShard subclass maps to which path. + * Domain never touches paths. + * + * Input: IndexShard (MetaShard | EdgeShard | LabelShard | PropertyShard | ReceiptShard) + * Output: [string, Uint8Array] — [Git tree path, CBOR bytes] + * + * @extends {Transform} + */ +export class IndexShardEncodeTransform extends Transform { + /** + * Creates an IndexShardEncodeTransform. + * + * @param {import('../../ports/CodecPort.js').default} codec + */ + constructor(codec) { + super(); + if (codec === null || codec === undefined) { + throw new WarpError('IndexShardEncodeTransform requires a codec', 'E_INVALID_DEPENDENCY'); + } + /** @type {import('../../ports/CodecPort.js').default} */ + this._codec = codec; + } + + /** + * Maps each IndexShard to [path, bytes] via instanceof dispatch. + * + * @param {AsyncIterable} source + * @returns {AsyncIterable<[string, Uint8Array]>} + */ + async *apply(source) { + for await (const shard of source) { + yield this._encode(shard); + } + } + + /** + * Maps a single IndexShard to [path, bytes]. + * + * @param {IndexShard} shard + * @returns {[string, Uint8Array]} + * @private + */ + _encode(shard) { + if (shard instanceof MetaShard) { + return [ + `meta_${shard.shardKey}.cbor`, + this._codec.encode({ + nodeToGlobal: shard.nodeToGlobal, + nextLocalId: shard.nextLocalId, + alive: shard.alive, + }), + ]; + } + if (shard instanceof EdgeShard) { + return [ + `${shard.direction}_${shard.shardKey}.cbor`, + this._codec.encode(shard.buckets), + ]; + } + if (shard instanceof LabelShard) { + return [ + 'labels.cbor', + this._codec.encode(shard.labels), + ]; + } + if (shard instanceof PropertyShard) { + return [ + `props_${shard.shardKey}.cbor`, + this._codec.encode(shard.entries), + ]; + } + if (shard instanceof ReceiptShard) { + return [ + 'receipt.cbor', + this._codec.encode({ + version: shard.version, + nodeCount: shard.nodeCount, + labelCount: shard.labelCount, + shardCount: shard.shardCount, + }), + ]; + } + throw new WarpError('Unknown IndexShard type', 'E_UNKNOWN_SHARD'); + } +} diff --git a/src/infrastructure/adapters/TreeAssemblerSink.js b/src/infrastructure/adapters/TreeAssemblerSink.js new file mode 100644 index 00000000..c95e2623 --- /dev/null +++ b/src/infrastructure/adapters/TreeAssemblerSink.js @@ -0,0 +1,49 @@ +import Sink from '../../domain/stream/Sink.js'; +import WarpError from '../../domain/errors/WarpError.js'; + +/** + * Stream sink that accumulates [path, oid] entries and assembles them + * into a Git tree on finalization. + * + * Consumes: `[string, string]` — path + blob OID + * Produces: `string` — the Git tree OID + * + * @extends {Sink<[string, string], string>} + */ +export class TreeAssemblerSink extends Sink { + /** + * Creates a TreeAssemblerSink. + * + * @param {import('../../ports/TreePort.js').default} treePort + */ + constructor(treePort) { + super(); + if (treePort === null || treePort === undefined) { + throw new WarpError('TreeAssemblerSink requires a treePort', 'E_INVALID_DEPENDENCY'); + } + /** @type {import('../../ports/TreePort.js').default} */ + this._treePort = treePort; + /** @type {string[]} mktree-formatted entries */ + this._entries = []; + } + + /** + * Accepts a [path, oid] entry and formats it for mktree. + * + * @param {[string, string]} item + */ + _accept(item) { + const [path, oid] = item; + this._entries.push(`100644 blob ${oid}\t${path}`); + } + + /** + * Builds the Git tree from accumulated entries. + * + * @returns {Promise} The tree OID + */ + async _finalize() { + this._entries.sort(); + return await this._treePort.writeTree(this._entries); + } +} diff --git a/src/ports/CheckpointStorePort.js b/src/ports/CheckpointStorePort.js new file mode 100644 index 00000000..1c30a31a --- /dev/null +++ b/src/ports/CheckpointStorePort.js @@ -0,0 +1,71 @@ +import WarpError from '../domain/errors/WarpError.js'; + +/** + * @typedef {{ + * state: import('../domain/services/JoinReducer.js').WarpStateV5, + * frontier: Map, + * appliedVV: import('../domain/crdt/VersionVector.js').default, + * stateHash: string, + * provenanceIndex?: import('../domain/services/provenance/ProvenanceIndex.js').ProvenanceIndex | null, + * }} CheckpointRecord + */ + +/** + * @typedef {{ + * stateBlobOid: string, + * frontierBlobOid: string, + * appliedVVBlobOid: string, + * provenanceIndexBlobOid: string | null, + * }} CheckpointWriteResult + */ + +/** + * @typedef {{ + * state: import('../domain/services/JoinReducer.js').WarpStateV5, + * frontier: Map, + * appliedVV: import('../domain/crdt/VersionVector.js').default | null, + * stateHash: string, + * schema: number, + * provenanceIndex?: import('../domain/services/provenance/ProvenanceIndex.js').ProvenanceIndex | null, + * indexShardOids: Record | null, + * }} CheckpointData + */ + +/** + * Port for checkpoint persistence. + * + * A checkpoint is one domain event with multiple persistence artifacts. + * The port speaks one semantic operation (writeCheckpoint, readCheckpoint), + * not individual blob writes. The adapter internally fans artifacts out + * through the stream pipeline. + * + * @abstract + * @see CborCheckpointStoreAdapter - Reference implementation + */ +export default class CheckpointStorePort { + /** + * Persists a complete checkpoint and returns write results. + * + * The adapter internally encodes and writes state, frontier, + * appliedVV (and optionally provenanceIndex) as separate blobs, + * assembles a Git tree, and returns the OIDs. + * + * @param {CheckpointRecord} _record - The checkpoint artifacts to persist + * @returns {Promise} + * @throws {Error} If not implemented by a concrete adapter + */ + async writeCheckpoint(_record) { + throw new WarpError('CheckpointStorePort.writeCheckpoint() not implemented', 'E_NOT_IMPLEMENTED'); + } + + /** + * Reads a checkpoint from a tree of OIDs. + * + * @param {Record} _treeOids - Map of path → blob OID from the checkpoint tree + * @returns {Promise} + * @throws {Error} If not implemented by a concrete adapter + */ + async readCheckpoint(_treeOids) { + throw new WarpError('CheckpointStorePort.readCheckpoint() not implemented', 'E_NOT_IMPLEMENTED'); + } +} diff --git a/src/ports/PatchJournalPort.js b/src/ports/PatchJournalPort.js new file mode 100644 index 00000000..a05ccbda --- /dev/null +++ b/src/ports/PatchJournalPort.js @@ -0,0 +1,71 @@ +import WarpError from '../domain/errors/WarpError.js'; + +/** + * Port for patch journal persistence. + * + * Domain-facing port that speaks PatchV2 domain objects. No bytes cross + * this boundary. The adapter implementation owns the codec and talks to + * the raw Git ports (BlobPort, BlobStoragePort) internally. + * + * This is part of the two-stage persistence boundary (P5 compliance): + * Domain Service → PatchJournalPort (domain objects) + * → Adapter (codec + raw Git ports) → Git + * + * @abstract + * @see CborPatchJournalAdapter - Reference implementation + */ +export default class PatchJournalPort { + /** + * Persists a patch and returns its storage OID. + * + * @param {import('../domain/types/WarpTypesV2.js').PatchV2} _patch - The patch to persist + * @returns {Promise} The storage OID (opaque handle — domain doesn't care it's a Git blob SHA) + * @throws {Error} If not implemented by a concrete adapter + */ + async writePatch(_patch) { + throw new WarpError('PatchJournalPort.writePatch() not implemented', 'E_NOT_IMPLEMENTED'); + } + + /** + * Reads a patch by its storage OID. + * + * @param {string} _patchOid - The storage OID returned by writePatch + * @param {{ encrypted?: boolean }} [_options] - Read options + * @returns {Promise} The decoded patch + * @throws {Error} If not implemented by a concrete adapter + * @throws {Error} If the patch blob is not found + */ + async readPatch(_patchOid, _options) { + throw new WarpError('PatchJournalPort.readPatch() not implemented', 'E_NOT_IMPLEMENTED'); + } + + /** + * Whether this journal uses external blob storage. + * + * When true, readers must use the `encrypted` flag in the commit + * message trailer to retrieve blobs via BlobStoragePort rather than + * reading them directly from Git. + * + * @returns {boolean} + */ + get usesExternalStorage() { + return false; + } + + /** + * Scans patches in a writer's chain between two SHAs, yielding + * PatchEntry instances in chronological order (oldest first). + * + * This is the unbounded streaming alternative to the legacy + * loadPatchRange() which returns a whole array. + * + * @param {string} _writerId - The writer whose chain to scan + * @param {string|null} _fromSha - Start SHA (exclusive), null for all + * @param {string} _toSha - End SHA (inclusive) + * @returns {import('../domain/stream/WarpStream.js').default} + * @throws {Error} If not implemented by a concrete adapter + */ + scanPatchRange(_writerId, _fromSha, _toSha) { + throw new WarpError('PatchJournalPort.scanPatchRange() not implemented', 'E_NOT_IMPLEMENTED'); + } +} diff --git a/test/helpers/MockBlobPort.js b/test/helpers/MockBlobPort.js new file mode 100644 index 00000000..dc901daa --- /dev/null +++ b/test/helpers/MockBlobPort.js @@ -0,0 +1,35 @@ +import { vi } from 'vitest'; +import BlobPort from '../../src/ports/BlobPort.js'; + +/** + * In-memory BlobPort for tests. + * + * Stores blobs in a Map and returns deterministic OIDs. + * Methods are Vitest spies so callers can assert on calls. + */ +export default class MockBlobPort extends BlobPort { + constructor() { + super(); + /** @type {Map} */ + this.store = new Map(); + /** @type {number} */ + this._counter = 0; + + // Bind spy wrappers so vitest assertions work + const self = this; + + /** @type {import('vitest').Mock} */ + this.writeBlob = vi.fn(async (/** @type {Uint8Array} */ content) => { + const oid = `blob_${String(self._counter++).padStart(40, '0')}`; + self.store.set(oid, content); + return oid; + }); + + /** @type {import('vitest').Mock} */ + this.readBlob = vi.fn(async (/** @type {string} */ oid) => { + const data = self.store.get(oid); + if (!data) { throw new Error(`Blob not found: ${oid}`); } + return data; + }); + } +} diff --git a/test/unit/boundary/patch-codec-tripwire.test.js b/test/unit/boundary/patch-codec-tripwire.test.js new file mode 100644 index 00000000..65f3a9d5 --- /dev/null +++ b/test/unit/boundary/patch-codec-tripwire.test.js @@ -0,0 +1,82 @@ +/** + * Hex Tripwire Test — Patch Serialization Boundary + * + * This test enforces P5: "Serialization Is the Codec's Problem." + * Domain services must not import defaultCodec, call codec.encode(), + * or call codec.decode() for patch persistence. + * + * When this test fails, it means a domain file is speaking bytes + * instead of domain objects. Fix the file, not the test. + */ +import { describe, it, expect } from 'vitest'; +import { readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import { dirname, resolve } from 'node:path'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const ROOT = resolve(__dirname, '..', '..', '..'); + +/** + * Files that must be codec-free after the P5 dissolution. + * Add files here as each artifact family is migrated. + */ +const PATCH_FILES = [ + 'src/domain/services/PatchBuilderV2.js', + 'src/domain/services/sync/SyncProtocol.js', + 'src/domain/warp/Writer.js', +]; + +// Checkpoint files that are already codec-free (CheckpointService routes +// through CheckpointStorePort when available). The serializer files +// (CheckpointSerializerV5, StateSerializerV5, Frontier) are NOT yet in +// the tripwire — they still export legacy serialize/deserialize functions +// used by callers that haven't been migrated (MaterializeController, +// BoundaryTransitionRecord, etc.). Add them when ALL callers are migrated. +const CHECKPOINT_FILES = [ + 'src/domain/services/state/CheckpointService.js', +]; + +/** + * Forbidden patterns in domain files that handle patch persistence. + * Each pattern indicates bytes leaking into the domain layer. + */ +const FORBIDDEN_PATTERNS = [ + { pattern: /import\s+.*defaultCodec/, label: 'imports defaultCodec' }, + { pattern: /from\s+['"].*defaultCodec/, label: 'imports from defaultCodec module' }, + { pattern: /['"]cbor-x['"]/, label: 'imports cbor-x directly' }, + { pattern: /this\._codec\.encode\(/, label: 'calls this._codec.encode()' }, + { pattern: /this\._codec\.decode\(/, label: 'calls this._codec.decode()' }, + { pattern: /codec\.encode\(/, label: 'calls codec.encode()' }, + { pattern: /codec\.decode\(/, label: 'calls codec.decode()' }, + { pattern: /codecOpt\.encode\(/, label: 'calls codecOpt.encode()' }, + { pattern: /codecOpt\.decode\(/, label: 'calls codecOpt.decode()' }, +]; + +/** + * Runs tripwire checks on a list of files. + * @param {string} suiteName + * @param {string[]} files + */ +function tripwireSuite(suiteName, files) { + describe(suiteName, () => { + for (const relPath of files) { + describe(relPath, () => { + const absPath = resolve(ROOT, relPath); + const source = readFileSync(absPath, 'utf-8'); + + for (const { pattern, label } of FORBIDDEN_PATTERNS) { + it(`must not contain: ${label}`, () => { + const matches = source.match(pattern); + expect( + matches, + `${relPath} violates P5: ${label}\nMatch: ${matches?.[0]}`, + ).toBeNull(); + }); + } + }); + } + }); +} + +tripwireSuite('P5 tripwire: patch files must not touch codec/bytes', PATCH_FILES); +tripwireSuite('P5 tripwire: checkpoint files must not touch codec/bytes', CHECKPOINT_FILES); diff --git a/test/unit/domain/artifacts/CheckpointArtifact.test.js b/test/unit/domain/artifacts/CheckpointArtifact.test.js new file mode 100644 index 00000000..8ae2a9e1 --- /dev/null +++ b/test/unit/domain/artifacts/CheckpointArtifact.test.js @@ -0,0 +1,81 @@ +import { describe, it, expect } from 'vitest'; +import { + CheckpointArtifact, + StateArtifact, + FrontierArtifact, + AppliedVVArtifact, +} from '../../../../src/domain/artifacts/CheckpointArtifact.js'; +import { createVersionVector } from '../../../../src/domain/crdt/VersionVector.js'; +import { createEmptyStateV5 } from '../../../../src/domain/services/JoinReducer.js'; + +describe('CheckpointArtifact family', () => { + describe('StateArtifact', () => { + it('constructs with valid fields', () => { + const a = new StateArtifact({ schemaVersion: 2, state: createEmptyStateV5() }); + expect(a).toBeInstanceOf(StateArtifact); + expect(a).toBeInstanceOf(CheckpointArtifact); + expect(a.schemaVersion).toBe(2); + expect(a.state).toBeDefined(); + }); + + it('is frozen', () => { + const a = new StateArtifact({ schemaVersion: 2, state: createEmptyStateV5() }); + expect(Object.isFrozen(a)).toBe(true); + }); + + it('rejects null state', () => { + expect(() => new StateArtifact({ schemaVersion: 2, state: /** @type {any} */ (null) })).toThrow('requires a state'); + }); + + it('rejects invalid schemaVersion', () => { + expect(() => new StateArtifact({ schemaVersion: 0, state: createEmptyStateV5() })).toThrow('positive integer'); + }); + }); + + describe('FrontierArtifact', () => { + it('constructs with a Map', () => { + const a = new FrontierArtifact({ schemaVersion: 2, frontier: new Map([['w1', 'abc']]) }); + expect(a).toBeInstanceOf(FrontierArtifact); + expect(a).toBeInstanceOf(CheckpointArtifact); + expect(a.frontier.get('w1')).toBe('abc'); + }); + + it('rejects non-Map frontier', () => { + expect(() => new FrontierArtifact({ schemaVersion: 2, frontier: /** @type {any} */ ({}) })).toThrow('requires a Map'); + }); + }); + + describe('AppliedVVArtifact', () => { + it('constructs with a VersionVector', () => { + const vv = createVersionVector(); + vv.set('w1', 5); + const a = new AppliedVVArtifact({ schemaVersion: 2, appliedVV: vv }); + expect(a).toBeInstanceOf(AppliedVVArtifact); + expect(a).toBeInstanceOf(CheckpointArtifact); + expect(a.appliedVV.get('w1')).toBe(5); + }); + + it('rejects null appliedVV', () => { + expect(() => new AppliedVVArtifact({ schemaVersion: 2, appliedVV: /** @type {any} */ (null) })).toThrow('requires an appliedVV'); + }); + }); + + describe('instanceof dispatch', () => { + it('dispatches correctly across all subtypes', () => { + const state = new StateArtifact({ schemaVersion: 2, state: createEmptyStateV5() }); + const frontier = new FrontierArtifact({ schemaVersion: 2, frontier: new Map() }); + const vv = new AppliedVVArtifact({ schemaVersion: 2, appliedVV: createVersionVector() }); + + expect(state instanceof StateArtifact).toBe(true); + expect(state instanceof FrontierArtifact).toBe(false); + expect(frontier instanceof FrontierArtifact).toBe(true); + expect(frontier instanceof StateArtifact).toBe(false); + expect(vv instanceof AppliedVVArtifact).toBe(true); + + // All are CheckpointArtifact + expect(state instanceof CheckpointArtifact).toBe(true); + expect(frontier instanceof CheckpointArtifact).toBe(true); + expect(vv instanceof CheckpointArtifact).toBe(true); + }); + }); +}); diff --git a/test/unit/domain/artifacts/IndexShard.test.js b/test/unit/domain/artifacts/IndexShard.test.js new file mode 100644 index 00000000..02356263 --- /dev/null +++ b/test/unit/domain/artifacts/IndexShard.test.js @@ -0,0 +1,131 @@ +import { describe, it, expect } from 'vitest'; +import { + IndexShard, + MetaShard, + EdgeShard, + LabelShard, + PropertyShard, + ReceiptShard, +} from '../../../../src/domain/artifacts/IndexShard.js'; + +describe('IndexShard family', () => { + describe('MetaShard', () => { + it('constructs with valid fields', () => { + const s = new MetaShard({ + shardKey: 'ab', + nodeToGlobal: [['user:alice', 0]], + nextLocalId: 1, + alive: new Uint8Array([1, 2, 3]), + }); + expect(s).toBeInstanceOf(MetaShard); + expect(s).toBeInstanceOf(IndexShard); + expect(s.shardKey).toBe('ab'); + expect(s.nodeToGlobal).toHaveLength(1); + expect(s.nextLocalId).toBe(1); + }); + + it('is frozen', () => { + const s = new MetaShard({ + shardKey: 'ab', nodeToGlobal: [], nextLocalId: 0, alive: new Uint8Array(0), + }); + expect(Object.isFrozen(s)).toBe(true); + }); + + it('defaults schemaVersion to 1', () => { + const s = new MetaShard({ + shardKey: 'ab', nodeToGlobal: [], nextLocalId: 0, alive: new Uint8Array(0), + }); + expect(s.schemaVersion).toBe(1); + }); + }); + + describe('EdgeShard', () => { + it('constructs with fwd direction', () => { + const s = new EdgeShard({ + shardKey: 'ab', direction: 'fwd', buckets: { all: { '0': new Uint8Array(0) } }, + }); + expect(s).toBeInstanceOf(EdgeShard); + expect(s.direction).toBe('fwd'); + }); + + it('constructs with rev direction', () => { + const s = new EdgeShard({ + shardKey: 'ab', direction: 'rev', buckets: {}, + }); + expect(s.direction).toBe('rev'); + }); + + it('rejects invalid direction', () => { + expect(() => new EdgeShard({ + shardKey: 'ab', direction: /** @type {any} */ ('up'), buckets: {}, + })).toThrow("must be 'fwd' or 'rev'"); + }); + }); + + describe('LabelShard', () => { + it('constructs with labels', () => { + const s = new LabelShard({ labels: [['knows', 0], ['likes', 1]] }); + expect(s).toBeInstanceOf(LabelShard); + expect(s.labels).toHaveLength(2); + expect(s.shardKey).toBe('global'); + }); + }); + + describe('PropertyShard', () => { + it('constructs with entries', () => { + const s = new PropertyShard({ + shardKey: 'ab', entries: [['user:alice', { name: 'Alice' }]], + }); + expect(s).toBeInstanceOf(PropertyShard); + expect(s.entries).toHaveLength(1); + }); + }); + + describe('ReceiptShard', () => { + it('constructs with build metadata', () => { + const s = new ReceiptShard({ + version: 1, nodeCount: 100, labelCount: 5, shardCount: 16, + }); + expect(s).toBeInstanceOf(ReceiptShard); + expect(s).toBeInstanceOf(IndexShard); + expect(s.nodeCount).toBe(100); + expect(s.shardKey).toBe('receipt'); + }); + }); + + describe('instanceof dispatch', () => { + it('dispatches correctly across all subtypes', () => { + const meta = new MetaShard({ shardKey: 'ab', nodeToGlobal: [], nextLocalId: 0, alive: new Uint8Array(0) }); + const edge = new EdgeShard({ shardKey: 'ab', direction: 'fwd', buckets: {} }); + const label = new LabelShard({ labels: [] }); + const prop = new PropertyShard({ shardKey: 'ab', entries: [] }); + const receipt = new ReceiptShard({ version: 1, nodeCount: 0, labelCount: 0, shardCount: 0 }); + + expect(meta instanceof MetaShard).toBe(true); + expect(meta instanceof EdgeShard).toBe(false); + expect(edge instanceof EdgeShard).toBe(true); + expect(label instanceof LabelShard).toBe(true); + expect(prop instanceof PropertyShard).toBe(true); + expect(receipt instanceof ReceiptShard).toBe(true); + + // All are IndexShard + for (const s of [meta, edge, label, prop, receipt]) { + expect(s instanceof IndexShard).toBe(true); + } + }); + }); + + describe('constructor validation', () => { + it('rejects non-string shardKey', () => { + expect(() => new MetaShard({ + shardKey: /** @type {any} */ (42), nodeToGlobal: [], nextLocalId: 0, alive: new Uint8Array(0), + })).toThrow('shardKey must be a non-empty string'); + }); + + it('rejects invalid schemaVersion', () => { + expect(() => new MetaShard({ + shardKey: 'ab', schemaVersion: 0, nodeToGlobal: [], nextLocalId: 0, alive: new Uint8Array(0), + })).toThrow('positive integer'); + }); + }); +}); diff --git a/test/unit/domain/artifacts/PatchEntry.test.js b/test/unit/domain/artifacts/PatchEntry.test.js new file mode 100644 index 00000000..c8f215a3 --- /dev/null +++ b/test/unit/domain/artifacts/PatchEntry.test.js @@ -0,0 +1,53 @@ +import { describe, it, expect } from 'vitest'; +import PatchEntry from '../../../../src/domain/artifacts/PatchEntry.js'; +import ProvenanceEntry from '../../../../src/domain/artifacts/ProvenanceEntry.js'; +import { PatchV2 } from '../../../../src/domain/types/WarpTypesV2.js'; + +/** @returns {PatchV2} */ +function minimalPatch() { + return new PatchV2({ schema: 2, writer: 'w1', lamport: 1, context: {}, ops: [] }); +} + +describe('PatchEntry', () => { + it('constructs with valid fields', () => { + const e = new PatchEntry({ patch: minimalPatch(), sha: 'a'.repeat(40) }); + expect(e).toBeInstanceOf(PatchEntry); + expect(e.patch.schema).toBe(2); + expect(e.sha).toBe('a'.repeat(40)); + }); + + it('is frozen', () => { + const e = new PatchEntry({ patch: minimalPatch(), sha: 'a'.repeat(40) }); + expect(Object.isFrozen(e)).toBe(true); + }); + + it('rejects null patch', () => { + expect(() => new PatchEntry({ patch: /** @type {any} */ (null), sha: 'abc' })).toThrow('requires a patch'); + }); + + it('rejects empty sha', () => { + expect(() => new PatchEntry({ patch: minimalPatch(), sha: '' })).toThrow('non-empty sha'); + }); +}); + +describe('ProvenanceEntry', () => { + it('constructs with valid fields', () => { + const e = new ProvenanceEntry({ entityId: 'user:alice', patchShas: new Set(['abc']) }); + expect(e).toBeInstanceOf(ProvenanceEntry); + expect(e.entityId).toBe('user:alice'); + expect(e.patchShas.has('abc')).toBe(true); + }); + + it('is frozen', () => { + const e = new ProvenanceEntry({ entityId: 'x', patchShas: new Set() }); + expect(Object.isFrozen(e)).toBe(true); + }); + + it('rejects empty entityId', () => { + expect(() => new ProvenanceEntry({ entityId: '', patchShas: new Set() })).toThrow('non-empty entityId'); + }); + + it('rejects non-Set patchShas', () => { + expect(() => new ProvenanceEntry({ entityId: 'x', patchShas: /** @type {any} */ ([]) })).toThrow('requires a Set'); + }); +}); diff --git a/test/unit/domain/services/PatchBuilderV2.cas.test.js b/test/unit/domain/services/PatchBuilderV2.cas.test.js index 466a8f84..ab2363a2 100644 --- a/test/unit/domain/services/PatchBuilderV2.cas.test.js +++ b/test/unit/domain/services/PatchBuilderV2.cas.test.js @@ -2,6 +2,8 @@ import { describe, it, expect, vi } from 'vitest'; import { PatchBuilderV2 } from '../../../../src/domain/services/PatchBuilderV2.js'; import { WriterError } from '../../../../src/domain/warp/Writer.js'; import { createVersionVector } from '../../../../src/domain/crdt/VersionVector.js'; +import { CborPatchJournalAdapter } from '../../../../src/infrastructure/adapters/CborPatchJournalAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; /** * Creates a mock persistence adapter for CAS testing. @@ -22,6 +24,18 @@ function createMockPersistence(overrides = {}) { }; } +/** + * Creates a CborPatchJournalAdapter wired to the given persistence's blob ops. + * @param {ReturnType} persistence + * @returns {CborPatchJournalAdapter} + */ +function createPatchJournal(persistence) { + return new CborPatchJournalAdapter({ + codec: new CborCodec(), + blobPort: persistence, + }); +} + describe('PatchBuilderV2 CAS conflict detection', () => { // --------------------------------------------------------------- // CAS conflict: ref advanced between createPatch and commit @@ -193,6 +207,7 @@ describe('PatchBuilderV2 CAS conflict detection', () => { const builder = new PatchBuilderV2({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -222,6 +237,7 @@ describe('PatchBuilderV2 CAS conflict detection', () => { const builder = new PatchBuilderV2({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, diff --git a/test/unit/domain/services/PatchBuilderV2.content.test.js b/test/unit/domain/services/PatchBuilderV2.content.test.js index 3e07de59..a03e19a6 100644 --- a/test/unit/domain/services/PatchBuilderV2.content.test.js +++ b/test/unit/domain/services/PatchBuilderV2.content.test.js @@ -4,6 +4,8 @@ import { createVersionVector } from '../../../../src/domain/crdt/VersionVector.j import { createORSet, orsetAdd } from '../../../../src/domain/crdt/ORSet.js'; import { createDot } from '../../../../src/domain/crdt/Dot.js'; import { encodeEdgeKey } from '../../../../src/domain/services/KeyCodec.js'; +import { CborPatchJournalAdapter } from '../../../../src/infrastructure/adapters/CborPatchJournalAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; /** * Creates a mock blob storage with configurable OID return. @@ -50,6 +52,18 @@ function createMockState() { }; } +/** + * Creates a CborPatchJournalAdapter wired to the given persistence's blob ops. + * @param {ReturnType} persistence + * @returns {CborPatchJournalAdapter} + */ +function createPatchJournal(persistence) { + return new CborPatchJournalAdapter({ + codec: new CborCodec(), + blobPort: persistence, + }); +} + describe('PatchBuilderV2 content attachment', () => { describe('attachContent()', () => { it('writes blob and sets content reference metadata properties', async () => { @@ -687,6 +701,7 @@ describe('PatchBuilderV2 content attachment', () => { }); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'g', writerId: 'w1', lamport: 1, @@ -711,6 +726,7 @@ describe('PatchBuilderV2 content attachment', () => { const persistence = createMockPersistence(); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'g', writerId: 'w1', lamport: 1, @@ -742,6 +758,7 @@ describe('PatchBuilderV2 content attachment', () => { }); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'g', writerId: 'w1', lamport: 1, @@ -773,6 +790,7 @@ describe('PatchBuilderV2 content attachment', () => { }); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'g', writerId: 'w1', lamport: 1, diff --git a/test/unit/domain/services/PatchBuilderV2.test.js b/test/unit/domain/services/PatchBuilderV2.test.js index d8ce533c..045fa3b4 100644 --- a/test/unit/domain/services/PatchBuilderV2.test.js +++ b/test/unit/domain/services/PatchBuilderV2.test.js @@ -6,6 +6,8 @@ import { createDot } from '../../../../src/domain/crdt/Dot.js'; import { encodeEdgeKey } from '../../../../src/domain/services/JoinReducer.js'; import { decodePatchMessage } from '../../../../src/domain/services/codec/WarpMessageCodec.js'; import { decode } from '../../../../src/infrastructure/codecs/CborCodec.js'; +import { CborPatchJournalAdapter } from '../../../../src/infrastructure/adapters/CborPatchJournalAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; /** * Creates a mock V5 state for testing. @@ -35,6 +37,18 @@ function createMockPersistence() { }; } +/** + * Creates a CborPatchJournalAdapter wired to the given mock persistence's blob ops. + * @param {ReturnType} persistence + * @returns {CborPatchJournalAdapter} + */ +function createPatchJournal(persistence) { + return new CborPatchJournalAdapter({ + codec: new CborCodec(), + blobPort: persistence, + }); +} + describe('PatchBuilderV2', () => { describe('building patch with node add', () => { it('creates NodeAdd operation with dot', () => { @@ -555,6 +569,7 @@ describe('PatchBuilderV2', () => { const persistence = createMockPersistence(); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -593,6 +608,7 @@ describe('PatchBuilderV2', () => { const persistence = createMockPersistence(); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -625,6 +641,7 @@ describe('PatchBuilderV2', () => { const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, // Constructor lamport is 1, but commit should use 6 @@ -649,6 +666,7 @@ describe('PatchBuilderV2', () => { const persistence = createMockPersistence(); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -667,11 +685,13 @@ describe('PatchBuilderV2', () => { it('writes patch blob with CBOR encoding', async () => { const persistence = createMockPersistence(); + const patchJournal = createPatchJournal(persistence); const vv = createVersionVector(); vv.set('otherWriter', 3); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal, graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -704,6 +724,7 @@ describe('PatchBuilderV2', () => { const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -728,6 +749,7 @@ describe('PatchBuilderV2', () => { const persistence = createMockPersistence(); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -798,6 +820,7 @@ describe('PatchBuilderV2', () => { persistence.readRef.mockImplementation(() => readRefPromise); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -819,6 +842,7 @@ describe('PatchBuilderV2', () => { const persistence = createMockPersistence(); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -842,6 +866,7 @@ describe('PatchBuilderV2', () => { persistence.updateRef.mockRejectedValueOnce(new Error('simulated updateRef failure')); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -1215,8 +1240,10 @@ describe('PatchBuilderV2', () => { describe('commit() includes reads/writes', () => { it('committed patch includes reads/writes arrays', async () => { const persistence = createMockPersistence(); + const patchJournal = createPatchJournal(persistence); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal, graphName: 'test-graph', writerId: 'writer1', lamport: 1, @@ -1238,8 +1265,10 @@ describe('PatchBuilderV2', () => { it('committed patch omits empty reads array', async () => { const persistence = createMockPersistence(); + const patchJournal = createPatchJournal(persistence); const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal, graphName: 'test-graph', writerId: 'writer1', lamport: 1, diff --git a/test/unit/domain/services/SyncController.test.js b/test/unit/domain/services/SyncController.test.js index e7c21be8..6502b67a 100644 --- a/test/unit/domain/services/SyncController.test.js +++ b/test/unit/domain/services/SyncController.test.js @@ -501,7 +501,9 @@ describe('SyncController', () => { it('delegates to SyncProtocol.processSyncRequest with correct args', async () => { const mockResponse = { type: 'sync-response', frontier: {}, patches: [] }; processSyncRequestMock.mockResolvedValue(mockResponse); + const mockPatchJournal = { writePatch: vi.fn(), readPatch: vi.fn() }; const host = createMockHost({ + _patchJournal: mockPatchJournal, discoverWriters: vi.fn().mockResolvedValue(['alice']), _persistence: { readRef: vi.fn().mockResolvedValue('sha-alice'), @@ -519,7 +521,7 @@ describe('SyncController', () => { expect.any(Map), host['_persistence'], 'test-graph', - expect.objectContaining({ codec: host['_codec'] }), + expect.objectContaining({ patchJournal: mockPatchJournal }), ); }); }); diff --git a/test/unit/domain/services/SyncProtocol.divergence.test.js b/test/unit/domain/services/SyncProtocol.divergence.test.js index 46b7880a..bb896a42 100644 --- a/test/unit/domain/services/SyncProtocol.divergence.test.js +++ b/test/unit/domain/services/SyncProtocol.divergence.test.js @@ -10,6 +10,8 @@ import { processSyncRequest } from '../../../../src/domain/services/sync/SyncPro import { encodePatchMessage } from '../../../../src/domain/services/codec/WarpMessageCodec.js'; import { encode } from '../../../../src/infrastructure/codecs/CborCodec.js'; import { createVersionVector } from '../../../../src/domain/crdt/VersionVector.js'; +import { CborPatchJournalAdapter } from '../../../../src/infrastructure/adapters/CborPatchJournalAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; const SHA_A = 'a'.repeat(40); const SHA_B = 'b'.repeat(40); @@ -52,6 +54,14 @@ function createMockPersistence(/** @type {Record} */ commits = {}, }; } +function createPatchJournal(/** @type {any} */ persistence) { + return new CborPatchJournalAdapter({ + codec: new CborCodec(), + blobPort: persistence, + commitPort: persistence, + }); +} + function createMockLogger() { return { debug: vi.fn(), @@ -81,7 +91,7 @@ describe('B65 — Sync divergence logging', () => { const localFrontier = new Map([['w1', SHA_B]]); const response = await processSyncRequest( - /** @type {*} */ (request), localFrontier, /** @type {any} */ (persistence), 'events', { logger }, + /** @type {*} */ (request), localFrontier, /** @type {any} */ (persistence), 'events', { patchJournal: createPatchJournal(persistence), logger }, ); // Should return empty patches (diverged writer skipped) @@ -115,7 +125,7 @@ describe('B65 — Sync divergence logging', () => { const localFrontier = new Map([['w1', SHA_B]]); const response = await processSyncRequest( - /** @type {*} */ (request), localFrontier, /** @type {any} */ (persistence), 'events', { logger }, + /** @type {*} */ (request), localFrontier, /** @type {any} */ (persistence), 'events', { patchJournal: createPatchJournal(persistence), logger }, ); expect(response.patches).toHaveLength(1); @@ -138,7 +148,7 @@ describe('B65 — Sync divergence logging', () => { const localFrontier = new Map([['w1', SHA_B]]); const response = await processSyncRequest( - /** @type {*} */ (request), localFrontier, /** @type {any} */ (persistence), 'events', + /** @type {*} */ (request), localFrontier, /** @type {any} */ (persistence), 'events', { patchJournal: createPatchJournal(persistence) }, ); expect(response.patches).toHaveLength(0); @@ -162,7 +172,7 @@ describe('B65 — Sync divergence logging', () => { const localFrontier = new Map([['w1', SHA_A], ['w2', SHA_C]]); const response = await processSyncRequest( - /** @type {*} */ (request), localFrontier, /** @type {any} */ (persistence), 'events', { logger }, + /** @type {*} */ (request), localFrontier, /** @type {any} */ (persistence), 'events', { patchJournal: createPatchJournal(persistence), logger }, ); // w1 skipped (diverged), w2 returned diff --git a/test/unit/domain/services/SyncProtocol.stateCoherence.test.js b/test/unit/domain/services/SyncProtocol.stateCoherence.test.js index b5a2787b..9ab1c8f1 100644 --- a/test/unit/domain/services/SyncProtocol.stateCoherence.test.js +++ b/test/unit/domain/services/SyncProtocol.stateCoherence.test.js @@ -20,6 +20,8 @@ import { orsetElements } from '../../../../src/domain/crdt/ORSet.js'; import { encodePatchMessage } from '../../../../src/domain/services/codec/WarpMessageCodec.js'; import { encode } from '../../../../src/infrastructure/codecs/CborCodec.js'; import { createVersionVector } from '../../../../src/domain/crdt/VersionVector.js'; +import { CborPatchJournalAdapter } from '../../../../src/infrastructure/adapters/CborPatchJournalAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; // --------------------------------------------------------------------------- // Helpers @@ -68,6 +70,14 @@ function stateSignature(/** @type {any} */ state) { return { nodes, edges, props }; } +function createPatchJournal(/** @type {any} */ persistence) { + return new CborPatchJournalAdapter({ + codec: new CborCodec(), + blobPort: persistence, + commitPort: persistence, + }); +} + function createMockLogger() { return { debug: vi.fn(), @@ -293,7 +303,7 @@ describe('SyncProtocol — state coherence (Phase 4, Invariant 5)', () => { localFrontier, /** @type {any} */ (persistence), 'events', - { logger }, + { patchJournal: createPatchJournal(persistence), logger }, )); // Patches for diverged writer should be empty diff --git a/test/unit/domain/services/SyncProtocol.test.js b/test/unit/domain/services/SyncProtocol.test.js index f529be94..1a8a4df9 100644 --- a/test/unit/domain/services/SyncProtocol.test.js +++ b/test/unit/domain/services/SyncProtocol.test.js @@ -20,6 +20,8 @@ import { orsetContains } from '../../../../src/domain/crdt/ORSet.js'; import { createVersionVector } from '../../../../src/domain/crdt/VersionVector.js'; import { encodePatchMessage } from '../../../../src/domain/services/codec/WarpMessageCodec.js'; import { encode } from '../../../../src/infrastructure/codecs/CborCodec.js'; +import { CborPatchJournalAdapter } from '../../../../src/infrastructure/adapters/CborPatchJournalAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; // ----------------------------------------------------------------------------- // Test Fixtures and Helpers @@ -92,6 +94,19 @@ function createMockPersistence(commits = /** @type {any} */ ({}), blobs = /** @t }; } +/** + * Creates a CborPatchJournalAdapter wired to the given mock persistence's blob ops. + * @param {ReturnType} persistence + * @returns {CborPatchJournalAdapter} + */ +function createPatchJournal(persistence) { + return new CborPatchJournalAdapter({ + codec: new CborCodec(), + blobPort: persistence, + commitPort: persistence, + }); +} + /** * Creates a commit message and blob for a test patch. */ @@ -208,7 +223,7 @@ describe('SyncProtocol', () => { const persistence = createMockPersistence(commits, blobs); // Load from SHA_A (exclusive) to SHA_C (inclusive) - const patches = await loadPatchRange(persistence, 'events', 'w1', SHA_A, SHA_C); + const patches = await loadPatchRange(persistence, 'events', 'w1', SHA_A, SHA_C, { patchJournal: createPatchJournal(persistence) }); expect(patches).toHaveLength(2); expect(/** @type {any} */ (patches)[0].sha).toBe(SHA_B); @@ -230,7 +245,7 @@ describe('SyncProtocol', () => { const persistence = createMockPersistence(commits, blobs); - const patches = await loadPatchRange(persistence, 'events', 'w1', null, SHA_B); + const patches = await loadPatchRange(persistence, 'events', 'w1', null, SHA_B, { patchJournal: createPatchJournal(persistence) }); expect(patches).toHaveLength(2); expect(/** @type {any} */ (patches)[0].sha).toBe(SHA_A); @@ -250,7 +265,7 @@ describe('SyncProtocol', () => { const persistence = createMockPersistence(commits, blobs); - await expect(loadPatchRange(persistence, 'events', 'w1', SHA_A, SHA_B)).rejects.toThrow( + await expect(loadPatchRange(persistence, 'events', 'w1', SHA_A, SHA_B, { patchJournal: createPatchJournal(persistence) })).rejects.toThrow( /Divergence detected/ ); }); @@ -267,7 +282,7 @@ describe('SyncProtocol', () => { const persistence = createMockPersistence(commits, blobs); - const patches = await loadPatchRange(persistence, 'events', 'w1', SHA_A, SHA_B); + const patches = await loadPatchRange(persistence, 'events', 'w1', SHA_A, SHA_B, { patchJournal: createPatchJournal(persistence) }); expect(patches).toHaveLength(1); expect(/** @type {any} */ (patches)[0].sha).toBe(SHA_B); @@ -325,7 +340,7 @@ describe('SyncProtocol', () => { const request = { type: 'sync-request', frontier: { w1: SHA_A } }; const localFrontier = new Map([['w1', SHA_B]]); - const response = await processSyncRequest(/** @type {any} */ (request), localFrontier, persistence, 'events'); + const response = await processSyncRequest(/** @type {any} */ (request), localFrontier, persistence, 'events', { patchJournal: createPatchJournal(persistence) }); expect(response.type).toBe('sync-response'); expect(response.patches).toHaveLength(1); @@ -353,7 +368,7 @@ describe('SyncProtocol', () => { ['w2', SHA_B], ]); - const response = await processSyncRequest(/** @type {any} */ (request), localFrontier, persistence, 'events'); + const response = await processSyncRequest(/** @type {any} */ (request), localFrontier, persistence, 'events', { patchJournal: createPatchJournal(persistence) }); // Response should include complete local frontier expect(response.frontier).toEqual({ @@ -371,7 +386,7 @@ describe('SyncProtocol', () => { const request = { type: 'sync-request', frontier: { w1: SHA_A } }; const localFrontier = new Map([['w1', SHA_A]]); - const response = await processSyncRequest(/** @type {any} */ (request), localFrontier, persistence, 'events'); + const response = await processSyncRequest(/** @type {any} */ (request), localFrontier, persistence, 'events', { patchJournal: createPatchJournal(persistence) }); expect(response.patches).toHaveLength(0); }); @@ -602,7 +617,7 @@ describe('SyncProtocol', () => { // B requests sync from A const requestB = createSyncRequest(frontierB); - const responseA = await processSyncRequest(requestB, frontierA, persistence, 'events'); + const responseA = await processSyncRequest(requestB, frontierA, persistence, 'events', { patchJournal: createPatchJournal(persistence) }); // B applies response from A const resultB = /** @type {any} */ (applySyncResponse(responseA, stateB, frontierB)); @@ -611,7 +626,7 @@ describe('SyncProtocol', () => { // A requests sync from B const requestA = createSyncRequest(frontierA); - const responseB = await processSyncRequest(requestA, frontierB, persistence, 'events'); + const responseB = await processSyncRequest(requestA, frontierB, persistence, 'events', { patchJournal: createPatchJournal(persistence) }); // A applies response from B const resultA = /** @type {any} */ (applySyncResponse(responseB, stateA, frontierA)); @@ -658,7 +673,8 @@ describe('SyncProtocol', () => { request1, new Map([['w1', SHA_A]]), persistence, - 'events' + 'events', + { patchJournal: createPatchJournal(persistence) }, ); const result1 = /** @type {any} */ (applySyncResponse(response1, state, frontier)); @@ -763,7 +779,7 @@ describe('SyncProtocol', () => { localFrontier, /** @type {any} */ (persistence), 'events', - { logger }, + { logger, patchJournal: createPatchJournal(persistence) }, )); // w1 should be skipped via isAncestor, no chain walk needed @@ -806,6 +822,7 @@ describe('SyncProtocol', () => { localFrontier, /** @type {any} */ (persistence), 'events', + { patchJournal: createPatchJournal(persistence) }, )); expect(persistence.isAncestor).toHaveBeenCalledWith(SHA_A, SHA_B); @@ -833,6 +850,7 @@ describe('SyncProtocol', () => { localFrontier, /** @type {any} */ (persistence), 'events', + { patchJournal: createPatchJournal(persistence) }, )); // Should still work via chain walk diff --git a/test/unit/domain/services/TreeConstruction.determinism.test.js b/test/unit/domain/services/TreeConstruction.determinism.test.js index 6a50849f..983c2a8a 100644 --- a/test/unit/domain/services/TreeConstruction.determinism.test.js +++ b/test/unit/domain/services/TreeConstruction.determinism.test.js @@ -12,6 +12,8 @@ import { CONTENT_PROPERTY_KEY, encodeEdgePropKey } from '../../../../src/domain/ import InMemoryGraphAdapter from '../../../../src/infrastructure/adapters/InMemoryGraphAdapter.js'; import InMemoryBlobStorageAdapter from '../../../../src/domain/utils/defaultBlobStorage.js'; import NodeCryptoAdapter from '../../../../src/infrastructure/adapters/NodeCryptoAdapter.js'; +import { CborPatchJournalAdapter } from '../../../../src/infrastructure/adapters/CborPatchJournalAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; const PROPERTY_TEST_SEED = 4242; const FIXED_CLOCK = { now: () => 42 }; @@ -44,6 +46,10 @@ async function createPatchTreeOid(contentIds, shuffleSeed) { const builder = new PatchBuilderV2(/** @type {any} */ ({ persistence, + patchJournal: new CborPatchJournalAdapter({ + codec: new CborCodec(), + blobPort: persistence, + }), graphName: 'g', writerId: 'alice', lamport: 1, diff --git a/test/unit/domain/services/state/StateHashService.test.js b/test/unit/domain/services/state/StateHashService.test.js new file mode 100644 index 00000000..941e1dda --- /dev/null +++ b/test/unit/domain/services/state/StateHashService.test.js @@ -0,0 +1,47 @@ +import { describe, it, expect, vi } from 'vitest'; +import StateHashService from '../../../../../src/domain/services/state/StateHashService.js'; +import { createEmptyStateV5 } from '../../../../../src/domain/services/JoinReducer.js'; +import { CborCodec } from '../../../../../src/infrastructure/codecs/CborCodec.js'; +import CryptoPort from '../../../../../src/ports/CryptoPort.js'; + +/** + * Creates a mock CryptoPort with a hash spy. + * @param {(algo: string, data: Uint8Array) => Promise} [hashImpl] + * @returns {CryptoPort} + */ +function createMockCrypto(hashImpl) { + const mock = new CryptoPort(); + mock.hash = vi.fn(hashImpl ?? (async () => 'deadbeef'.repeat(8))); + return mock; +} + +describe('StateHashService', () => { + it('computes a hex hash string', async () => { + const crypto = createMockCrypto(); + const svc = new StateHashService({ codec: new CborCodec(), crypto }); + + const hash = await svc.compute(createEmptyStateV5()); + + expect(typeof hash).toBe('string'); + expect(hash).toBe('deadbeef'.repeat(8)); + expect(crypto.hash).toHaveBeenCalledOnce(); + expect(crypto.hash).toHaveBeenCalledWith('sha256', expect.any(Uint8Array)); + }); + + it('produces deterministic output for the same state', async () => { + /** @type {Uint8Array[]} */ + const captured = []; + const crypto = createMockCrypto(async (_algo, data) => { + captured.push(data); + return 'abc'; + }); + const svc = new StateHashService({ codec: new CborCodec(), crypto }); + + await svc.compute(createEmptyStateV5()); + await svc.compute(createEmptyStateV5()); + + // Same state → same bytes → same hash + expect(captured).toHaveLength(2); + expect(Array.from(/** @type {Uint8Array} */ (captured[0]))).toEqual(Array.from(/** @type {Uint8Array} */ (captured[1]))); + }); +}); diff --git a/test/unit/domain/stream/LogicalBitmapIndexBuilder.stream.test.js b/test/unit/domain/stream/LogicalBitmapIndexBuilder.stream.test.js new file mode 100644 index 00000000..60af11df --- /dev/null +++ b/test/unit/domain/stream/LogicalBitmapIndexBuilder.stream.test.js @@ -0,0 +1,98 @@ +/** + * Tests that LogicalBitmapIndexBuilder.yieldShards() produces output + * equivalent to serialize() when piped through the encode pipeline. + */ +import { describe, it, expect } from 'vitest'; +import LogicalBitmapIndexBuilder from '../../../../src/domain/services/index/LogicalBitmapIndexBuilder.js'; +import WarpStream from '../../../../src/domain/stream/WarpStream.js'; +import { IndexShardEncodeTransform } from '../../../../src/infrastructure/adapters/IndexShardEncodeTransform.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; +import { MetaShard, EdgeShard, LabelShard, ReceiptShard, IndexShard } from '../../../../src/domain/artifacts/IndexShard.js'; + +/** + * Builds a small index with nodes and edges for testing. + * @returns {LogicalBitmapIndexBuilder} + */ +function buildTestIndex() { + const builder = new LogicalBitmapIndexBuilder(); + builder.registerNode('user:alice'); + builder.registerNode('user:bob'); + builder.registerNode('user:carol'); + builder.registerLabel('knows'); + builder.registerLabel('likes'); + builder.addEdge('user:alice', 'user:bob', 'knows'); + builder.addEdge('user:bob', 'user:carol', 'knows'); + builder.addEdge('user:alice', 'user:carol', 'likes'); + return builder; +} + +describe('LogicalBitmapIndexBuilder.yieldShards() — IndexShard records', () => { + it('yields IndexShard instances', () => { + const builder = buildTestIndex(); + const shards = [...builder.yieldShards()]; + for (const shard of shards) { + expect(shard).toBeInstanceOf(IndexShard); + } + }); + + it('produces MetaShard, LabelShard, EdgeShard, ReceiptShard', () => { + const builder = buildTestIndex(); + const shards = [...builder.yieldShards()]; + expect(shards.some((s) => s instanceof MetaShard)).toBe(true); + expect(shards.some((s) => s instanceof LabelShard)).toBe(true); + expect(shards.some((s) => s instanceof EdgeShard)).toBe(true); + expect(shards.some((s) => s instanceof ReceiptShard)).toBe(true); + }); + + it('produces byte-identical output via IndexShardEncodeTransform', async () => { + const codec = new CborCodec(); + const builder = buildTestIndex(); + + // Old path: serialize() produces Record + const serialized = builder.serialize(); + + // New path: yieldShards() → IndexShardEncodeTransform → collect + const streamed = await WarpStream.from(builder.yieldShards()) + .pipe(new IndexShardEncodeTransform(codec)) + .collect(); + + // Convert both to hex maps for comparison + /** @type {Record} */ + const serializedHex = {}; + for (const [path, bytes] of Object.entries(serialized)) { + serializedHex[path] = Array.from(bytes).map( + (/** @type {number} */ b) => b.toString(16).padStart(2, '0'), + ).join(''); + } + + /** @type {Record} */ + const streamedHex = {}; + for (const [path, bytes] of streamed) { + streamedHex[path] = Array.from(/** @type {Uint8Array} */ (bytes)).map( + (/** @type {number} */ b) => b.toString(16).padStart(2, '0'), + ).join(''); + } + + expect(streamedHex).toEqual(serializedHex); + }); + + it('ReceiptShard has correct counts', () => { + const builder = buildTestIndex(); + const shards = [...builder.yieldShards()]; + const receipt = shards.find((s) => s instanceof ReceiptShard); + expect(receipt).toBeInstanceOf(ReceiptShard); + const r = /** @type {ReceiptShard} */ (receipt); + expect(r.version).toBe(1); + expect(r.nodeCount).toBe(3); + expect(r.labelCount).toBe(2); + }); + + it('EdgeShards have correct directions', () => { + const builder = buildTestIndex(); + const shards = [...builder.yieldShards()]; + const edgeShards = shards.filter((s) => s instanceof EdgeShard); + const directions = edgeShards.map((s) => /** @type {EdgeShard} */ (s).direction); + expect(directions).toContain('fwd'); + expect(directions).toContain('rev'); + }); +}); diff --git a/test/unit/domain/stream/WarpStream.test.js b/test/unit/domain/stream/WarpStream.test.js new file mode 100644 index 00000000..b6e071c9 --- /dev/null +++ b/test/unit/domain/stream/WarpStream.test.js @@ -0,0 +1,408 @@ +import { describe, it, expect, vi } from 'vitest'; +import WarpStream from '../../../../src/domain/stream/WarpStream.js'; +import Transform from '../../../../src/domain/stream/Transform.js'; +import Sink from '../../../../src/domain/stream/Sink.js'; + +// ── Helpers ─────────────────────────────────────────────────────────── + +/** + * Creates an async generator that yields the given items. + * @param {unknown[]} items + */ +async function* asyncOf(...items) { + for (const item of items) { + yield item; + } +} + +/** + * A simple counting Sink that counts elements and returns the total. + * @extends {Sink} + */ +class CountSink extends Sink { + constructor() { + super(); + /** @type {number} */ + this._count = 0; + } + _accept() { this._count++; } + _finalize() { return this._count; } +} + +/** + * A collecting Sink that accumulates items into an array. + * @extends {Sink} + */ +class ArraySink extends Sink { + constructor() { + super(); + /** @type {unknown[]} */ + this._items = []; + } + /** @param {unknown} item */ + _accept(item) { this._items.push(item); } + _finalize() { return this._items; } +} + +// ── WarpStream Construction ─────────────────────────────────────────── + +describe('WarpStream', () => { + describe('construction', () => { + it('accepts an async iterable', () => { + const s = new WarpStream(asyncOf(1, 2, 3)); + expect(s).toBeInstanceOf(WarpStream); + }); + + it('rejects null source', () => { + expect(() => new WarpStream(/** @type {any} */ (null))).toThrow('requires an async iterable'); + }); + + it('rejects undefined source', () => { + expect(() => new WarpStream(/** @type {any} */ (undefined))).toThrow('requires an async iterable'); + }); + + it('rejects non-iterable source', () => { + expect(() => new WarpStream(/** @type {any} */ (42))).toThrow('must implement Symbol.asyncIterator'); + }); + }); + + describe('from()', () => { + it('wraps an async iterable', async () => { + const s = WarpStream.from(asyncOf(1, 2, 3)); + expect(await s.collect()).toEqual([1, 2, 3]); + }); + + it('wraps a sync iterable (array)', async () => { + const s = WarpStream.from([1, 2, 3]); + expect(await s.collect()).toEqual([1, 2, 3]); + }); + + it('returns the same WarpStream if already one', () => { + const s = WarpStream.from([1, 2]); + expect(WarpStream.from(s)).toBe(s); + }); + + it('rejects non-iterables', () => { + expect(() => WarpStream.from(/** @type {any} */ (42))).toThrow('requires an iterable'); + }); + }); + + describe('of()', () => { + it('creates a stream from explicit values', async () => { + const s = WarpStream.of('a', 'b', 'c'); + expect(await s.collect()).toEqual(['a', 'b', 'c']); + }); + + it('creates an empty stream with no args', async () => { + const s = WarpStream.of(); + expect(await s.collect()).toEqual([]); + }); + }); + + // ── Symbol.asyncIterator ────────────────────────────────────────── + + describe('Symbol.asyncIterator', () => { + it('works with for-await', async () => { + const results = []; + for await (const item of WarpStream.of(1, 2, 3)) { + results.push(item); + } + expect(results).toEqual([1, 2, 3]); + }); + }); + + // ── pipe() ──────────────────────────────────────────────────────── + + describe('pipe()', () => { + it('transforms each element', async () => { + const doubled = WarpStream.of(1, 2, 3) + .pipe(new Transform((x) => x * 2)); + expect(await doubled.collect()).toEqual([2, 4, 6]); + }); + + it('chains multiple transforms', async () => { + const result = await WarpStream.of(1, 2, 3) + .pipe(new Transform((x) => x * 2)) + .pipe(new Transform((x) => x + 1)) + .collect(); + expect(result).toEqual([3, 5, 7]); + }); + + it('supports async transform functions', async () => { + const result = await WarpStream.of(1, 2, 3) + .pipe(new Transform(async (x) => x * 10)) + .collect(); + expect(result).toEqual([10, 20, 30]); + }); + + it('rejects null transform', () => { + expect(() => WarpStream.of(1).pipe(/** @type {any} */ (null))).toThrow('requires a Transform'); + }); + }); + + // ── drain() ─────────────────────────────────────────────────────── + + describe('drain()', () => { + it('consumes stream and returns sink result', async () => { + const count = await WarpStream.of(1, 2, 3).drain(new CountSink()); + expect(count).toBe(3); + }); + + it('calls _accept for each element', async () => { + const items = await WarpStream.of('a', 'b').drain(new ArraySink()); + expect(items).toEqual(['a', 'b']); + }); + + it('rejects null sink', async () => { + await expect(WarpStream.of(1).drain(/** @type {any} */ (null))).rejects.toThrow('requires a Sink'); + }); + }); + + // ── reduce() ────────────────────────────────────────────────────── + + describe('reduce()', () => { + it('reduces to a single value', async () => { + const sum = await WarpStream.of(1, 2, 3).reduce((acc, x) => acc + x, 0); + expect(sum).toBe(6); + }); + + it('supports async reducer', async () => { + const sum = await WarpStream.of(1, 2, 3) + .reduce(async (acc, x) => acc + x, 0); + expect(sum).toBe(6); + }); + + it('returns init for empty stream', async () => { + const result = await WarpStream.of().reduce((acc) => acc, 42); + expect(result).toBe(42); + }); + }); + + // ── forEach() ───────────────────────────────────────────────────── + + describe('forEach()', () => { + it('calls function for each element', async () => { + /** @type {unknown[]} */ + const seen = []; + await WarpStream.of(1, 2, 3).forEach((x) => { seen.push(x); }); + expect(seen).toEqual([1, 2, 3]); + }); + }); + + // ── collect() ───────────────────────────────────────────────────── + + describe('collect()', () => { + it('materializes all elements', async () => { + expect(await WarpStream.of(1, 2, 3).collect()).toEqual([1, 2, 3]); + }); + + it('returns empty array for empty stream', async () => { + expect(await WarpStream.of().collect()).toEqual([]); + }); + }); + + // ── tee() ───────────────────────────────────────────────────────── + + describe('tee()', () => { + it('produces two branches with identical elements', async () => { + const [a, b] = WarpStream.of(1, 2, 3).tee(); + const [ra, rb] = await Promise.all([a.collect(), b.collect()]); + expect(ra).toEqual([1, 2, 3]); + expect(rb).toEqual([1, 2, 3]); + }); + + it('both branches are independent WarpStreams', () => { + const [a, b] = WarpStream.of(1).tee(); + expect(a).toBeInstanceOf(WarpStream); + expect(b).toBeInstanceOf(WarpStream); + expect(a).not.toBe(b); + }); + }); + + // ── mux() ───────────────────────────────────────────────────────── + + describe('mux()', () => { + it('merges multiple streams', async () => { + const merged = WarpStream.mux( + WarpStream.of(1, 3, 5), + WarpStream.of(2, 4, 6), + ); + const items = await merged.collect(); + // All items present (order may vary due to interleaving) + expect(items.sort()).toEqual([1, 2, 3, 4, 5, 6]); + }); + + it('returns empty for no streams', async () => { + const merged = WarpStream.mux(); + expect(await merged.collect()).toEqual([]); + }); + + it('returns the single stream for one input', () => { + const s = WarpStream.of(1); + expect(WarpStream.mux(s)).toBe(s); + }); + }); + + // ── demux() ─────────────────────────────────────────────────────── + + describe('demux()', () => { + it('routes elements to named branches', async () => { + const branches = WarpStream.of( + { type: 'a', value: 1 }, + { type: 'b', value: 2 }, + { type: 'a', value: 3 }, + ).demux((item) => item.type, ['a', 'b']); + + const branchA = /** @type {WarpStream} */ (branches.get('a')); + const branchB = /** @type {WarpStream} */ (branches.get('b')); + const [aItems, bItems] = await Promise.all([ + branchA.collect(), + branchB.collect(), + ]); + expect(aItems).toEqual([{ type: 'a', value: 1 }, { type: 'a', value: 3 }]); + expect(bItems).toEqual([{ type: 'b', value: 2 }]); + }); + + it('rejects empty keys array', () => { + expect(() => WarpStream.of(1).demux(() => 'a', [])).toThrow('requires a non-empty keys'); + }); + + it('propagates source errors to waiting branches', async () => { + const source = { + async *[Symbol.asyncIterator]() { + yield { type: 'a', value: 1 }; + throw new Error('demux-boom'); + }, + }; + + const branches = new WarpStream(source).demux((item) => item.type, ['a', 'b']); + const errBranchA = /** @type {WarpStream} */ (branches.get('a')); + const errBranchB = /** @type {WarpStream} */ (branches.get('b')); + + await expect( + Promise.all([ + errBranchA.collect(), + errBranchB.collect(), + ]), + ).rejects.toThrow('demux-boom'); + }); + }); + + // ── Error Propagation ───────────────────────────────────────────── + + describe('error propagation', () => { + it('propagates transform errors to the consumer', async () => { + const s = WarpStream.of(1, 2, 3).pipe( + new Transform((x) => { + if (x === 2) { throw new Error('boom'); } + return x; + }), + ); + await expect(s.collect()).rejects.toThrow('boom'); + }); + + it('calls upstream return() on downstream error (teardown)', async () => { + const returnCalled = vi.fn(); + const source = { + [Symbol.asyncIterator]() { + let i = 0; + return { + async next() { + if (i >= 3) { return { value: undefined, done: true }; } + return { value: i++, done: false }; + }, + async return() { + returnCalled(); + return { value: undefined, done: true }; + }, + }; + }, + }; + + const s = new WarpStream(source).pipe( + new Transform((x) => { + if (x === 1) { throw new Error('stop'); } + return x; + }), + ); + + await expect(s.collect()).rejects.toThrow('stop'); + expect(returnCalled).toHaveBeenCalled(); + }); + }); + + // ── AbortSignal Cancellation ────────────────────────────────────── + + describe('AbortSignal cancellation', () => { + it('aborts mid-stream when signal fires', async () => { + const controller = new AbortController(); + let count = 0; + + const s = new WarpStream(asyncOf(1, 2, 3, 4, 5), { signal: controller.signal }); + + await expect( + s.forEach(() => { + count++; + if (count === 2) { controller.abort(); } + }), + ).rejects.toThrow(); + + expect(count).toBe(2); + }); + }); +}); + +// ── Transform ─────────────────────────────────────────────────────── + +describe('Transform', () => { + it('requires a function or subclass override', () => { + expect(() => new Transform(/** @type {any} */ (42))).toThrow('requires a function'); + }); + + it('apply() throws if no function and not overridden', async () => { + const t = new Transform(); + const iterable = t.apply(asyncOf(1)); + const iter = iterable[Symbol.asyncIterator](); + await expect(iter.next()).rejects.toThrow('must be overridden'); + }); + + it('subclass can override apply()', async () => { + /** + * @extends {Transform} + */ + class DoubleTransform extends Transform { + /** @param {AsyncIterable} source */ + async *apply(source) { + for await (const item of source) { + yield item; + yield item; + } + } + } + + const result = await WarpStream.of(1, 2) + .pipe(new DoubleTransform()) + .collect(); + expect(result).toEqual([1, 1, 2, 2]); + }); +}); + +// ── Sink ──────────────────────────────────────────────────────────── + +describe('Sink', () => { + it('_accept throws if not overridden', () => { + const s = new Sink(); + expect(() => s._accept(1)).toThrow('not implemented'); + }); + + it('_finalize throws if not overridden', () => { + const s = new Sink(); + expect(() => s._finalize()).toThrow('not implemented'); + }); + + it('consume() calls _accept for each item and _finalize at end', async () => { + const sink = new ArraySink(); + const result = await sink.consume(asyncOf('x', 'y')); + expect(result).toEqual(['x', 'y']); + }); +}); diff --git a/test/unit/domain/types/WorldlineSelector.test.js b/test/unit/domain/types/WorldlineSelector.test.js new file mode 100644 index 00000000..6a9624f8 --- /dev/null +++ b/test/unit/domain/types/WorldlineSelector.test.js @@ -0,0 +1,345 @@ +import { describe, it, expect } from 'vitest'; +import WorldlineSelector from '../../../../src/domain/types/WorldlineSelector.js'; +import LiveSelector from '../../../../src/domain/types/LiveSelector.js'; +import CoordinateSelector from '../../../../src/domain/types/CoordinateSelector.js'; +import StrandSelector from '../../../../src/domain/types/StrandSelector.js'; + +// ─── LiveSelector ──────────────────────────────────────────────────────────── + +describe('LiveSelector', () => { + it('extends WorldlineSelector', () => { + const sel = new LiveSelector(); + expect(sel).toBeInstanceOf(WorldlineSelector); + expect(sel).toBeInstanceOf(LiveSelector); + }); + + it('defaults ceiling to null', () => { + const sel = new LiveSelector(); + expect(sel.ceiling).toBe(null); + }); + + it('accepts null ceiling', () => { + const sel = new LiveSelector(null); + expect(sel.ceiling).toBe(null); + }); + + it('accepts non-negative integer ceiling', () => { + const sel = new LiveSelector(42); + expect(sel.ceiling).toBe(42); + }); + + it('accepts zero ceiling', () => { + const sel = new LiveSelector(0); + expect(sel.ceiling).toBe(0); + }); + + it('rejects negative ceiling', () => { + expect(() => new LiveSelector(-1)).toThrow(TypeError); + }); + + it('rejects non-integer ceiling', () => { + expect(() => new LiveSelector(3.14)).toThrow(TypeError); + }); + + it('rejects string ceiling', () => { + expect(() => new LiveSelector(/** @type {any} */ ('42'))).toThrow(TypeError); + }); + + it('is frozen', () => { + const sel = new LiveSelector(10); + expect(Object.isFrozen(sel)).toBe(true); + }); + + it('clone returns an independent LiveSelector', () => { + const sel = new LiveSelector(42); + const copy = sel.clone(); + expect(copy).toBeInstanceOf(LiveSelector); + expect(copy).not.toBe(sel); + expect(copy.ceiling).toBe(42); + }); + + it('clone of no-ceiling returns no-ceiling', () => { + const sel = new LiveSelector(); + const copy = sel.clone(); + expect(copy.ceiling).toBe(null); + }); + + it('toDTO returns plain object with kind', () => { + const sel = new LiveSelector(42); + const dto = sel.toDTO(); + expect(dto).toEqual({ kind: 'live', ceiling: 42 }); + expect(dto.constructor).toBe(Object); + }); + + it('toDTO with null ceiling omits ceiling key', () => { + const sel = new LiveSelector(); + const dto = sel.toDTO(); + expect(dto).toEqual({ kind: 'live' }); + expect('ceiling' in dto).toBe(false); + }); +}); + +// ─── CoordinateSelector ───────────────────────────────────────────────────── + +describe('CoordinateSelector', () => { + it('extends WorldlineSelector', () => { + const sel = new CoordinateSelector(new Map([['alice', 'abc']])); + expect(sel).toBeInstanceOf(WorldlineSelector); + expect(sel).toBeInstanceOf(CoordinateSelector); + }); + + it('accepts Map frontier', () => { + const frontier = new Map([['alice', 'abc'], ['bob', 'def']]); + const sel = new CoordinateSelector(frontier); + expect(sel.frontier).toEqual(frontier); + }); + + it('accepts plain object frontier and normalizes to Map', () => { + const sel = new CoordinateSelector({ alice: 'abc', bob: 'def' }); + expect(sel.frontier).toBeInstanceOf(Map); + expect(sel.frontier.get('alice')).toBe('abc'); + expect(sel.frontier.get('bob')).toBe('def'); + }); + + it('accepts empty frontier', () => { + const sel = new CoordinateSelector(new Map()); + expect(sel.frontier.size).toBe(0); + }); + + it('accepts empty object frontier', () => { + const sel = new CoordinateSelector({}); + expect(sel.frontier.size).toBe(0); + }); + + it('defaults ceiling to null', () => { + const sel = new CoordinateSelector(new Map()); + expect(sel.ceiling).toBe(null); + }); + + it('accepts ceiling', () => { + const sel = new CoordinateSelector(new Map(), 42); + expect(sel.ceiling).toBe(42); + }); + + it('rejects null frontier', () => { + expect(() => new CoordinateSelector(/** @type {any} */ (null))).toThrow(TypeError); + }); + + it('rejects non-object frontier', () => { + expect(() => new CoordinateSelector(/** @type {any} */ ('bad'))).toThrow(TypeError); + }); + + it('rejects negative ceiling', () => { + expect(() => new CoordinateSelector(new Map(), -1)).toThrow(TypeError); + }); + + it('is frozen', () => { + const sel = new CoordinateSelector(new Map([['a', 'b']])); + expect(Object.isFrozen(sel)).toBe(true); + }); + + it('frontier getter returns a defensive copy', () => { + const sel = new CoordinateSelector(new Map([['alice', 'abc']])); + const f1 = sel.frontier; + const f2 = sel.frontier; + expect(f1).not.toBe(f2); + expect(f1).toEqual(f2); + }); + + it('mutating the returned frontier does not affect the selector', () => { + const sel = new CoordinateSelector(new Map([['alice', 'abc']])); + const f = sel.frontier; + f.set('evil', 'mutation'); + expect(sel.frontier.has('evil')).toBe(false); + expect(sel.frontier.size).toBe(1); + }); + + it('mutating the input Map does not affect the selector', () => { + const input = new Map([['alice', 'abc']]); + const sel = new CoordinateSelector(input); + input.set('evil', 'mutation'); + expect(sel.frontier.has('evil')).toBe(false); + }); + + it('clone returns an independent CoordinateSelector', () => { + const sel = new CoordinateSelector(new Map([['alice', 'abc']]), 42); + const copy = sel.clone(); + expect(copy).toBeInstanceOf(CoordinateSelector); + expect(copy).not.toBe(sel); + expect(copy.frontier).toEqual(sel.frontier); + expect(copy.frontier).not.toBe(sel.frontier); + expect(copy.ceiling).toBe(42); + }); + + it('toDTO returns plain object with kind and Map frontier', () => { + const sel = new CoordinateSelector(new Map([['alice', 'abc']]), 10); + const dto = sel.toDTO(); + expect(dto.kind).toBe('coordinate'); + expect(dto.frontier).toBeInstanceOf(Map); + expect(dto.frontier.get('alice')).toBe('abc'); + expect(dto.ceiling).toBe(10); + expect(dto.constructor).toBe(Object); + }); +}); + +// ─── StrandSelector ───────────────────────────────────────────────────────── + +describe('StrandSelector', () => { + it('extends WorldlineSelector', () => { + const sel = new StrandSelector('strand-abc'); + expect(sel).toBeInstanceOf(WorldlineSelector); + expect(sel).toBeInstanceOf(StrandSelector); + }); + + it('stores strandId', () => { + const sel = new StrandSelector('strand-abc'); + expect(sel.strandId).toBe('strand-abc'); + }); + + it('defaults ceiling to null', () => { + const sel = new StrandSelector('strand-abc'); + expect(sel.ceiling).toBe(null); + }); + + it('accepts ceiling', () => { + const sel = new StrandSelector('strand-abc', 42); + expect(sel.ceiling).toBe(42); + }); + + it('rejects empty strandId', () => { + expect(() => new StrandSelector('')).toThrow(TypeError); + }); + + it('rejects non-string strandId', () => { + expect(() => new StrandSelector(/** @type {any} */ (123))).toThrow(TypeError); + }); + + it('rejects null strandId', () => { + expect(() => new StrandSelector(/** @type {any} */ (null))).toThrow(TypeError); + }); + + it('rejects negative ceiling', () => { + expect(() => new StrandSelector('strand-abc', -1)).toThrow(TypeError); + }); + + it('is frozen', () => { + const sel = new StrandSelector('strand-abc'); + expect(Object.isFrozen(sel)).toBe(true); + }); + + it('clone returns an independent StrandSelector', () => { + const sel = new StrandSelector('strand-abc', 42); + const copy = sel.clone(); + expect(copy).toBeInstanceOf(StrandSelector); + expect(copy).not.toBe(sel); + expect(copy.strandId).toBe('strand-abc'); + expect(copy.ceiling).toBe(42); + }); + + it('toDTO returns plain object with kind', () => { + const sel = new StrandSelector('strand-abc', 10); + const dto = sel.toDTO(); + expect(dto).toEqual({ kind: 'strand', strandId: 'strand-abc', ceiling: 10 }); + expect(dto.constructor).toBe(Object); + }); +}); + +// ─── WorldlineSelector.from() ─────────────────────────────────────────────── + +describe('WorldlineSelector.from()', () => { + it('returns existing selector instance as-is', () => { + const sel = new LiveSelector(42); + expect(WorldlineSelector.from(sel)).toBe(sel); + }); + + it('converts { kind: "live" } to LiveSelector', () => { + const sel = WorldlineSelector.from({ kind: 'live' }); + expect(sel).toBeInstanceOf(LiveSelector); + const live = /** @type {LiveSelector} */ (sel); + expect(live.ceiling).toBe(null); + }); + + it('converts { kind: "live", ceiling: 42 } to LiveSelector', () => { + const sel = WorldlineSelector.from({ kind: 'live', ceiling: 42 }); + expect(sel).toBeInstanceOf(LiveSelector); + const live = /** @type {LiveSelector} */ (sel); + expect(live.ceiling).toBe(42); + }); + + it('converts { kind: "coordinate" } to CoordinateSelector', () => { + const sel = WorldlineSelector.from({ + kind: 'coordinate', + frontier: new Map([['alice', 'abc']]), + ceiling: null, + }); + expect(sel).toBeInstanceOf(CoordinateSelector); + const coord = /** @type {CoordinateSelector} */ (sel); + expect(coord.frontier.get('alice')).toBe('abc'); + }); + + it('converts { kind: "coordinate" } with plain object frontier', () => { + const sel = WorldlineSelector.from({ + kind: 'coordinate', + frontier: { alice: 'abc' }, + }); + expect(sel).toBeInstanceOf(CoordinateSelector); + const coord = /** @type {CoordinateSelector} */ (sel); + expect(coord.frontier).toBeInstanceOf(Map); + }); + + it('converts { kind: "strand" } to StrandSelector', () => { + const sel = WorldlineSelector.from({ + kind: 'strand', + strandId: 'strand-abc', + ceiling: 10, + }); + expect(sel).toBeInstanceOf(StrandSelector); + const strand = /** @type {StrandSelector} */ (sel); + expect(strand.strandId).toBe('strand-abc'); + expect(strand.ceiling).toBe(10); + }); + + it('converts null to LiveSelector', () => { + const sel = WorldlineSelector.from(null); + expect(sel).toBeInstanceOf(LiveSelector); + const live = /** @type {LiveSelector} */ (sel); + expect(live.ceiling).toBe(null); + }); + + it('converts undefined to LiveSelector', () => { + const sel = WorldlineSelector.from(undefined); + expect(sel).toBeInstanceOf(LiveSelector); + }); + + it('throws on coordinate without frontier', () => { + expect(() => WorldlineSelector.from({ kind: 'coordinate' })).toThrow(TypeError); + }); + + it('returns frozen selector as-is without mutation', () => { + const sel = new LiveSelector(42); + expect(Object.isFrozen(sel)).toBe(true); + const result = WorldlineSelector.from(sel); + expect(result).toBe(sel); + const live = /** @type {LiveSelector} */ (result); + expect(live.ceiling).toBe(42); + }); + + it('throws on unknown kind', () => { + expect(() => WorldlineSelector.from({ kind: 'bogus' })).toThrow(TypeError); + }); +}); + +// ─── WorldlineSelector base class ─────────────────────────────────────────── + +describe('WorldlineSelector (base)', () => { + it('clone() throws on base class', () => { + // Cannot construct directly in normal use, but verify the guard + const base = Object.create(WorldlineSelector.prototype); + expect(() => base.clone()).toThrow(); + }); + + it('toDTO() throws on base class', () => { + const base = Object.create(WorldlineSelector.prototype); + expect(() => base.toDTO()).toThrow(); + }); +}); diff --git a/test/unit/domain/utils/canonicalCbor.test.js b/test/unit/domain/utils/canonicalCbor.test.js deleted file mode 100644 index 0617ceb5..00000000 --- a/test/unit/domain/utils/canonicalCbor.test.js +++ /dev/null @@ -1,57 +0,0 @@ -import { describe, it, expect } from 'vitest'; -import { encodeCanonicalCbor, decodeCanonicalCbor } from '../../../../src/domain/utils/canonicalCbor.js'; - -describe('canonicalCbor', () => { - it('round-trips plain objects', () => { - const obj = { name: 'alice', age: 30 }; - const bytes = encodeCanonicalCbor(obj); - expect(decodeCanonicalCbor(bytes)).toEqual(obj); - }); - - it('round-trips arrays', () => { - const arr = [1, 'two', { three: 3 }]; - const bytes = encodeCanonicalCbor(arr); - expect(decodeCanonicalCbor(bytes)).toEqual(arr); - }); - - it('round-trips nested objects', () => { - const nested = { a: { b: { c: [1, 2, 3] } } }; - const bytes = encodeCanonicalCbor(nested); - expect(decodeCanonicalCbor(bytes)).toEqual(nested); - }); - - it('{z:1, a:2} and {a:2, z:1} produce identical bytes', () => { - const a = encodeCanonicalCbor({ z: 1, a: 2 }); - const b = encodeCanonicalCbor({ a: 2, z: 1 }); - expect(Buffer.from(a).equals(Buffer.from(b))).toBe(true); - }); - - it('Map produces same bytes as equivalent sorted object', () => { - const fromMap = encodeCanonicalCbor(new Map([['z', 1], ['a', 2]])); - const fromObj = encodeCanonicalCbor({ a: 2, z: 1 }); - expect(Buffer.from(fromMap).equals(Buffer.from(fromObj))).toBe(true); - }); - - it('null-prototype objects round-trip', () => { - const npo = Object.create(null); - npo.x = 1; - npo.a = 2; - const bytes = encodeCanonicalCbor(npo); - const decoded = decodeCanonicalCbor(bytes); - expect(decoded).toEqual({ a: 2, x: 1 }); - }); - - it('Uint8Array survives round-trip as binary', () => { - const data = new Uint8Array([0xde, 0xad, 0xbe, 0xef]); - const bytes = encodeCanonicalCbor(data); - const decoded = decodeCanonicalCbor(bytes); - expect(decoded).toBeInstanceOf(Uint8Array); - expect([.../** @type {Uint8Array} */ (decoded)]).toEqual([0xde, 0xad, 0xbe, 0xef]); - }); - - it('round-trips null and primitives', () => { - for (const val of [null, true, false, 42, -1, 0, 'hello']) { - expect(decodeCanonicalCbor(encodeCanonicalCbor(val))).toEqual(val); - } - }); -}); diff --git a/test/unit/domain/warp/Writer.test.js b/test/unit/domain/warp/Writer.test.js index fc4ad63a..5d77181e 100644 --- a/test/unit/domain/warp/Writer.test.js +++ b/test/unit/domain/warp/Writer.test.js @@ -11,6 +11,8 @@ import { PatchSession } from '../../../../src/domain/warp/PatchSession.js'; import { buildWriterRef, validateWriterId } from '../../../../src/domain/utils/RefLayout.js'; import { createVersionVector } from '../../../../src/domain/crdt/VersionVector.js'; import { encodePatchMessage } from '../../../../src/domain/services/codec/WarpMessageCodec.js'; +import { CborPatchJournalAdapter } from '../../../../src/infrastructure/adapters/CborPatchJournalAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; /** * Creates a minimal mock persistence adapter. @@ -28,6 +30,18 @@ function createMockPersistence() { }; } +/** + * Creates a CborPatchJournalAdapter wired to the given persistence's blob ops. + * @param {ReturnType} persistence + * @returns {CborPatchJournalAdapter} + */ +function createPatchJournal(persistence) { + return new CborPatchJournalAdapter({ + codec: new CborCodec(), + blobPort: persistence, + }); +} + /** * Creates a mock patch commit message. */ @@ -62,18 +76,29 @@ describe('Writer (WARP schema:2)', () => { }); it('throws on invalid writerId', () => { - expect(() => new Writer({ + expect(() => new Writer(/** @type {any} */ ({ persistence, graphName: 'events', writerId: 'a/b', versionVector, getCurrentState, - })).toThrow('Invalid writer ID'); + }))).toThrow('Invalid writer ID'); + }); + + it('throws when patchJournal is missing', () => { + expect(() => new Writer(/** @type {any} */ ({ + persistence, + graphName: 'events', + writerId: 'alice', + versionVector, + getCurrentState, + }))).toThrow('patchJournal is required'); }); it('accepts valid writerId', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -97,6 +122,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -114,6 +140,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -133,6 +160,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -150,6 +178,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -165,6 +194,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -182,6 +212,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -205,6 +236,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -228,6 +260,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -252,6 +285,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -283,6 +317,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -311,6 +346,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -336,6 +372,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -359,6 +396,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -388,6 +426,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -429,6 +468,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -459,6 +499,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -489,6 +530,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -512,6 +554,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -537,6 +580,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -561,6 +605,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -586,6 +631,7 @@ describe('Writer (WARP schema:2)', () => { const writer = new Writer({ persistence, + patchJournal: createPatchJournal(persistence), graphName: 'events', writerId: 'alice', versionVector, @@ -632,17 +678,21 @@ describe('PatchSession operations', () => { let versionVector; /** @type {any} */ let getCurrentState; + /** @type {CborPatchJournalAdapter} */ + let patchJournal; beforeEach(() => { persistence = createMockPersistence(); versionVector = createVersionVector(); getCurrentState = vi.fn(() => null); persistence.readRef.mockResolvedValue(null); + patchJournal = createPatchJournal(persistence); }); it('addNode creates node-add op', async () => { const writer = new Writer({ persistence, + patchJournal, graphName: 'events', writerId: 'alice', versionVector, @@ -661,6 +711,7 @@ describe('PatchSession operations', () => { it('removeNode creates node-remove op', async () => { const writer = new Writer({ persistence, + patchJournal, graphName: 'events', writerId: 'alice', versionVector, @@ -679,6 +730,7 @@ describe('PatchSession operations', () => { it('addEdge creates edge-add op', async () => { const writer = new Writer({ persistence, + patchJournal, graphName: 'events', writerId: 'alice', versionVector, @@ -699,6 +751,7 @@ describe('PatchSession operations', () => { it('removeEdge creates edge-remove op', async () => { const writer = new Writer({ persistence, + patchJournal, graphName: 'events', writerId: 'alice', versionVector, @@ -716,6 +769,7 @@ describe('PatchSession operations', () => { it('setProperty creates prop-set op', async () => { const writer = new Writer({ persistence, + patchJournal, graphName: 'events', writerId: 'alice', versionVector, @@ -736,6 +790,7 @@ describe('PatchSession operations', () => { it('supports various property value types', async () => { const writer = new Writer({ persistence, + patchJournal, graphName: 'events', writerId: 'alice', versionVector, diff --git a/test/unit/infrastructure/adapters/CborCheckpointStoreAdapter.test.js b/test/unit/infrastructure/adapters/CborCheckpointStoreAdapter.test.js new file mode 100644 index 00000000..566b8e29 --- /dev/null +++ b/test/unit/infrastructure/adapters/CborCheckpointStoreAdapter.test.js @@ -0,0 +1,138 @@ +import { describe, it, expect } from 'vitest'; +import { CborCheckpointStoreAdapter } from '../../../../src/infrastructure/adapters/CborCheckpointStoreAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; +import CheckpointStorePort from '../../../../src/ports/CheckpointStorePort.js'; +import { createORSet, orsetAdd } from '../../../../src/domain/crdt/ORSet.js'; +import { createVersionVector } from '../../../../src/domain/crdt/VersionVector.js'; +import { createDot } from '../../../../src/domain/crdt/Dot.js'; +import WarpStateV5 from '../../../../src/domain/services/state/WarpStateV5.js'; +import MockBlobPort from '../../../helpers/MockBlobPort.js'; + +/** + * Builds a small but representative checkpoint state. + * @returns {WarpStateV5} + */ +function createGoldenState() { + const nodeAlive = createORSet(); + orsetAdd(nodeAlive, 'user:alice', createDot('w1', 1)); + orsetAdd(nodeAlive, 'user:bob', createDot('w1', 2)); + + const edgeAlive = createORSet(); + orsetAdd(edgeAlive, 'user:alice\x00user:bob\x00knows', createDot('w1', 3)); + + /** @type {Map>} */ + const prop = new Map(); + prop.set('user:alice\x00name', { + eventId: { lamport: 1, writerId: 'w1', patchSha: 'a'.repeat(40), opIndex: 0 }, + value: 'Alice', + }); + + const observedFrontier = createVersionVector(); + observedFrontier.set('w1', 3); + + return new WarpStateV5({ nodeAlive, edgeAlive, prop, observedFrontier }); +} + +/** + * Creates an in-memory BlobPort backed by MockBlobPort. + * @returns {MockBlobPort} + */ +function createMemoryBlobPort() { + return new MockBlobPort(); +} + +describe('CborCheckpointStoreAdapter (collapsed)', () => { + it('extends CheckpointStorePort', () => { + const adapter = new CborCheckpointStoreAdapter({ + codec: new CborCodec(), blobPort: createMemoryBlobPort(), + }); + expect(adapter).toBeInstanceOf(CheckpointStorePort); + }); + + describe('writeCheckpoint', () => { + it('returns OIDs for state, frontier, appliedVV', async () => { + const blobPort = createMemoryBlobPort(); + const adapter = new CborCheckpointStoreAdapter({ + codec: new CborCodec(), blobPort, + }); + + const vv = createVersionVector(); + vv.set('w1', 3); + + const result = await adapter.writeCheckpoint({ + state: createGoldenState(), + frontier: new Map([['w1', 'abc123']]), + appliedVV: vv, + stateHash: 'deadbeef', + }); + + expect(typeof result.stateBlobOid).toBe('string'); + expect(typeof result.frontierBlobOid).toBe('string'); + expect(typeof result.appliedVVBlobOid).toBe('string'); + expect(result.provenanceIndexBlobOid).toBeNull(); + expect(blobPort.writeBlob).toHaveBeenCalledTimes(3); + }); + + it('writes 4 blobs when provenanceIndex is provided', async () => { + const blobPort = createMemoryBlobPort(); + const adapter = new CborCheckpointStoreAdapter({ + codec: new CborCodec(), blobPort, + }); + + const { ProvenanceIndex } = await import('../../../../src/domain/services/provenance/ProvenanceIndex.js'); + const provIndex = new ProvenanceIndex(); + + const vv = createVersionVector(); + + const result = await adapter.writeCheckpoint({ + state: createGoldenState(), + frontier: new Map(), + appliedVV: vv, + stateHash: 'deadbeef', + provenanceIndex: provIndex, + }); + + expect(result.provenanceIndexBlobOid).not.toBeNull(); + expect(blobPort.writeBlob).toHaveBeenCalledTimes(4); + }); + }); + + describe('readCheckpoint', () => { + it('round-trips state, frontier, appliedVV', async () => { + const blobPort = createMemoryBlobPort(); + const codec = new CborCodec(); + const adapter = new CborCheckpointStoreAdapter({ codec, blobPort }); + + const vv = createVersionVector(); + vv.set('w1', 3); + + const writeResult = await adapter.writeCheckpoint({ + state: createGoldenState(), + frontier: new Map([['w1', 'abc123']]), + appliedVV: vv, + stateHash: 'deadbeef', + }); + + const treeOids = { + 'state.cbor': writeResult.stateBlobOid, + 'frontier.cbor': writeResult.frontierBlobOid, + 'appliedVV.cbor': writeResult.appliedVVBlobOid, + }; + + const data = await adapter.readCheckpoint(treeOids); + + expect(data.state).toBeDefined(); + expect(data.state.nodeAlive).toBeDefined(); + expect(data.frontier.get('w1')).toBe('abc123'); + expect(data.appliedVV).not.toBeNull(); + expect(/** @type {NonNullable} */ (data.appliedVV).get('w1')).toBe(3); + }); + + it('throws on missing state.cbor', async () => { + const adapter = new CborCheckpointStoreAdapter({ + codec: new CborCodec(), blobPort: createMemoryBlobPort(), + }); + await expect(adapter.readCheckpoint({})).rejects.toThrow('missing state.cbor'); + }); + }); +}); diff --git a/test/unit/infrastructure/adapters/CborPatchJournalAdapter.test.js b/test/unit/infrastructure/adapters/CborPatchJournalAdapter.test.js new file mode 100644 index 00000000..499db27b --- /dev/null +++ b/test/unit/infrastructure/adapters/CborPatchJournalAdapter.test.js @@ -0,0 +1,149 @@ +import { describe, it, expect, vi } from 'vitest'; +import { CborPatchJournalAdapter } from '../../../../src/infrastructure/adapters/CborPatchJournalAdapter.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; +import { createPatchV2 } from '../../../../src/domain/types/WarpTypesV2.js'; +import PatchJournalPort from '../../../../src/ports/PatchJournalPort.js'; + +/** + * Golden fixture: a known PatchV2 encoded with the canonical CBOR codec. + * If this test breaks, the wire format changed — investigate before fixing. + * + * Note: ops use tuple form `['alice', 1]` for dot — this is the wire format + * that CBOR (de)serializes. The domain typedef uses Dot class, but the codec + * boundary handles the tuple ↔ Dot mapping. + */ +const GOLDEN_PATCH = createPatchV2({ + schema: 2, + writer: 'alice', + lamport: 1, + context: { alice: 0 }, + ops: /** @type {import('../../../../src/domain/types/WarpTypesV2.js').OpV2[]} */ ([ + { type: 'NodeAdd', id: 'user:alice', dot: ['alice', 1] }, + { type: 'PropSet', node: 'user:alice', key: 'name', value: 'Alice' }, + ]), + reads: [], + writes: ['user:alice'], +}); + +const GOLDEN_HEX = + 'b9000767636f6e74657874b9000165616c69636500676c616d706f727401636f707382b9000363646f748265616c696365016269646a757365723a616c6963656474797065674e6f6465416464b90004636b6579646e616d65646e6f64656a757365723a616c69636564747970656750726f705365746576616c756565416c696365657265616473f766736368656d61026677726974657265616c69636566777269746573816a757365723a616c696365'; + +import MockBlobPort from '../../../helpers/MockBlobPort.js'; +import BlobStoragePort from '../../../../src/ports/BlobStoragePort.js'; + +/** + * Creates an in-memory BlobPort backed by MockBlobPort. + * @returns {MockBlobPort} + */ +function createMemoryBlobPort() { + return new MockBlobPort(); +} + +describe('CborPatchJournalAdapter', () => { + it('extends PatchJournalPort', () => { + const codec = new CborCodec(); + const blobPort = createMemoryBlobPort(); + const adapter = new CborPatchJournalAdapter({ codec, blobPort }); + expect(adapter).toBeInstanceOf(PatchJournalPort); + }); + + it('writePatch returns a string OID', async () => { + const codec = new CborCodec(); + const blobPort = createMemoryBlobPort(); + const adapter = new CborPatchJournalAdapter({ codec, blobPort }); + + const oid = await adapter.writePatch(GOLDEN_PATCH); + expect(typeof oid).toBe('string'); + expect(oid.length).toBeGreaterThan(0); + }); + + it('readPatch returns the same PatchV2 object', async () => { + const codec = new CborCodec(); + const blobPort = createMemoryBlobPort(); + const adapter = new CborPatchJournalAdapter({ codec, blobPort }); + + const oid = await adapter.writePatch(GOLDEN_PATCH); + const result = await adapter.readPatch(oid); + + expect(result.schema).toBe(2); + expect(result.writer).toBe('alice'); + expect(result.lamport).toBe(1); + expect(result.ops).toHaveLength(2); + expect(/** @type {NonNullable<(typeof result.ops)[0]>} */ (result.ops[0]).type).toBe('NodeAdd'); + expect(/** @type {NonNullable<(typeof result.ops)[0]>} */ (result.ops[1]).type).toBe('PropSet'); + expect(result.writes).toEqual(['user:alice']); + }); + + describe('golden fixture (wire format stability)', () => { + it('produces byte-identical output to the known golden hex', async () => { + const codec = new CborCodec(); + const blobPort = createMemoryBlobPort(); + const adapter = new CborPatchJournalAdapter({ codec, blobPort }); + + await adapter.writePatch(GOLDEN_PATCH); + const storedBytes = /** @type {Uint8Array} */ (blobPort.store.values().next().value); + const storedHex = Array.from(storedBytes).map( + (/** @type {number} */ b) => b.toString(16).padStart(2, '0'), + ).join(''); + + expect(storedHex).toBe(GOLDEN_HEX); + }); + + it('round-trips the golden bytes back to the same domain object', async () => { + const codec = new CborCodec(); + const hexPairs = /** @type {string[]} */ (GOLDEN_HEX.match(/.{2}/g)); + const goldenBytes = new Uint8Array( + hexPairs.map((h) => parseInt(h, 16)), + ); + const blobPort = createMemoryBlobPort(); + blobPort.store.set('golden', goldenBytes); + const adapter = new CborPatchJournalAdapter({ codec, blobPort }); + + const result = await adapter.readPatch('golden'); + expect(result.schema).toBe(2); + expect(result.writer).toBe('alice'); + expect(result.ops).toHaveLength(2); + }); + }); + + describe('encrypted patch support', () => { + /** + * Creates a mock BlobStoragePort with vitest spies. + * @param {{ storeResult?: string, retrieveResult?: Uint8Array }} [opts] + * @returns {BlobStoragePort} + */ + function createMockBlobStorage(opts = {}) { + const mock = new BlobStoragePort(); + mock.store = vi.fn().mockResolvedValue(opts.storeResult ?? 'encrypted_oid'); + mock.retrieve = vi.fn().mockResolvedValue(opts.retrieveResult ?? new Uint8Array(0)); + mock.storeStream = vi.fn(); + mock.retrieveStream = vi.fn(); + return mock; + } + + it('uses patchBlobStorage when provided for writePatch', async () => { + const codec = new CborCodec(); + const blobPort = createMemoryBlobPort(); + const patchBlobStorage = createMockBlobStorage({ storeResult: 'encrypted_oid' }); + const adapter = new CborPatchJournalAdapter({ codec, blobPort, patchBlobStorage }); + + const oid = await adapter.writePatch(GOLDEN_PATCH); + expect(oid).toBe('encrypted_oid'); + expect(patchBlobStorage.store).toHaveBeenCalledOnce(); + expect(blobPort.writeBlob).not.toHaveBeenCalled(); + }); + + it('uses patchBlobStorage for readPatch when encrypted flag is set', async () => { + const codec = new CborCodec(); + const blobPort = createMemoryBlobPort(); + const goldenBytes = codec.encode(GOLDEN_PATCH); + const patchBlobStorage = createMockBlobStorage({ retrieveResult: goldenBytes }); + const adapter = new CborPatchJournalAdapter({ codec, blobPort, patchBlobStorage }); + + const result = await adapter.readPatch('some_oid', { encrypted: true }); + expect(result.writer).toBe('alice'); + expect(patchBlobStorage.retrieve).toHaveBeenCalledWith('some_oid'); + expect(blobPort.readBlob).not.toHaveBeenCalled(); + }); + }); +}); diff --git a/test/unit/infrastructure/adapters/StreamPipeline.test.js b/test/unit/infrastructure/adapters/StreamPipeline.test.js new file mode 100644 index 00000000..9a24b80b --- /dev/null +++ b/test/unit/infrastructure/adapters/StreamPipeline.test.js @@ -0,0 +1,116 @@ +import { describe, it, expect, vi } from 'vitest'; +import WarpStream from '../../../../src/domain/stream/WarpStream.js'; +import Transform from '../../../../src/domain/stream/Transform.js'; +import { CborEncodeTransform } from '../../../../src/infrastructure/adapters/CborEncodeTransform.js'; +import { CborDecodeTransform } from '../../../../src/infrastructure/adapters/CborDecodeTransform.js'; +import { GitBlobWriteTransform } from '../../../../src/infrastructure/adapters/GitBlobWriteTransform.js'; +import { TreeAssemblerSink } from '../../../../src/infrastructure/adapters/TreeAssemblerSink.js'; +import { CborCodec } from '../../../../src/infrastructure/codecs/CborCodec.js'; + +/** + * Creates an in-memory BlobPort + TreePort stub. + */ +function createMemoryGit() { + /** @type {Map} */ + const blobs = new Map(); + let blobCounter = 0; + /** @type {string[]} */ + let lastTree = []; + + return { + blobs, + writeBlob: vi.fn(async (/** @type {Uint8Array} */ content) => { + const oid = `blob_${String(blobCounter++).padStart(40, '0')}`; + blobs.set(oid, content); + return oid; + }), + readBlob: vi.fn(async (/** @type {string} */ oid) => { + const data = blobs.get(oid); + if (!data) { throw new Error(`Blob not found: ${oid}`); } + return data; + }), + writeTree: vi.fn(async (/** @type {string[]} */ entries) => { + lastTree = entries; + return 'tree_' + '0'.repeat(36); + }), + get lastTreeEntries() { return lastTree; }, + }; +} + +describe('Stream Pipeline Integration', () => { + it('domain objects → encode → blob write → tree assembly', async () => { + const codec = new CborCodec(); + const git = createMemoryGit(); + + // Domain objects: index shards + const shards = [ + ['meta_ab.cbor', { nodeToGlobal: [['user:alice', 0]], nextLocalId: 1 }], + ['labels.cbor', [['knows', 0], ['likes', 1]]], + ['receipt.cbor', { version: 1, nodeCount: 1, labelCount: 2 }], + ]; + + const treeOid = await WarpStream.from(shards) + .pipe(/** @type {any} */ (new CborEncodeTransform(codec))) + .pipe(/** @type {any} */ (new GitBlobWriteTransform(/** @type {any} */ (git)))) + .drain(/** @type {any} */ (new TreeAssemblerSink(/** @type {any} */ (git)))); + + // Tree was assembled + expect(treeOid).toBe('tree_' + '0'.repeat(36)); + expect(git.writeTree).toHaveBeenCalledOnce(); + + // 3 blobs were written + expect(git.writeBlob).toHaveBeenCalledTimes(3); + expect(git.blobs.size).toBe(3); + + // Tree entries are sorted and contain expected paths + const entries = git.lastTreeEntries; + expect(entries).toHaveLength(3); + const paths = entries.map((e) => e.split('\t')[1]); + expect(paths).toContain('labels.cbor'); + expect(paths).toContain('meta_ab.cbor'); + expect(paths).toContain('receipt.cbor'); + + // Round-trip: decode a shard and verify contents + const metaEntry = /** @type {string} */ (/** @type {any} */ (entries.find((e) => e.includes('meta_ab.cbor')))); + const metaParts = /** @type {string} */ (/** @type {any} */ (metaEntry.split('\t')[0])).split(' '); + const metaOid = /** @type {string} */ (/** @type {any} */ (metaParts[metaParts.length - 1])); + const metaBytes = /** @type {Uint8Array} */ (/** @type {any} */ (git.blobs.get(metaOid))); + expect(metaBytes).toBeDefined(); + const decoded = codec.decode(metaBytes); + expect(decoded).toEqual({ nodeToGlobal: [['user:alice', 0]], nextLocalId: 1 }); + }); + + it('blob read → decode: reverse pipeline', async () => { + const codec = new CborCodec(); + const git = createMemoryGit(); + + // Pre-populate blobs + const data1 = { name: 'Alice' }; + const data2 = { name: 'Bob' }; + git.blobs.set('oid1', codec.encode(data1)); + git.blobs.set('oid2', codec.encode(data2)); + + // Read + decode pipeline + const entries = [['user1.cbor', 'oid1'], ['user2.cbor', 'oid2']]; + + /** @type {Array<[string, unknown]>} */ + const results = []; + const readTransform = new Transform(); + readTransform.apply = async function *(/** @type {AsyncIterable<[string, string]>} */ source) { + for await (const [path, oid] of source) { + const bytes = await git.readBlob(oid); + yield /** @type {[string, Uint8Array]} */ ([path, bytes]); + } + }; + + await WarpStream.from(entries) + .pipe(/** @type {any} */ (readTransform)) + .pipe(/** @type {any} */ (new CborDecodeTransform(codec))) + .forEach(([path, obj]) => { results.push([path, obj]); }); + + expect(results).toEqual([ + ['user1.cbor', { name: 'Alice' }], + ['user2.cbor', { name: 'Bob' }], + ]); + }); +}); diff --git a/test/unit/ports/CheckpointStorePort.test.js b/test/unit/ports/CheckpointStorePort.test.js new file mode 100644 index 00000000..012081a4 --- /dev/null +++ b/test/unit/ports/CheckpointStorePort.test.js @@ -0,0 +1,14 @@ +import { describe, it, expect } from 'vitest'; +import CheckpointStorePort from '../../../src/ports/CheckpointStorePort.js'; + +describe('CheckpointStorePort', () => { + it('throws on direct call to writeCheckpoint()', async () => { + const port = new CheckpointStorePort(); + await expect(port.writeCheckpoint(/** @type {any} */ ({}))).rejects.toThrow('not implemented'); + }); + + it('throws on direct call to readCheckpoint()', async () => { + const port = new CheckpointStorePort(); + await expect(port.readCheckpoint(/** @type {any} */ ({}))).rejects.toThrow('not implemented'); + }); +}); diff --git a/test/unit/ports/PatchJournalPort.test.js b/test/unit/ports/PatchJournalPort.test.js new file mode 100644 index 00000000..a7d982d6 --- /dev/null +++ b/test/unit/ports/PatchJournalPort.test.js @@ -0,0 +1,14 @@ +import { describe, it, expect } from 'vitest'; +import PatchJournalPort from '../../../src/ports/PatchJournalPort.js'; + +describe('PatchJournalPort', () => { + it('throws on direct call to writePatch()', async () => { + const port = new PatchJournalPort(); + await expect(port.writePatch(/** @type {any} */ ({}))).rejects.toThrow('not implemented'); + }); + + it('throws on direct call to readPatch()', async () => { + const port = new PatchJournalPort(); + await expect(port.readPatch('abc123')).rejects.toThrow('not implemented'); + }); +});