Conversation
…ut cohorts with protocol vector replay
…ling Add the rust implementation to the shared coordinator protocol the way Go is integrated: an identity native binding, a strict JSONL report gate (check-rust-replay.mjs) and report adapter, default source bindings for the crate, and check-rust / formal-rust validation lanes with a cargo 1.98.1 prerequisite probe. The smoke target replays the committed histories through the Rust harness as well. Declare the rust implementation in profiles.json and refresh the two pins of its digest (generated-fixtures.lock.json inputs, go-parity.json inputs). Wire the pinned Rust toolchain into setup-validation, add the rust CI job and the rust-parity job of the full formal workflow, and pin the crate toolchain.
…vocation resolution Port go/policy.go into rust/src/policy.rs: Policy::from_json accepts the TypeScript JSON shape and rejects shadowRamp, null leaves and non-object containers; to_json emits Go's staticPolicyMap shape; validate enforces the TTL, ramp, recovery-age and read-deadline domains; resolve_policy merges a sparse runtime overlay once and applies the portable failure scopes (a malformed container, flag or deadline fails the invocation, an invalid TTL or ramp disables only that layer, an invalid recovery or shadow option preserves serving with a diagnostic flag). Tests port every case of go/policy_test.go and add JSON-shape, preset and SPEC overlay-table cases, using only ramps that short-circuit cohort sampling.
… and shadow validation
…e compression envelope Port the portable wire protocol to the Rust crate: frame encoding and the decode classification precedence, watermark parsing, the semantic read-result trust boundary (untrusted JSON replies and normalization), the timestamp and duration domains, WHATWG replacement UTF-8 conversion, and the payload envelope (escaping, zstd compression selection and first-frame decompression with the output cap). Add a vector runner for the eight frame/decode/timestamp/duration/envelope groups and a conformance test that replays every fixed and Quint-generated protocol vector (985 in total), plus module unit tests for the PROTOCOL.md replacement table, first-frame trailer handling, truncated bodies and the read/write limit outcomes. The test loads fixtures through a temporary local copy of the shared fixture helpers (tests/formal/fixtures_tmp.rs) until the shared tests/formal/fixtures.rs lands.
Add the language-neutral replay plumbing the Rust conformance drivers plug into: a JSON-lines client for the shared Node coordinator with a real-time watchdog and exact envelope checks, a local interpreter for protocol.schema.json, strict JSON and ITF decoding, corpus and inventory-id discovery, the JSONL assertion report and the witness-evidence check with a self-contained SHA-256. harness_infra ports the Go transport, JSON, registry and witness controls.
…est-util runtime and codec adapter
…rivers, fixed scenarios and settlement control
d7e8d60 to
530dcd2
Compare
…vent-to-metric mapping Add metrics.rs (MetricKind: the nineteen kinds, label derivation in wire order, values in seconds/bytes/ratio), prometheus.rs (feature "prometheus": the nineteen collectors with the TypeScript names, help, labels and buckets; reuse per registry and prefix; conflicts error and roll back) and datadog.rs (DogStatsD client trait, namespace validation, 200-character name limit, histogram or distribution observations). Both observers opt in to shadow outcomes. Tests port the Go names/units/labels, wire-schema, reuse and conflict-isolation checks and prove a panicking observer never changes a cached result.
…nvalidation vector tests RedisAdapter implements Remote over a caller-owned redis crate handle (ConnectionManager, MultiplexedConnection or ClusterConnection) through a small RedisConnection trait. Wire behavior matches Go and TypeScript: untracked GET, tracked MGET routed explicitly to the slot primary on a cluster, exactly one SET ... PX per write, EVALSHA with one EVAL retry carrying identical arguments for invalidation, strict reply validation that is never retried, and cooperative read cancellation. The Lua transition is byte-identical to go/redis_adapter.go and pinned by a unit test; the redis dependency gains the connection-manager feature, which the crate requires for ConnectionManager. tests/redis_integration.rs starts Redis 6.2, Redis 7 and Valkey 8 standalone servers plus a single-node Redis 7 cluster with Docker when DIALCACHE_RUST_INTEGRATION=1 and replays frame round trips, watermark fencing, SCRIPT FLUSH recovery and all 337 invalidation vectors through tests/formal/invalidation_vectors.rs.
…heus collectors by clone, and run the production clock alignment in the local-clock replay Review follow-ups on the Rust port: - DialCache::enable now closes its scope through the RAII guard, so a dropped or unwinding callback future no longer leaves retained Scope clones enabled. - execute hands a scope owned by another instance through to the source unchanged instead of replacing it with Scope::outside(). - Error::is_fallback_timeout also recognises a nested dialcache::Error boxed by a source. - Flight registration and detached work are RAII: a leader task dropped at runtime shutdown unregisters its flight and settles followers with an error, shadow slots free themselves, and start_pending settles its cell from Drop. - LocalStore hands displaced and expired entries back and Owner::close retires its memo outside the lock, so value destructors never run under a cache lock; cancel callbacks are panic-isolated. - TokioRuntime captures a runtime handle at construction; building outside a tokio context is a ConfigError rather than a panic on first use. A tokio integration test covers coalescing, deadlines and the cancellation contracts. - SystemClock::with_sources runs the production grid alignment over caller-supplied sources; the local-clock replay now uses it over the virtual clock instead of a separate test double. - PrometheusObserver drops the address-keyed reuse table: clone one observer per registry, a second registration is a Conflict. - Harness: case timestamps bound the actual run, the settlement control replays each profile once, redis_integration declares its feature. - Validation: cargo runs inside rust/ so rustup honours the toolchain pin; make integration-rust runs the real-server tests and CI calls it; the formal guides describe three replaying ports.
# Conflicts: # rust/src/clock.rs # rust/src/testing.rs
…cond-round review fixes Mutation lane. make mutations-rust / make mutations-merge-rust measure the Rust fault catalog formal/rust-mutations.json (13 single-site faults with the same contract cases as the TypeScript and Go catalogs) through formal/measure-rust-semantics.mjs: each mutant is applied to an isolated copy of the crate, compiled in release mode with dependencies shared through rust/target/semantic, and run against three cohorts. Ordinary is the crate's unit and native tests; generated and fixed are the conformance harness under the new DIALCACHE_RUST_SUITE selector over the complete corpus or the fixed scenarios. The lane shares the shard partition, fingerprint, gate and merge with the other ports (languages.rust; fingerprints now accept file paths and exclude the build directory), runs as three shards plus a merge in the full formal workflow, and is required by the aggregate. First complete run: 13/13 detected by the generated cohort, 12/13 by the fixed scenarios (M09 survives fixed, as in Go), 5/13 by ordinary tests, 778 s unsharded. Cluster routing. A Docker-backed test builds a Redis 7 primary plus replica on a private network, remaps their announced addresses with node_address_map, enables replica reads, and asserts from INFO commandstats that every tracked MGET executed on the primary and none on the replica; the mutation to ReplicaOptional fails it. The Docker tests are #[ignore]d so a plain cargo test reports them as ignored, and make integration-rust runs them with --ignored. Review round two (19 confirmed findings): a displaced request-memo value now drops outside the owner lock; the Prometheus rollback docs state what the prometheus crate keeps; the exporter tests assert each kind's full tag list instead of a vacuous never-exported check; the scripted Redis double fails loudly on an unexpected dispatch and hangs only on request; the smoke and exploration workflows install the pinned Rust toolchain; explore probes cargo; an incomplete Rust exploration report is diagnosed as such; the tokio deadline test releases its source explicitly instead of racing timers; and the rustdoc for Request, Fallback, Warn, config_error, build and the compression read result now matches the code.
Takes the Go API reshape (#190) and re-pins the source audit and Go-parity ledger inputs for the guides this branch edits.
lan17
left a comment
There was a problem hiding this comment.
The Rust port has broad feature coverage and substantial shared conformance evidence, but it is not yet fully behaviorally equivalent to the TypeScript implementation. I found four implementation gaps and one validation flaw at commit e263903f9af74e11e14964ed5f396d6d696533fa.
Recommendation: address the five findings before accepting the full TypeScript parity claim. The inline comments contain the reproduction details and suggested fixes. All five are P2: they have specific triggers, but affect supported configuration, failure isolation, cross-language identity, or the reliability of the acceptance evidence.
| Finding | Consequence | Evidence |
|---|---|---|
| Synchronous codec invocation falls outside the panic boundary | A cache decode panic prevents source fallback; an encode panic replaces a successful source result with an error | Two native Tokio regressions reproduced both outcomes |
| Numeric key formatting differs from JavaScript | Identical IEEE-754 inputs produce different Redis keys and potentially different rollout cohorts | Direct Rust and TypeScript key construction with the same binary float |
| Typed runtime overlays materialize omitted flags | A TTL-only update disables inherited request memoization and re-enables explicitly disabled coalescing | Rust regression fails; equivalent TypeScript test and Rust raw-JSON control pass |
| Local capacity is eagerly allocated | A valid large cache limit can cause an enormous allocation or process abort during construction | Rust dependency implementation compared with TypeScript's explicit sparse-allocation regression |
| Mutation measurement accepts infrastructure failures as detections | A broken replay can satisfy the assertion-strength gate without a behavioral assertion detecting the mutation | Direct call to evaluateRustReport accepted a malformed-observation failure as detected |
What appears aligned
The reviewed engine paths closely track TypeScript for explicit enablement and closed-scope pass-through; request, local and remote traversal; request/process coalescing and coalesce: false; source and read deadlines; tracked invalidation and primary reads; retained stale recovery; and dark/served-hit shadow admission, confirmation and publication. The protocol and adapter review also found matching frame-validation precedence, watermark rules, complete-frame native SET writes, invalidation Lua/retry behavior, compression envelopes, and exporter metric schemas.
These findings therefore concern concrete boundaries in a largely implemented port. They do not indicate that entire cache layers or major features are missing.
Validation performed on this head
make check-rustpassed: formatting, Clippy with warnings denied, library/native tests, exporter tests, settlement control, and all 1,736 default conformance cases.- The targeted TypeScript run of
test/dialcache-local.test.ts,test/formal-rust-semantic-runner.test.ts, andtest/formal-rust-replay.test.tspassed: 64 tests. - The Rust protocol-key, protocol-frame, and exporter test targets passed.
- Additional review regressions reproduced the codec, numeric-key, and typed-overlay divergences described inline. These failures are outside the passing committed suite.
- The mutation-classification issue was independently reproduced by passing a completed report containing a malformed-observation failure to
evaluateRustReport. - Temporary regression files were removed after review. No production changes were made.
The complete generated corpus, mutation campaign, and Docker Redis/Valkey/Cluster integrations were not rerun during this review. The successful local default suite should not be interpreted as a replacement for those full gates.
Limits of the parity evidence
- Cross-profile publication causality is not checked to the same extent as TypeScript. The Rust README explicitly acknowledges the missing monitor. TypeScript independently checks publication causality after behavior commands; the Rust feature replay does not carry the equivalent cross-profile attribution check. This is reduced independent assurance, not evidence that a specific Rust publication is incorrect.
- The new Redis integration lane does not exercise actual TypeScript-to-Rust and Rust-to-TypeScript round trips. It exercises Rust round trips, invalidation vectors, and cluster routing. Shared vectors are valuable, but the porting acceptance guidance also calls for bidirectional interoperability tests. The numeric-key discrepancy found here is a concrete reason to include cross-port identity and wire round trips in acceptance.
The explicit Scope handle and Arc<T> return values are reasonable Rust API adaptations. Other documented differences—TypeScript undefined decoding as JSON null/an optional value, default shadow equality using PartialEq, and sharing Prometheus observers by cloning—should remain visible in the compatibility documentation. They should be distinguished from the unintended behavior differences above.
Suggested acceptance work
Fix the four implementation boundaries and the mutation-result classifier, retain native regressions for Rust-specific callback/conversion behavior, add the numeric tie to the shared key vectors, and add actual bidirectional interoperability coverage. Run the full required conformance and mutation validation against the resulting head. If the independent publication monitor remains deferred, keep that explicit limitation in the parity claim.
Summary
Add the Rust
dialcachecrate with the same portable cache behavior and Redis wire format as the TypeScript and Go implementations. All three ports replay the shared Quint-generated histories, fixed scenarios, protocol vectors, and witness evidence.The crate includes explicit request scopes; request, local, and Redis layers; sparse runtime policy; deterministic rollout; request/process coalescing; tracked invalidation; read/source deadlines; stale recovery; shadow validation; compression; and failure-isolated observability. Optional adapters support Redis standalone and cluster connections, Prometheus, and DogStatsD.
Compatibility and conformance
Operation<T>cloning without requiring the cached value to implementClone.The Rust replay, integration, and mutation lanes are wired into the existing Make targets and CI. The shared inventory now contains 1,477 protocol vectors, 1,736 default Rust conformance cases, and 7,265 full required cases: 5,280 sampled histories, 250 named regressions, 244 fixed scenarios, 1,477 vectors, and 14 witness leaves. The shared Quint models, protocol vectors, witness definitions, and audit ledgers are unchanged by the parity fixes. Shared specification work remains in #188; numeric formatting regressions live in native Rust tests.
Rust API adaptations
Rust uses explicit
Scopehandles, typedUseCase<Args, T>handles, andArc<T>values. Sources returnResult<T, BoxError>, codecs and remote adapters use traits, and injectable clock/runtime interfaces support deterministic replay. TypeScriptundefineddecodes as JSONnull(Nonefor an optional destination); default shadow comparison usesPartialEq, so NaN differs from itself; Prometheus observers are shared by cloning.KeySpec::newandDialCache::invalidateuseIntoKeyIdfor strings, integers, floats, and their shared references. Custom displayable IDs must pass.to_string()or implementIntoKeyId.KeySpec::argaccepts every integer primitive without lossy float casts, plusf32/f64and borrowed inputs. Cross-language users must use matching namespace, identifier text, key dimensions, codecs, and policy. The core crate requires Rust 1.85; the currently locked Redis feature dependency requires Rust 1.88. See rust/README.md for the API and intentional adaptations.Validation
make checkpassed: TypeScript types, full native coverage, build/package checks; Go formatting, vet, and race checks; Rust formatting, Clippy, and native/default tests; documentation and source audits.make integrationpassed for TypeScript, Go, and Rust against Redis 6.2, Redis 7, Valkey 8, and Redis Cluster, including actual bidirectional TypeScript/Rust round trips, 337 invalidation vectors per server, and tracked-read routing to cluster primaries with replicas enabled.4ef4a07, all 12 focused API regressions and the full Rust native/integration checks passed. The complete Rust replay passed all 7,265 required cases from the unchanged shared corpus. Documentation and source audits passed again after the API changes.4ef4a07.4ef4a07: 13/13 faults detected by the generated corpus, 12/13 by fixed cases, and 5/13 by ordinary native tests. M09's fixed-case survival is the existing catalogued exception; all required detections passed. Six shards used distinct Cargo target directories and were validated together bymake mutations-merge-rust.