Krikos's simulation platform is built around capability injection: production and simulation run the same endpoint and Noq code while supplying different clocks, executors, behavioral decisions, sockets, and infrastructure. The source audit and approved architecture are in determinism-audit.md and deterministic-simulation-architecture.md.
Stage 2 runs production Krikos, Noq, TLS, QUIC stream, and QUIC datagram code over an in-memory IPv4/IPv6 packet network. The simulator owns link latency and bandwidth, MTU and queue bounds, routes, partitions, deterministic packet faults, virtual Krikos clocks, behavioral decisions, and resource accounting.
Stage 3 adds one strict declarative schema shared by JSON, Rust builders, deterministic generation, replay, minimization, and the permanent corpus. The runner executes those actions through the same production endpoint/Noq/TLS/QUIC path, continuously feeds typed observations to ordered safety/liveness/cleanup invariants, and checks action outcomes against a pure reference model.
Stage 4 activates stateful IPv4 NAT and ordered firewalls, double NAT/CGNAT chains, deterministic port mapping, mapping expiry/rebind, mutable interfaces/addresses/routes, sleep/resume, bounded discovery providers, and injected DNS timeout/stagger jitter. Production QUIC, endpoint identity, address aggregation, socket rebind, and monitor consumption remain the real implementations. Capability requirements still fail closed.
Stage 5 runs the production Krikos relay actor plus production relay client/server WebSocket framing, challenge authentication, authorization, client actor, registry, and routing over bounded simulator-owned byte pipes. The synthetic boundary replaces only DNS/TCP/TLS/HTTP listener setup. Relay-only, restart/outage, initially unavailable home relay, direct upgrade, direct failure and fallback, multiple-relay isolation, endpoint reincarnation, overload, duplicate identity, and shutdown-during-reconnect cases retain production QUIC above the relay path. Per-relay coverage observations prove connection attempts, authenticated sessions, and forwarded frames. A separate pure routing oracle is used only for differential checks and never satisfies production coverage.
Stage 6 moves production endpoint, Noq, socket-actor, and relay-actor roots onto the deterministic
kernel executor. A kernel/ready-task stream selects eligible tasks within causal ready waves;
the scheduler reports decisions, maximum wait, and fairness-forced choices, and forces an eligible
waiter after 32 selections. Trace schema 2 records task_scheduled events with the selected task
and ownership metadata. Terminal reports and failure artifacts include scheduler and historical
task-ownership snapshots, while runnable budget exhaustion is distinct from blocked quiescence.
The socket actor's periodic re-STUN and network-change timers use the injected clock, re-STUN
jitter uses a per-endpoint decision stream, and internal multi-ready selection has an explicit
branch order. Same-seed direct and relay production-QUIC replay are regression-tested.
Resource admission is fail-closed. The kernel bounds the simultaneously retained scheduled-event
queue separately from the cumulative execution budget, and it rejects timers, sockets,
connections, streams, and relays before their per-family ledger ceiling is exceeded. Schema-v3
declarative runs require explicit budgets.resources ceilings for scheduled events, trace events,
timers, sockets, connections, streams, and relays; tasks and queued packets retain their dedicated
max_tasks and max_packets limits. The trace-event ceiling wraps any installed trace sink with
an exact admission limit, while standalone in-memory TraceBuffers also have an explicit finite
capacity. Exhaustion is returned as a typed kernel, clock, ledger, or trace error; failure
signatures identify the exhausted resource, and no limit silently evicts or truncates evidence.
Runner-level regressions additionally cap live connections and relays at one, then verify that a
rejected second admission leaves trace, scheduler, task, and runtime state unchanged; partial relay
reservations roll back, shutdown returns the ledger to zero, and same-seed evidence is identical.
krikos-runtimedefines stable IDs, a monotonicClock,WallClock, resettable timers, structured task groups, domain-separated decision streams, the global trace schema, andRuntimeContext.- Normal
Endpoint::builder(...).bind()installs Tokio, system wall time, an OS-backed behavioral root seed, and a no-op trace sink. Endpoint identities, TLS token keys, and QUIC reset keys still use cryptographic randomness. - The explicit, doc-hidden
Builder::runtime_context_for_testpath requiresUnsafeTestOnly::acknowledge(). It is constructor injection, never an environment-variable or feature selection. The marker means “not a production default”; it does not weaken Rust memory safety. - Krikos's Noq adapter delegates
now, timers, and task spawning to one context. The endpoint supplies Noq with a behavioral RNG seed fromendpoint/<endpoint-id>/noq. Token validity uses the context wall clock. - Noq, socket, relay, direct-address report, active-relay, and remote-state actors run under the injected runtime and participate in structured cancellation and shutdown snapshots.
The production-endpoint lanes drive harness roots directly around kernel steps; they no longer
spawn paired Tokio harness tasks. Remote-map, relay-client, relay-impairment, net-report,
and shutdown deadlines use the injected clock, and relay ping/backoff choices use named decision
streams. Relay-server client actors now use the same injected executor, clock, wall clock, and
decision source as the endpoints. A narrow workspace Rustls 0.23.41 fork changes provider
component ownership from static references to owned Arcs. This permits run-owned,
endpoint-scoped deterministic random/X25519 components without leaks, process globals, or worker
coupling. Simulation-only QUIC connection IDs and relay authentication challenges are likewise
domain-separated from the run seed.
Manifest schema 3 records two explicit lanes. deterministic_test has no escapes, records
deterministic_test_crypto as a fidelity exception, receives fully_deterministic, and requires
raw trace equality. production_provider retains exactly production_crypto_entropy, receives
semantically_deterministic, and requires normalized replay masking only opaque ciphertext.
krikos-sim::RunManifest is strict JSON with unknown-field rejection. It records:
- source revision and dirty-tree digest;
- behavioral root seed;
- normalized scenario ID and digest;
- simulator and schema versions;
- sorted feature/configuration identity and Cargo.lock digest;
- deterministic wall-clock epoch;
- backend capabilities, determinism grade, crypto mode, and trace-comparison mode;
- sorted fidelity exceptions and observed escapes;
- event, virtual-time, task, and packet budgets;
- scheduling/fault profiles, unsafe-test marker, and explicit escapes.
Seeds and digests are lowercase fixed-width hexadecimal. Lists are sorted and unique. Host paths are rejected. Exact replay checks schema, simulator, source, dirty tree, scenario, features/configuration, and lockfile before execution; mismatches never silently proceed.
Runtime events contain a global sequence, run-relative virtual timestamp, typed entity references, and a versioned payload. Task spawn/completion/cancellation/panic/rejection, timer create/reset/fire/drop, decisions, state transitions, packet creation/per-hop scheduling/terminal outcomes, and fault observations have stable serialized forms.
ArtifactStore requires an explicit absolute directory internally; the CLI resolves relative paths before construction. It writes immutable manifest.json, normalized trace.jsonl, and forensic trace.raw.jsonl artifacts through a same-directory temporary file, sync_all, and atomic rename. During execution it also publishes bounded trace.chunk.*.jsonl and trace.raw.chunk.*.jsonl prefixes atomically, so a harness crash retains every completed chunk. It never overwrites an existing artifact. Trace normalization recursively redacts absolute host paths and opaque encrypted packet hashes. Replay uses the immutable manifest mode: deterministic-test artifacts compare raw events, while production-provider artifacts compare normalized events. Repeated direct and relay runs assert byte equality; production-provider repeats and cross-lane runs assert semantic equality.
Build or inspect the command with:
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- --helpThe stable command surface is run, campaign, soak, gate-select, replay, minimize,
corpus, explain, parity, and identity. Stage 6 activates deterministic execution across direct IP,
NAT/firewall, discovery, mobility, relay lifecycle, relay/direct paths, and seeded production-task
ready ordering.
The identity lane has a separate strict schema because account-control operations are not network
topology actions. It still uses the same Kernel, seeded ready-task scheduler, injected virtual
clock, RootSeed, trace schema, source-bound manifest, and immutable artifact store. Every action,
including a fault-only or rejected action, is a kernel-owned task and evaluates all twelve identity
invariants. A report is invalid unless each invariant counter equals the executed step count.
Scenario validation rejects zero identities or weights, duplicate or oversized authority lists,
unsatisfied or arithmetically unrepresentable recovery declarations, excess fork branches,
undeclared fork-resolution targets, and replica/provider bounds before the kernel is constructed.
The simulator-only invariant_fault action mutates independently captured oracle evidence; it
exists solely to exercise invariant failure capture and is visibly retained in canonical scenarios
and replay artifacts.
An action defaults to a required-success terminal, which is omitted from canonical JSON. A
semantically invalid transition may instead declare an expectation with terminal
model_rejection and one exact stable model-error discriminant. Only the exact declared rejection
is classified as a correct non-product terminal. An unmarked or mismatched model error, or a
declared rejection that unexpectedly succeeds, remains a product failure; semantically impossible
expectation/action pairings are rejected during scenario validation. Correct expected rejections
write a versioned identity-rejection-report.json with exact report and trace evidence and replay
through that terminal. They are never confirmed, action-deletion minimized, labeled as product
failures, or staged for corpus promotion.
The invariant observations distinguish the transition that first exposes a fork from later actions
that retain its complete head set. Provider/storage faults and freshness probes may run while the
fork remains visible, and only an explicit authorized resolution may reduce it. Offline validation
must report the exact captured sequence/epoch basis (or no_basis while forked), and the model's
persisted state contains no ordinary-account private-key inventory. The invariant oracle carries a
transient typed recipient set, initialized empty, only so the negative-control mutation can prove
that forbidden private-key replication would be detected; that set is neither model authority nor
durable simulator inventory.
The two reviewed corpus histories have distinct fixed seeds and jointly cover partition/heal, delay/reorder/loss/duplicate, concurrent children and explicit fork resolution, crash/reopen with storage loss, provider outage/equivocation, recovery, controller and device revocation, migration, and group-recipient rotation. Corpus loading rejects extra files, duplicate IDs or seeds, unreviewed entries, ID mismatch, and incomplete coverage. Confirmed failures are minimized only by bounded action deletion, and a candidate is retained only when it reproduces the exact normalized failure class and evidence digest.
Failure-bundle schema v2 indexes the original and minimized scenarios, terminal reports, raw traces, normalized traces, confirmation record, signature, and complete reduction transcript. Replay executes both scenarios twice, requires their exact common signature/report/trace evidence, checks canonical scenario digests, and reconstructs every deletion candidate through the real runner. Recomputing the file index cannot make a substituted original scenario, confirmation digest, or candidate digest semantically valid. Promotion records the verified artifact-index root and remains unreviewed until an explicit corpus review.
This minimized promotion lifecycle applies only to identity scenario-runner product-failure
terminals. Differential checks use an explicit retained RootSeed (the fixed CI seed is shown
below), while the formal checker is deterministic and seedless; those two lanes fail CI with their
bounded witness and are reproduced by their exact command rather than being action-deletion
minimized.
Run a scenario, replay its report plus raw and normalized traces byte-for-byte, check the corpus, and compare a generated production/reference history with:
cargo +1.91.0 run --locked --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- identity run \
krikos-sim/identity-corpus/network-storage-provider.json \
--seed 8181818181818181818181818181818181818181818181818181818181818181 \
--artifacts /tmp/krikos-identity-run
cargo +1.91.0 run --locked --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- identity replay \
/tmp/krikos-identity-run/manifest.json
cargo +1.91.0 run --locked --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- identity corpus-test \
krikos-sim/identity-corpus
cargo +1.91.0 run --locked --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- identity differential \
--seed 7373737373737373737373737373737373737373737373737373737373737373The executable reference model owns its controller, device, policy, fork, recovery, migration, and group-recipient transitions. It cannot import production identity code; a source-inventory test enforces that only the differential adapter may do so. Migration observations come from production lifecycle and checkpoint crypto commitments, including successful new-suite and rejected old-suite probes after retirement. Group recipients come from the production post-state distribution snapshot and deterministic key wrapping, not from the expected model projection.
This lane substitutes deterministic test cryptography and records that fidelity exception. It does not claim external provider realism, external TLC execution, independent language interoperability, or a production security audit. Encoded scenario inputs are rejected above 4 MiB before JSON deserialization and are further bounded to 256 actions and 60 seconds of virtual time; the model, queue, replica, provider, kernel event, task, trace, corpus, minimizer, and formal-state collections all have explicit fail-closed bounds.
Run either checked-in Stage 2 scenario with an explicit behavioral seed:
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- run \
krikos-sim/tests/fixtures/ipv4-stream.json \
--seed 1111111111111111111111111111111111111111111111111111111111111111 \
--artifacts /tmp/krikos-sim-ipv4
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- replay \
/tmp/krikos-sim-ipv4/manifest.json
# Exercise the production cryptographic provider with semantic replay.
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- run \
krikos-sim/tests/fixtures/ipv4-stream.json \
--seed 2222222222222222222222222222222222222222222222222222222222222222 \
--crypto production-provider \
--artifacts /tmp/krikos-sim-ipv4-production-crypto
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- replay \
/tmp/krikos-sim-ipv4-production-crypto/manifest.jsonRun a declarative scenario, test the reviewed corpus, and execute a bounded campaign:
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- run \
krikos-sim/tests/fixtures/v2-ipv4-stream.json \
--seed 1212121212121212121212121212121212121212121212121212121212121212 \
--artifacts /tmp/krikos-sim-v2
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- corpus test krikos-sim/corpus
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- campaign \
krikos-sim/tests/fixtures/v2-ipv4-stream.json \
--seeds 0..100 --jobs 4 --generated --continue-on-failure \
--artifacts /tmp/krikos-sim-campaign
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- campaign \
krikos-sim/corpus/stage4-nat-rebind-expiry/scenario.json \
--seeds 0..100 --jobs 4 --continue-on-failure \
--artifacts /tmp/krikos-sim-nat-campaign
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- campaign \
krikos-sim/corpus/stage5-relay-restart/scenario.json \
--seeds 0..100 --jobs 4 --continue-on-failure \
--artifacts /tmp/krikos-sim-relay-campaign
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- campaign \
--swarm krikos-sim/swarms/direct-smoke.json \
--seeds 0..100 --jobs 4 --continue-on-failure --max-runs 100 \
--artifacts /tmp/krikos-sim-swarm-campaignThe strict daily plan at krikos-sim/soaks/daily.json pins fourteen domain/crypto lanes and each swarm
template digest. cargo sim soak executes one bounded epoch, checks wall time only between
fixed-size batches, atomically checkpoints aggregate and per-lane counters, and retains no
successful trace. The daily shell runner starts eight fresh processes so process-global leakage
cannot accumulate across the full four-hour service:
cargo build --release --manifest-path krikos-sim/Cargo.toml --bin cargo-sim
scripts/run-daily-simulation-soak.sh \
--seed-window 1 \
--artifacts /tmp/krikos-daily-soak \
--sim-bin krikos-sim/target/release/cargo-simThe production service starts a window every six hours. It builds one source-bound simulator, fans the fourteen lanes out to standard GitHub-hosted runners, and gives each lane eight 30-minute epochs, four workers, and 64 scenarios per deadline boundary. A lane is capped at 83,328 runs, so each window is capped at 1,166,592 runs and the four daily windows at 4,666,368. Each lane retains at most sixteen failures and 256 MiB of failure data. Every epoch records a typed half-open seed lease bound to workflow run, policy digest, lane, and ordinal range. The final aggregate requires every lane exactly once, rejects lease overlap, reconciles all counters, and exposes configuration, transition, oracle, phase, and failure-signature coverage.
The aggregate job merges compatible evidence into a rolling seven-day ledger, rejects duplicate run IDs or cross-run lease overlap, and reports every unmet obligation with a typed reason. It emits a bounded next-lane selection rather than hiding gaps behind run totals. A policy revision starts a new window: valid older-policy and pre-policy reports are ignored, while malformed current-policy evidence is infrastructure failure. Confirmed product failures carry an integrity-indexed, versioned operational outcome bound to the normalized signature digest, are exactly replayed, minimized with at most 512 attempts, and are deduplicated into GitHub issues by normalized signature. Non-product, unindexed, replay, minimization, issue API, or evidence failures stay infrastructure-classified. Tracked issue closure is separately fail-closed: the default-branch closure workflow reopens the issue unless exactly one reviewed minimized corpus entry matches its signature and both same-revision deterministic GitHub Actions checks succeeded. A promotion cannot retain the original product-failure signature as its expected terminal, and its scenario SHA-256 must match both the tracked issue and checked-in bytes.
Scenario JSON is strict and currently supports direct-ip/ipv4-stream, direct-ip/ipv4-stream-loss, direct-ip/ipv4-stream-corruption, direct-ip/ipv6-stream, and direct-ip/ipv6-datagram. The checked-in loss fixture and seed demonstrate QUIC recovery after real packet loss. The corruption fixture requires the injected corruption to occur and treats the resulting authenticated-transport failure as its expected terminal result. Unknown fields, schemas, and scenario IDs fail closed. run writes the manifest before endpoint execution and prints one replay command. replay verifies source revision, dirty-tree digest, dependency lockfile, simulator/schema version, scenario digest, normalized configuration, features, backend identity, crypto/grade/comparison matrix, budgets, and seed before comparing the manifest-selected raw or semantic trace.
Schema v3 contains metadata, exact backend requirements, hard budgets with explicit per-resource admission ceilings, hosts/interfaces/links, endpoints, stable action IDs, schedules, fault rules, fairness assumptions, completion policy, allowed terminals, and enabled invariants. Canonical encoding sorts set-like collections and rejects duplicates, dangling references, host paths, unbounded values, unknown fields, and unsupported schemas. Schema v1 and v2 documents migrate only through the explicit versioned loader; strict current-schema parsing accepts v3 only.
Actions may start/stop endpoints, connect, exchange a stream or datagram, close, partition/heal, update a link, advance virtual time, wait on an injected virtual-clock timer, rebind a NAT, request a port mapping, mutate discovery records, change relay lifecycle, change interface/address/route state, or sleep/resume a host. Observation-triggered actions use endpoint, connection, or satisfied-invariant predicates. The invariant families cover authenticated peer identity, delivery integrity and misdelivery, per-stream ordering, monotonic lifecycle, relay routing, resource ceilings, shutdown cleanup, and fairness-qualified reachable-connect liveness. Safety checks run on the first matching observation; liveness is bounded by virtual time and event count.
The reviewed resource corpus contains deterministic expected failures for connection, stream,
socket, timer, trace-buffer, and relay admission. Each ceiling is part of the canonical scenario,
each failure signature names the typed resource, and failed-run cleanup must leave the live ledger
empty. The nightly resource_exhaustion matrix runs and replays all six with one fixed seed and
retains their artifacts for 14 days.
Swarm schema v1 embeds a canonical scenario or references a workspace-relative, BLAKE3-bound base
and declares a sorted, bounded set of weighted choices. Supported mutations cover payload size,
sleep-action duration, link latency, bandwidth, MTU, and queue capacity, packet-fault probability,
relay availability/impairment, NAT behavior, discovery timing/state, mobility action timing, and
co-scheduled ready pressure. Empty choices,
zero or excessive weights, invalid bounds, dangling targets, path traversal, digest drift, unknown
fields, and noncanonical ordering fail before backend construction. The materialization
seed is domain-separated from the runtime seed. Bandwidth and queue mutations may only tighten the
validated base-link bounds. A sleep-duration mutation must target an existing sleep action and stay
positive and within the scenario's maximum virtual time. A zero-probability reordering rule is
disabled rather than creating a reorder window. Each campaign run stores its selected option for
every choice in swarm-selection.json beside the fully materialized scenario.json; replay and
triage therefore consume the artifact rather than regenerating it. Checked templates cover direct,
link impairment, NAT, discovery, mobility, relay lifecycle, and ready-order pressure. Under both
cryptographic providers, the link-impairment template continuously selects bounded bandwidth,
queue pressure, duplication, reordering, and brief or sustained blackhole duration. It sends a
production QUIC datagram into an explicit directional partition, advances the selected duration,
observes a bounded 10 ms receive window for nondelivery, and requires the trace to record
dropped:partition. The window exceeds the fixture's maximum modeled one-way latency plus reorder
delay, and the fail-open regression exercises 64 guaranteed-reordering seeds. After healing, the same
connection carries the original 64 KiB stream workload before a fresh connection and stream round
trip satisfy the bounded liveness oracle. The relay template records an
explicit outage-to-recovery transition and dependency-ordered, fairness-qualified bounded liveness
probe. Campaigns accept either crypto lane and retain the selection in crypto-mode.txt.
A failing declarative run stores the canonical scenario, terminal report, invariant and resource snapshots, scenario-domain inventory, normalized FailureSignature, decision prefix, raw/normalized traces, and contiguous immutable chunks. The signature uses the invariant name when present, normalized responsible entities, typed terminal class, and a bounded causal-suffix digest. Replay verifies artifact integrity and the signature before applying the manifest-selected raw or semantic comparison, so disappearance, a different failure, a missing/truncated chunk, and trace divergence remain distinct.
Minimize a failure and resume an interrupted reduction with:
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- minimize /tmp/krikos-sim-failure/manifest.json \
--output /tmp/krikos-sim-minimized --max-attempts 10000
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- minimize /tmp/krikos-sim-failure/manifest.json \
--output /tmp/krikos-sim-minimized --resume --max-attempts 10000The reducer deterministically deletes action/fault chunks, NATs, firewall rules, discovery
providers/records, relays and relay lifecycle actions, interfaces and routes; prunes unused topology and endpoints; and reduces domain
scalars and representation budgets. Every candidate is revalidated and memoized by canonical
digest. Only an exact signature match is accepted. minimize.jsonl is synced after each attempt
and best.scenario.json is atomically replaced after every improvement, so budget exhaustion
retains the best valid candidate.
Each directory below krikos-sim/corpus must contain exactly metadata.json and scenario.json.
Corpus metadata schema v2 records seed, expected terminal/signature, provenance, issue,
schema/simulator compatibility, review state, and an exact ScenarioInventory. Historical fixtures
use a bounded symbolic issue reference. A real GitHub issue URL requires typed promotion evidence:
the original signature digest, minimized-scenario SHA-256, discovery revision and workflow run,
exact replay, and signature-preserving minimization. Unenumerated files, duplicate IDs,
incompatibility, missing provenance, invalid promotion evidence, changed domain counts, and changed
signatures fail corpus test. The reviewed entries cover NAT rebind/expiry, discovery
conflict/expiry/refresh, relay restart, direct-to-relay fallback, and a four-way production-task
ready-order seed promoted from the Stage 6 scheduler campaign.
Campaigns execute half-open seed ranges in deterministic worker batches. Results are sorted by seed, failure signatures are deduplicated independent of completion order, fail-fast stops only at a stable batch boundary, and every run gets its own artifact directory. Campaign summaries retain the template inventory.
PR and merge-queue CI run two required simulation checks. Deterministic simulation contracts and corpus executes the complete reviewed corpus plus model, property, replay, resource, workflow, and
policy contracts. Deterministic simulation change gate always runs one canary for every one of the
six domains under both cryptographic providers, then adds work selected by the versioned
source-path impact map. BLAKE3 binds each seed to candidate revision, both policy digests,
domain/provider lane, work kind, and ordinal. Unknown, global, or unavailable diffs fall back to
all domains. Pull-request selection is capped at 24 runs and 15 minutes; main selection is capped at
64 runs and 30 minutes.
The weekly hosted service audits those wall-time targets against the latest 20 compatible successful executions per tier. It publishes explicit insufficient-history evidence during rollout and fails on a measured P95 breach instead of weakening the deterministic gate.
Nightly no longer repeats fixed exploratory seed ranges. It consumes the latest compatible daily gap selection (at most eight selected gap lanes), falls back to the twelve universal canaries when fresh evidence is absent, and separately retains the five exact replay and six expected-resource audits as permanent tests. Weekly runs service/corpus/parity checks and correlated benchmarks; it does not run another deterministic seed-range explorer. Main also runs realistic Netsim, while the daily hosted Patchbay public case continuously executes the same canonical semantic workload as the deterministic parity exporter.
The release workflow gates every pinned candidate before build or publication. It requires both same-revision deterministic checks, same-revision Netsim, no open confirmed simulation product failure, and fresh successful public Patchbay parity evidence. Every query and result set is bounded and produces a retained typed readiness report before propagating its status.
The recurring soak uses the same deterministic batch semantics without retaining successful
per-seed directories. Twelve standard GitHub-hosted lane jobs run independently, checkpoint after
every batch, and each supply four internal workers. The workflow builds cargo-sim once, binds the
uploaded binary to the source SHA with a checksum, assigns disjoint run-number-derived seed
windows, and aggregates exact lane evidence after every matrix job has finished. Workflow
concurrency one queues overlapping scheduled or manual windows instead of cancelling them. The
relay lifecycle and link-impairment swarms additionally declare separate fault and recovery phases:
safety remains active during the bounded disruption and a dependency-ordered,
fairness-qualified connection probe must complete within virtual-time and event-count bounds after
healing.
The checked workflow and its seconds-long contract are definitions, not evidence that a hosted
window completed.
For a compact terminal, obligation, resource, causal-suffix, and command summary:
cargo run --manifest-path krikos-sim/Cargo.toml --bin cargo-sim -- explain /tmp/krikos-sim-failure/manifest.jsonCross-backend fixtures use cargo sim parity export and cargo sim parity compare. The complete
service SLO, retention, triage, corpus-promotion, schema-migration, realistic-backend, soak, and
performance-correlation runbook is ../simulation/operations.md.
Run the deterministic-closure gates with:
scripts/tests/check-determinism-boundaries.sh
scripts/tests/check-determinism-semantic.sh
scripts/check-determinism-boundaries.sh --check
scripts/check-determinism-semantic.sh --check
scripts/tests/check-determinism-docs.sh
scripts/tests/check-daily-simulation-soak.sh
scripts/tests/check-daily-simulation-aggregate.sh
scripts/tests/check-daily-simulation-workflow.sh
scripts/tests/check-simulation-coverage-history.sh
scripts/tests/check-simulation-coverage-collector.sh
scripts/tests/check-latest-simulation-gap-collector.sh
scripts/tests/check-simulation-gate-selector.sh
scripts/tests/check-simulation-gate-runner.sh
scripts/tests/check-simulation-gate-workflow.sh
scripts/tests/check-simulation-failure-triage.sh
scripts/tests/check-simulation-issue-upsert.sh
scripts/tests/check-simulation-nightly-workflow.sh
scripts/tests/check-simulation-weekly-workflow.sh
cargo test -p krikos-runtime
cargo test -p krikos-dns --lib
cargo test -p krikos-relay --all-features
cargo test --manifest-path krikos-sim/Cargo.toml
cargo test --manifest-path krikos-sim/Cargo.toml --test swarm
cargo test -p krikos --lib --all-features
cargo clippy -p krikos-runtime -p krikos --all-targets --all-features -- -D warnings
cargo clippy --manifest-path krikos-sim/Cargo.toml --all-targets --all-features -- -D warnings
cargo check -p krikos --no-default-featuresCI also builds the portable krikos-runtime contracts and Krikos for wasm32-unknown-unknown.
Patchbay remains the privileged realistic Linux backend and is intentionally separate from these
in-process contracts. The versioned semantic importer, canonical case catalog, and outcome matrix
are documented in ../simulation/patchbay-parity.md.
Topology construction rejects duplicate hosts, links, addresses, interfaces, ports, explicit routes, and equal connected-route prefixes. Routing is longest-prefix and multi-hop with loop detection. Source selection respects the bound family and host-owned addresses. Link serialization uses checked integer nanoseconds and rounds positive fractional transmission time upward.
Loss, duplication, corruption, and reorder delay have independent semantic decision streams per link. Partitions, queue overflow, MTU rejection, no route, invalid source, closed destination, and simulator budget failures retain distinct outcomes. Packet delivery, timers, tasks, sockets, and queued copies use RAII ledger ownership; successful endpoint shutdown must reconcile the ledger to zero.
Stage 6 scenarios inject synthetic UDP sockets, mutable monitor state, NAT/firewall/port-mapping capabilities, discovery providers, DNS behavioral time, and relay connectivity. They do not construct the corresponding OS adapters or relay DNS/TCP/TLS/HTTP/QAD adapters. The paired runner compatibility tasks, core client-side Tokio timers, and relay-server actor roots have been retired. The deterministic-test lane owns TLS, QUIC connection-ID, and relay-challenge entropy and compares raw traces. The production-provider lane intentionally retains secure production entropy as its only escape and masks only opaque ciphertext payload hashes for semantic comparison. Real net-report HTTP/STUN probes and platform interfaces remain realistic-backend coverage.
Stage 2 adds capability dispatch at IP socket construction and send/receive boundaries only when the transport is constructed; normal builders still install the concrete netwatch adapter and secure entropy. The production default uses a no-op trace sink. Existing connection, packet-throughput, and relay benchmark commands remain the authority for performance review; no threshold is changed by Stage 2. Record machine identity, revision, feature set, command line, and raw output before accepting a new baseline.
Stage 3 adds no observer branch to the production Krikos crate: ScenarioRunner, reference models, invariants, corpus, and minimization live in the separate non-published krikos-sim crate, so an application that does not construct the simulator has no Stage 3 runner/observer path to disable. A non-gating 2026-07-21 container measurement on this worktree ran 32 generated production-QUIC scenarios with four workers in 4.26 seconds wall time (7.51 scenarios/second; 7.11 seconds user, 0.18 seconds system):
/usr/bin/time -p krikos-sim/target/debug/cargo-sim campaign \
krikos-sim/tests/fixtures/v2-ipv4-stream.json \
--seeds 100..132 --jobs 4 --generated --continue-on-failure \
--artifacts /tmp/krikos-sim-throughputThis is evidence, not a CI threshold: production crypto entropy makes host-to-host timing comparisons unsuitable until stable runner distributions exist.
Stage 4 adds Criterion guardrails for NAT mapping reuse, ordered firewall decisions, discovery
replace/withdraw, and production builder construction with simulator hooks disabled. The measured
reference ranges, interpretation limits, and nightly retention policy are in
../simulation/stage4-performance.md.
Stage 5 adds connector-disabled construction, production in-memory authentication/session, and
relay datagram-routing guardrails described in
../simulation/stage5-performance.md. Relay semantic parity
and intentional backend differences are in
../simulation/relay-parity.md.
Stage 6 adds a FIFO-versus-seeded scheduler microbenchmark, nightly reports, and a measured 3.1%
median overhead on the 256-ready-task reference workload. Method and raw interpretation limits are
in ../simulation/stage6-performance.md.