Release: develop -> main - #197
Merged
Merged
Conversation
… tokens (#192) * feat: permissionless multi-asset — asset_id plumbing, circuit extension, API endpoints Add asset_id as a first-class concept throughout the protocol: - Types: AssetId type alias, NATIVE_ASSET_ID, calculate_asset_id(), asset_id field on Coin/CoinTemplate/Invoice/ProofData - Circuit: N_PROOF_DATA_PUBLIC_INPUTS 16→20, transition_asset_id extracted from PIs[16..20], source asset_id equality gate in the in-coin loop, out-coin identifier derivation extended to H(interim_asth || asset_id || slot_index) - Prover: asset_id parameter threaded through all prove_* functions - Node: Account.balances BTreeMap for per-asset tracking, asset CRUD in db.rs, POST /api/asset/create + GET /api/asset/list + GET /api/asset/info/:id endpoints, multi_asset capability flag - Schema: migration 0015_multi_asset.sql creates assets table (no name UNIQUE per issue #191 design) Closes #191 * fix: add multi_asset capability to api_remote integration test Thread the new multi_asset capability flag through the fetch_capabilities helper and the force-disable match arm. * fix: address logic-reviewer findings — wire asset_id through flows, add mixed-asset rejection - Account.balances is now updated in send_coins_inner after prove - flow.rs parses request.asset_id instead of hardcoding NATIVE_ASSET_ID - Off-circuit mixed-asset pre-check rejects mismatched asset_ids - multi_asset capability set to false until endpoints are wired - Stale aggregator PI count comment corrected (204 → 236) - Negative test: send_coins_rejects_mixed_asset_invoices * docs: correct stale aggregator PI layout comment (17→21 per slot) * fix: resolve clippy type_complexity in asset DB queries Use sqlx::FromRow derive on AssetRow instead of raw tuple decoding. * fix: add assets table to connect_and_migrate_creates_all_tables assertion * test: add same_name_different_creator negative test (issue #191) * refactor: defer asset-registration layer; keep additive circuit plumbing The 100% line+function coverage gate (node package) failed because the asset-registration surface added earlier had no production callers: the create/list/info handlers were 501/empty/404 stubs that never reached the DB CRUD, and get_asset_balances / Account.balances were write-only. Covering dead code (or shipping unwired endpoints) is the wrong fix. Multi-asset is an ADDITIVE extension over the existing off-circuit mint path, not a new public CRUD API. Per the node trust model send/receive stay trustless and asset support rides the same mint/send transition, so no asset-registry endpoint is required for the MVP. Removed (deferred to a follow-up built lockstep with the wallet app): - /api/asset/{create,list,info} handlers + routes - CreateAssetRequest / AssetResponse / AssetListResponse DTOs - db::insert_asset / get_asset / list_assets + AssetRow - migration 0015_multi_asset.sql (assets table) - AssetBalance + BalanceResponse.balances, Account.balances, get_asset_balances - openapi component registrations for the above Kept (additive, exercised end-to-end): - asset_id on Coin / Invoice / CoinTemplate / ProofData - calculate_asset_id / NATIVE_ASSET_ID / ASSET_GENESIS_DOMAIN_TAG - mixed-asset rejection in send_coins_inner (both in-coin branches) - MintRequest.asset_id / SendCoinRequest.asset_id wired through mint/send Hardening (no silent fallbacks): asset_id hex parsing in mint_flow / send_flow no longer defaults a present-but-malformed value to native. An absent field selects native; a present invalid or wrong-length value is a hard 422. Adds parse_optional_asset_id. Tests: add send_coins_rejects_queued_coin_with_foreign_asset (covers the coin_queue branch of the asset guard); fix warmup_prover formatting so the prove_initial `?` stays line-covered; restore connect_and_migrate expected-table list to the post-0014 schema. * test: drop assets table from connect_and_migrate assertion Reconciles the merged-in 4b8d907 (which added "assets" to the expected schema for migration 0015) with the registration-layer deferral: 0015 is removed, so the assets table is no longer created. Restore the expected list + comment to the post-0014 schema. * docs: make coverage gate + api_remote mandatory local pre-push gates Both jobs that most often go red after a push are now reproducible locally before pushing, turning a ~13 min red-CI round-trip into a local check: 1. Coverage gate — `cargo llvm-cov nextest ... --fail-under-lines 100 --fail-under-functions 100`, with the `--ignore-filename-regex` copied verbatim from .github/workflows/ci.yaml (node package, lines + functions). Verified locally: 442 tests, 100% lines + 100% functions. 2. api_remote (47 tests) against a local node pointed at public Mutinynet with an on-chain-funded publisher. Verified locally: 47/47. 39 are funding-free contract checks; the 8 mint/send/commit roundtrips broadcast real Taproot inscriptions and need the funded publisher (else "Failed to broadcast mint inscription on-chain"). Documents the env setup (~/.config/zkcoins/mutinynet.env) and the faucet reality (faucet.mutinynet.com is an L402 Lightning paywall, not a simple address faucet — fund the publisher P2TR address out-of-band).
… tokens (#192) (#194) * feat: permissionless multi-asset — asset_id plumbing, circuit extension, API endpoints Add asset_id as a first-class concept throughout the protocol: - Types: AssetId type alias, NATIVE_ASSET_ID, calculate_asset_id(), asset_id field on Coin/CoinTemplate/Invoice/ProofData - Circuit: N_PROOF_DATA_PUBLIC_INPUTS 16→20, transition_asset_id extracted from PIs[16..20], source asset_id equality gate in the in-coin loop, out-coin identifier derivation extended to H(interim_asth || asset_id || slot_index) - Prover: asset_id parameter threaded through all prove_* functions - Node: Account.balances BTreeMap for per-asset tracking, asset CRUD in db.rs, POST /api/asset/create + GET /api/asset/list + GET /api/asset/info/:id endpoints, multi_asset capability flag - Schema: migration 0015_multi_asset.sql creates assets table (no name UNIQUE per issue #191 design) Closes #191 * fix: add multi_asset capability to api_remote integration test Thread the new multi_asset capability flag through the fetch_capabilities helper and the force-disable match arm. * fix: address logic-reviewer findings — wire asset_id through flows, add mixed-asset rejection - Account.balances is now updated in send_coins_inner after prove - flow.rs parses request.asset_id instead of hardcoding NATIVE_ASSET_ID - Off-circuit mixed-asset pre-check rejects mismatched asset_ids - multi_asset capability set to false until endpoints are wired - Stale aggregator PI count comment corrected (204 → 236) - Negative test: send_coins_rejects_mixed_asset_invoices * docs: correct stale aggregator PI layout comment (17→21 per slot) * fix: resolve clippy type_complexity in asset DB queries Use sqlx::FromRow derive on AssetRow instead of raw tuple decoding. * fix: add assets table to connect_and_migrate_creates_all_tables assertion * test: add same_name_different_creator negative test (issue #191) * refactor: defer asset-registration layer; keep additive circuit plumbing The 100% line+function coverage gate (node package) failed because the asset-registration surface added earlier had no production callers: the create/list/info handlers were 501/empty/404 stubs that never reached the DB CRUD, and get_asset_balances / Account.balances were write-only. Covering dead code (or shipping unwired endpoints) is the wrong fix. Multi-asset is an ADDITIVE extension over the existing off-circuit mint path, not a new public CRUD API. Per the node trust model send/receive stay trustless and asset support rides the same mint/send transition, so no asset-registry endpoint is required for the MVP. Removed (deferred to a follow-up built lockstep with the wallet app): - /api/asset/{create,list,info} handlers + routes - CreateAssetRequest / AssetResponse / AssetListResponse DTOs - db::insert_asset / get_asset / list_assets + AssetRow - migration 0015_multi_asset.sql (assets table) - AssetBalance + BalanceResponse.balances, Account.balances, get_asset_balances - openapi component registrations for the above Kept (additive, exercised end-to-end): - asset_id on Coin / Invoice / CoinTemplate / ProofData - calculate_asset_id / NATIVE_ASSET_ID / ASSET_GENESIS_DOMAIN_TAG - mixed-asset rejection in send_coins_inner (both in-coin branches) - MintRequest.asset_id / SendCoinRequest.asset_id wired through mint/send Hardening (no silent fallbacks): asset_id hex parsing in mint_flow / send_flow no longer defaults a present-but-malformed value to native. An absent field selects native; a present invalid or wrong-length value is a hard 422. Adds parse_optional_asset_id. Tests: add send_coins_rejects_queued_coin_with_foreign_asset (covers the coin_queue branch of the asset guard); fix warmup_prover formatting so the prove_initial `?` stays line-covered; restore connect_and_migrate expected-table list to the post-0014 schema. * test: drop assets table from connect_and_migrate assertion Reconciles the merged-in 4b8d907 (which added "assets" to the expected schema for migration 0015) with the registration-layer deferral: 0015 is removed, so the assets table is no longer created. Restore the expected list + comment to the post-0014 schema. * docs: make coverage gate + api_remote mandatory local pre-push gates Both jobs that most often go red after a push are now reproducible locally before pushing, turning a ~13 min red-CI round-trip into a local check: 1. Coverage gate — `cargo llvm-cov nextest ... --fail-under-lines 100 --fail-under-functions 100`, with the `--ignore-filename-regex` copied verbatim from .github/workflows/ci.yaml (node package, lines + functions). Verified locally: 442 tests, 100% lines + 100% functions. 2. api_remote (47 tests) against a local node pointed at public Mutinynet with an on-chain-funded publisher. Verified locally: 47/47. 39 are funding-free contract checks; the 8 mint/send/commit roundtrips broadcast real Taproot inscriptions and need the funded publisher (else "Failed to broadcast mint inscription on-chain"). Documents the env setup (~/.config/zkcoins/mutinynet.env) and the faucet reality (faucet.mutinynet.com is an L402 Lightning paywall, not a simple address faucet — fund the publisher P2TR address out-of-band). Co-authored-by: TaprootFreak <142087526+TaprootFreak@users.noreply.github.com>
github-actions
Bot
added
the
ci:full
… subset parallelism (#196) The "DB Subset Tests" CI job runs a narrow nextest selection (`db::tests` + `job_store::tests` + `router::tests::jobs_*` + ...) under `--test-threads 8`. In that job the five SSE tests `router::tests::jobs_endpoint_tests::jobs_stream_*` intermittently fail with `create: PoolTimedOut` after running >100 s, while the full coverage gate (same `--test-threads 8`, on the same SHA) keeps them green. Root cause is migration-replay contention, not raw connection exhaustion. Every test that calls `crate::test_db::setup_pool()` CREATEs a fresh per-test schema and replays the full migration suite (16 DDL files: tables, triggers, views) into the single shared `postgres:17` container. Postgres serialises concurrent DDL on its system catalogs, so when the DB subset packs the migration-replaying tests together and eight run at once, each `setup_pool()` stretches from <1 s to tens of seconds. The `jobs_stream_*` tests additionally hold their pool across deliberate sleep/timeout windows, so under that contention their connection acquisition exceeds the pool's 60 s `acquire_timeout` and surfaces as `PoolTimedOut`. The full gate stays green because the same heavy tests are interleaved across the entire suite rather than clustered. Peak server connections stay ~14/100 throughout, confirming the bottleneck is DDL catalog locking. Fix: add a workspace-root `.config/nextest.toml` test-group that caps the `router::tests::jobs_endpoint_tests` module at 2 concurrent threads. This bounds simultaneous migration replays for the heaviest module so connection acquisition stays well under the 60 s timeout, while keeping useful parallelism for the rest of the suite. The config is honoured by both `cargo nextest run` (the subset gates) and `cargo llvm-cov nextest` (the coverage gate), carries no coverage semantics, and touches no test pool — the deliberately-narrow error-path `dead_pool` (`max_connections(1)` / 50 ms timeout) keeps exercising its `PoolTimedOut` arms verbatim. Verified locally: the exact DB-subset selection now passes 239/239 twice with no `PoolTimedOut` (the `jobs_stream_*` tests drop from ~34 s to ~15 s each), and the 100% line + function coverage gate is unchanged.
* docs: add decentralization roadmap (run-your-own-node, SPEC-anchored) * docs: rename to DECENTRALIZATION_ROADMAP.md (ROADMAP.md already taken) * docs(roadmap): set decentralization as the current focus (fold in S1-S7 + D2/D7/D8) * docs(roadmap): set decentralization as the current focus (fold in S1-S7 + D2/D7/D8)
…the jobs API (#198) The jobs-API admit handlers (`POST /api/jobs/mint`, `POST /api/jobs/send`) require the `Idempotency-Key` request header (`read_idempotency_key`). A browser sending that header triggers a CORS preflight (OPTIONS), but the router's `CorsLayer` only allowed `Content-Type` in `Access-Control-Allow-Headers`. The preflight therefore failed and the web frontend could not mint or send. Add `idempotency-key` to the CORS `allow_headers` list so the preflight succeeds. `HeaderName::from_static` requires the lowercase form. Cover the fix with a CORS preflight test (`OPTIONS /api/jobs/mint` with `Access-Control-Request-Headers: idempotency-key`) asserting the response echoes both `idempotency-key` and `content-type` in `Access-Control-Allow-Headers`.
…#195) * feat(jobs): expose account_state_hash + output_coins_root on awaiting_signature job result A pure-TypeScript wallet must know account_state_hash (ash) and output_coins_root (ocr) to sign the send commitment, but until now the awaiting_signature JobStatus carried only proof_id. The hashes were reachable solely via GET /api/proof/{id} as a binary bincode CoinProof blob that only Rust/wasm can decode — breaking the thin-client rule (wallet = key only, trusts the node, no heavy client-side logic). This change writes ash + ocr as lowercase hex into the job result when a send job transitions to awaiting_signature, so GET /api/jobs/:id and the SSE stream surface them under result.account_state_hash / result.output_coins_root — the exact keys @zkcoins/sdk's pay() reads. ash/ocr come from the same source the completed mint/commit results use: ProofData::from_field_elements over the send proof's public inputs, hex-encoded via digest_to_bytes. Extraction is factored into a shared flow::send_commit_hashes helper that mint_flow, send_flow, and commit_flow all call, so the hex is bit-identical to what createCommitment expects and commit_flow re-derives. Purely additive: completed result shape, proof_id top-level field, and the JobStatusResponse wire schema (result is already free-form JSON) are unchanged. No new endpoint, env var, or migration. set_awaiting_signature stores the result in the existing response_body column (the terminal complete body overwrites it later); the GET handler and SSE initial frame now surface result for awaiting_signature in addition to completed, and a post-restart resume re-publishes the persisted hashes. Tests: api_remote send roundtrip asserts the awaiting_signature result hex equals the proof-decoded ash/ocr; job_store + router unit tests cover the new persistence and snapshot paths (100% line + function gate green). * docs(jobs): correct response_body field comment for awaiting_signature
Add a typed, lowercase string enum field `bitcoin_network` to the /api/info response with exactly two variants: "mainnet" and "mutinynet". The value is derived from the existing NETWORK_CONFIG.is_mainnet flag via a pure, unit-testable helper (bitcoin_network_label) — no new env var, no new config source. The free-text `network` field (e.g. "Mainnet"/"Mutinynet" from NETWORK_CONFIG.network_name) is retained unchanged for backward compatibility; bitcoin_network is additive. This fixes the latent case-mismatch foot-gun documented for the wallet/SDK, which should switch behaviour on the typed identifier rather than matching the operator-overridable free-text label. Register BitcoinNetwork as a ToSchema component in the OpenAPI spec. Cover both helper arms with a unit test, assert the field in the existing /api/info handler tests, add OpenAPI smoke drift guards, and add a no-fallback contract assertion in the api_remote E2E suite.
* feat(state): self-heal persisted proofs on circuit change
The Plonky2 state-transition circuit is cyclic: every proof is fed back
as the recursive inner proof on the next transition. When a circuit
change breaks recursion, persisted account proofs become incompatible
and the next mint/send aborts witness generation with a "Partition ...
was set twice with different values" copy-constraint conflict, surfaced
to the wallet as "prove failed". This took DEV down and required a
manual reset-zkcoins-node.
Add a boot-time self-heal that detects the incompatibility and resets
the proof-dependent state to genesis (the documented tabula rasa,
permitted in the closed test env), storing the live circuit digest so
subsequent boots are an O(1) comparison.
Detector. Two stages: (1) compare the persisted circuit_digest against
the live one — the cheap steady-state fast path; (2) on the adoption
boundary (no digest recorded yet) run a canary recursion: recurse a
persisted proof through the live circuit's AccountUpdate branch with the
real commitment-merkle witnesses from the loaded state. Stale ⇒ reset.
Why the canary and not Prover::verify / a digest comparison alone:
verified against the live DEV dump, the breakage does NOT change the
verifier-key circuit_digest. Plonky2's circuit_digest hashes the
constants/sigmas cap + domain separator + degree but NOT the gate
constraints (upstream circuit_builder.rs "TODO: This should also include
an encoding of gate constraints"). The DEV proofs' embedded digest was
byte-identical to the current build's and Prover::verify passed on them,
yet the recursive prove still failed. Only running the real recursion
reproduces the failure.
The digest is deterministic across separate builds of identical circuit
code (no nonce/timestamp), so a digest CHANGE still reliably signals a
circuit change — it just has a blind spot for constraint-only changes
that the canary closes.
- migration 0015: singleton circuit_digest_meta table
- db: load/store circuit digest + transactional proof-dependent reset
- account_node: CanaryOutcome + canary_recursion (real AccountUpdate
recursion probe) + take_prover for the post-reset reload
- self_heal: reset_decision (pure, exhaustively unit-tested) +
heal_circuit_digest orchestrator
- main: build prover once, load state+accounts, heal, reload from
genesis on reset (prover reused — circuit built once)
* fix(state): use real account state in self-heal canary to avoid false-positive reset
The boot self-heal canary recursed a persisted proof through the live
circuit's AccountUpdate branch with a SYNTHETIC surrounding AccountState
({ owner: ZERO_HASH, balance: 0 }). That state violates the §8(b)/(c)
state-continuity constraints, so the canary returning Ok relied on the
fragile Plonky2 invariant that arithmetic gate constraints are not
evaluated at witness/prove time. Worse, there was no proof the canary
returns Compatible (not a false Stale -> genesis wipe -> production data
loss) on a genuinely compatible but digest-less DB — the first-boot case
of every existing node adopting this fix when no breaking change occurred.
Rebuild the REAL account state, exactly as the production prove path
(account_state_for_prove): owner = account address, balance =
account.balance, public_key = the account's CURRENT key. The current key
is NOT the persisted commitment_public_key: the circuit commits
ProofData.account_state_hash as final_account_state_hash, which embeds
the producing transition's next_public_key (the key it rotated TO). By
the rotation chain that equals the next transition's public_key; for the
minting account it is generate_public_key(derive_num_pubkeys_from_smt()).
commitment_public_key is still used, but only to look the commitment up
in the SMT via get_merkle_proofs (mirroring send_coins_inner's prev_cmp).
The boot path supplies the current-key resolver, reconstructed from the
same compile-time minting secret the node already uses and resolved off
the SMT the canary already holds (the resolver MUST NOT re-lock state —
the canary holds it, and a re-lock deadlocks the non-reentrant guard).
With the real state both §8(b)/(c) are satisfiable for a compatible
proof, so the only remaining prove-time failure path is the recursion
copy-constraint set_proof_with_pis imposes on the inner proof — exactly
what a breaking circuit change violates. Err => Stale no longer depends
on which constraints Plonky2 evaluates at prove time. Prover::verify
stays out of the detector.
Verified by a live boot-gate in BOTH directions: a stale DEV dump still
resets to genesis (Canary Stale -> Reset, post-reset mint completes), and
a genuinely compatible digest-less DB now baselines without wiping
(Canary Compatible -> Baseline, accounts preserved, mint completes).
Also:
- canary_recursion: document the append-only proof-data PI-slot
assumption (a future circuit change reordering the first
N_PROOF_DATA_PUBLIC_INPUTS slots would make get_merkle_proofs Err for
every sample -> NoSample -> Baseline -> no reset despite staleness, a
False Negative). Emit a tracing::warn when proof-carrying accounts
exist but all are skipped. NoSample stays Baseline (the data-loss-safe
direction for benign state gaps), not Stale, by design.
- reset_proof_dependent_state_tx: state the exact wipe set, note
usernames is intentionally preserved (not proof-dependent), and note
coin_proof_store (migration 0008) is unused schema groundwork with a
MIGRATION_RESEARCH note to add it to the reset if the DB-backed
ProofStore bootstrap later lands.
Promote: staging -> develop
…205) * ci: collapse test gating to 2 tiers (lint&build default, ci:full = full gate) Replace the 3-tier test-gating model with a clean 2-tier model: - Tier 1 `Lint & Build` (GitHub-hosted) — the default; runs on every non-draft PR and every push, no label required. - Tier 2 `Tests + Coverage Gate (M3 Ultra)` — opt-in via the `ci:full` label; the full node + shared nextest suite under llvm-cov including the Postgres db_tests, the Plonky2 prover flows, and the 100% line + function coverage gate, in one job. Changes: - Remove the `DB Subset Tests` and `Prover Subset Tests` jobs, the `ci:db` / `ci:prover` labels, and the `!contains(... 'ci:full')` mutual-exclusion clauses entirely. The heavy gate is a strict superset of both subsets, so they only added filter-drift maintenance burden without extending coverage. - Rewrite the ci.yaml header / inline comments to the 2-tier model. - Keep `.config/nextest.toml` (jobs-endpoint max-threads=2 from #196); update its comment to note the cap now guards the full gate generally rather than the removed DB Subset job. - Auto-promote: label the staging -> develop Promote PR with `ci:full` automatically (mirrors the develop -> main Release PR), so every promotion is validated against the full gate. Adds `issues: write` for `gh label create`. - CONTRIBUTING.md: rewrite all 3-tier / subset references to 2-tier. * ci: scrub internal runner hostname from ci.yaml comments The Tier-2 comment block named the internal CI host (dfx01) and its agent/core topology. This repo is public; replace with neutral 'shared self-hosted M3 Ultra runner pool' wording. No workflow logic changes.
…206) * docs: scrub internal infrastructure references from the public repo Replace internal host names, agent names, and host topology details (dfx01/dfxdev/dfxprd/dfxai and concrete runner-agent names) with neutral placeholders across docs and source comments. Public facts are kept intentionally: the Mac Studio M3 Ultra / 96 GB hardware target, the m3-ultra runner label, R2 probe perf numbers, the zk-coins/server -> zk-coins/node RUNNER_DIR names, and the *.zkcoins.local test fixtures. The node/src/*.rs changes are comment-only; runtime behaviour is unchanged. .github/workflows/ci.yaml is intentionally untouched here (scrubbed separately in PR #205). * docs: remove internal Kuma monitor URL and align runner pool count Review caught a remaining internal infra hostname: router.rs named the internal Uptime-Kuma URL (kuma.dfxserve.com) in a doc comment. Replace with a neutral 'external uptime monitor (Uptime-Kuma)' reference. Also restore the concrete '6 agents' pool count in the ci-runner README for consistency with the rest of the doc (the count is not an identifier; only the host-derived agent names were scrubbed).
Promote: staging -> develop
TaprootFreak
marked this pull request as ready for review
June 4, 2026 20:30
…overy) (#208) * Promote: staging -> develop (#185) * perf(tests): shared Postgres container + per-test schema (Issue #181 Opt B) (#182) * perf(tests): shared Postgres container + per-test schema (issue #181 Opt B) Replaces the per-test `Postgres::default().start()` model with a single shared Postgres container that every test process attaches to via testcontainers' `with_reuse(ReuseDirective::Always)` and a stable container name (`zkcoins-test-shared-pg`). Each test still gets a fully isolated state via a UUID-named schema with `search_path` pinned to it; migrations are run per-schema. The reuse flag is load-bearing: `cargo nextest` defaults to one process per test, so a process-local `OnceCell<Postgres>` does not actually share state across tests — it degrades to one container per test. Verified on a local M5 Max (OrbStack): 6 db-tests finish in 1.5 s with exactly 1 container running, vs. ~24 s with 6 containers under the old per-process model. CI gains three `docker rm -f zkcoins-test-shared-pg` cleanup steps (one per test job, always-on) so the shared container does not leak across PR runs on the self-hosted runner. Coverage gate's `--ignore-filename-regex` is extended to skip `test_db.rs` — the new `#[cfg(test)]`-only test-infra module would otherwise drag its Drop-future uncovered lines into the 100% gate. `db_tests::connect_and_migrate_creates_all_tables` is rewritten to route through the real `db::connect_and_migrate` (via the `?options=-c search_path=<schema>` URL trick) so the success-path of that function stays covered. Expected wall on the M3 Ultra runner per issue #181: 47 min → ~37 min at `--test-threads=1` (Optimisation A — flipping the test isolation to multi-thread — is a follow-up that depends on this landing first; see #181 Recommendation section). Test files migrated to the shared helper: db_tests, state_tests, r2_probe_tests, username_tests, main_tests, runtime_tests, router_tests (incl. the jobs_test_state factory from #161), job_store_tests, account_node_tests, audit_tests, publisher_tests. * test(db): include jobs table in connect_and_migrate assertion Migration 0014 (introduced by #161, async Job-API) adds the jobs table to the production schema. The rebase of #182 onto staging left the hard-coded expected-tables list in connect_and_migrate_creates_all_tables unchanged, so the assertion sees an extra row ("jobs") it does not expect and fails fast under nextest's default fail-fast mode — masking the rest of the suite. Adds "jobs" at its alphabetic position and bumps the migration range in the comment from 0001-0013 to 0001-0014. * perf(tests): enable parallel execution (--test-threads=8) (#181 Opt A) (#183) With per-test schema isolation + shared-container reuse from #182, the suite is parallel-safe. This PR: - Flips `--test-threads=1` to `--test-threads=8` across the 3 CI test jobs (db-tests, prover-tests, test-and-coverage) and the matching CONTRIBUTING.md references. - Adds a `fs2` cross-process file lock around `init_shared_pg` in test_db.rs. testcontainers 0.27 does NOT atomicise its attach-or-create path: 8 concurrent nextest processes all see "container not present", all POST /containers/create, 1 wins and 7 fail with Docker 409 Conflict. The lock serialises the attach-or-create call; the container creation cost (~3 s once) amortises across the whole test run. - runtime_tests.rs: env mutation consolidated behind a `OnceLock`-backed `ensure_test_env()` so concurrent callers do not race on process-wide env. `PROOFS_DIR` removed from env entirely and passed as a parameter on `start_rest_node` (main.rs reads the env at the binary edge). - router_tests.rs: 2 hard-coded `/tmp/zkcoins-*-proofs` paths replaced with `tempfile::tempdir().keep()` so each parallel test gets a unique ProofStore directory and `next_id` cannot race. Empirical on an Apple M5 Max workstation (OrbStack): a wide DB + state + router + username + audit subset of 146 tests passes under --test-threads=8 in 183 s wall (CPU 1325 %, exactly one shared postgres:17 container live during the run). Expected on the M3 Ultra runner per #181: 47 min (pre-Opt-B) -> ~44 min (after #182, measured) -> ~10-12 min (after this PR). --------- Co-authored-by: TaprootFreak <142087526+TaprootFreak@users.noreply.github.com> * fix(db): reset proof-dependent state to genesis (DEV + PRD) DEV's mint prover started failing 100% with "prove failed" on 2026-06-05 with no deploy and an unchanged circuit_digest: persisted account proofs stopped recursing through the live circuit (the constraint-only / digest-unchanged staleness class that migration 0015 documents as detectable only by the canary, which the steady-state self-heal Keep-path does not run). This migration is the recovery for the already-stale state. Wipes the same proof-dependent table set as db::reset_proof_dependent_state_tx — accounts, smt_state, mmr_state, mmr_root_index, latest_block — plus the circuit_digest_meta singleton. Clearing the digest row (rather than rewriting it; SQL cannot compute the live circuit digest) puts the DB in the fresh-genesis shape the boot path already handles: no persisted digest -> canary on the now- empty accounts -> NoSample -> Baseline records the live digest. No new code path, reuses the integration-tested self_heal flow. usernames / append-only history / jobs / coin_proof_store are preserved exactly as the existing reset does. On-disk proof files are left as inert orphans (ProofStore::new resumes next_id at max_id+1 so ids never collide; the Jobs-API no longer writes the file store). Closed test env, no data to preserve, PRD genesis wipe explicitly authorized (CONTRIBUTING "Closed test environment"). sqlx applies it once per database: develop -> DEV, main -> PRD. Validated against postgres:17: full 0001..0016 chain applies clean, the six tables empty, usernames/history intact, re-apply is a no-op. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…boot self-heal arming (#209) * Promote: staging -> develop (#185) * perf(tests): shared Postgres container + per-test schema (Issue #181 Opt B) (#182) * perf(tests): shared Postgres container + per-test schema (issue #181 Opt B) Replaces the per-test `Postgres::default().start()` model with a single shared Postgres container that every test process attaches to via testcontainers' `with_reuse(ReuseDirective::Always)` and a stable container name (`zkcoins-test-shared-pg`). Each test still gets a fully isolated state via a UUID-named schema with `search_path` pinned to it; migrations are run per-schema. The reuse flag is load-bearing: `cargo nextest` defaults to one process per test, so a process-local `OnceCell<Postgres>` does not actually share state across tests — it degrades to one container per test. Verified on a local M5 Max (OrbStack): 6 db-tests finish in 1.5 s with exactly 1 container running, vs. ~24 s with 6 containers under the old per-process model. CI gains three `docker rm -f zkcoins-test-shared-pg` cleanup steps (one per test job, always-on) so the shared container does not leak across PR runs on the self-hosted runner. Coverage gate's `--ignore-filename-regex` is extended to skip `test_db.rs` — the new `#[cfg(test)]`-only test-infra module would otherwise drag its Drop-future uncovered lines into the 100% gate. `db_tests::connect_and_migrate_creates_all_tables` is rewritten to route through the real `db::connect_and_migrate` (via the `?options=-c search_path=<schema>` URL trick) so the success-path of that function stays covered. Expected wall on the M3 Ultra runner per issue #181: 47 min → ~37 min at `--test-threads=1` (Optimisation A — flipping the test isolation to multi-thread — is a follow-up that depends on this landing first; see #181 Recommendation section). Test files migrated to the shared helper: db_tests, state_tests, r2_probe_tests, username_tests, main_tests, runtime_tests, router_tests (incl. the jobs_test_state factory from #161), job_store_tests, account_node_tests, audit_tests, publisher_tests. * test(db): include jobs table in connect_and_migrate assertion Migration 0014 (introduced by #161, async Job-API) adds the jobs table to the production schema. The rebase of #182 onto staging left the hard-coded expected-tables list in connect_and_migrate_creates_all_tables unchanged, so the assertion sees an extra row ("jobs") it does not expect and fails fast under nextest's default fail-fast mode — masking the rest of the suite. Adds "jobs" at its alphabetic position and bumps the migration range in the comment from 0001-0013 to 0001-0014. * perf(tests): enable parallel execution (--test-threads=8) (#181 Opt A) (#183) With per-test schema isolation + shared-container reuse from #182, the suite is parallel-safe. This PR: - Flips `--test-threads=1` to `--test-threads=8` across the 3 CI test jobs (db-tests, prover-tests, test-and-coverage) and the matching CONTRIBUTING.md references. - Adds a `fs2` cross-process file lock around `init_shared_pg` in test_db.rs. testcontainers 0.27 does NOT atomicise its attach-or-create path: 8 concurrent nextest processes all see "container not present", all POST /containers/create, 1 wins and 7 fail with Docker 409 Conflict. The lock serialises the attach-or-create call; the container creation cost (~3 s once) amortises across the whole test run. - runtime_tests.rs: env mutation consolidated behind a `OnceLock`-backed `ensure_test_env()` so concurrent callers do not race on process-wide env. `PROOFS_DIR` removed from env entirely and passed as a parameter on `start_rest_node` (main.rs reads the env at the binary edge). - router_tests.rs: 2 hard-coded `/tmp/zkcoins-*-proofs` paths replaced with `tempfile::tempdir().keep()` so each parallel test gets a unique ProofStore directory and `next_id` cannot race. Empirical on an Apple M5 Max workstation (OrbStack): a wide DB + state + router + username + audit subset of 146 tests passes under --test-threads=8 in 183 s wall (CPU 1325 %, exactly one shared postgres:17 container live during the run). Expected on the M3 Ultra runner per #181: 47 min (pre-Opt-B) -> ~44 min (after #182, measured) -> ~10-12 min (after this PR). --------- Co-authored-by: TaprootFreak <142087526+TaprootFreak@users.noreply.github.com> * fix(prover): detect systemic prove failures — health signal + self-heal arming The 2026-06-05 DEV outage exposed two gaps around the digest-unchanged proof-staleness class that migration 0015 documents: 1. /health/ready lied. Its prover tag only reflected the one-shot boot warmup flag, so a node failing 100% of mint jobs with "prove failed" kept reporting prover: ready for ~100 minutes — invisible to the deploy smoke-test, Kuma, and any orchestration keyed on readiness. 2. The boot self-heal never re-checks in steady state. reset_decision consults the canary recursion only on the no-persisted-digest adoption branch; with a persisted digest equal to the live one it takes the Keep fast path. Constraint-only circuit changes (and any other event that stops persisted proofs from recursing while the digest stays byte-identical) therefore brick the node permanently — no restart heals it. New prover_health module: the job dispatcher counts CONSECUTIVE "prove failed" outcomes (the collapsed message is matched exactly, so request-level errors never move the streak; any successful prove resets it). At PROVE_FAILURE_THRESHOLD consecutive failures: * /health/ready reports prover: failing + 503 for the duration of the streak (gap 1) — the outage is now visible and gates traffic. * the dispatcher clears the persisted circuit digest via the new db::clear_circuit_digest (gap 2). This only ARMS the boot self-heal: the next restart finds no persisted digest, runs the canary recursion, and resets to genesis IFF the canary confirms the persisted proofs are stale — Compatible/NoSample just re-record the baseline, so a transient prover blip that is over by the restart causes no reset and no data loss. The destructive reset stays gated behind the authoritative canary; nothing is wiped at runtime. The steady-state boot keeps its O(1) digest comparison (the ~5 s canary still never runs on a healthy boot); the arming path is the only way a matching-digest boot reaches the canary. Coverage: prover_health is unit-tested exhaustively (threshold boundary, one-shot arming, streak reset); clear_circuit_digest gets a testcontainer round-trip incl. idempotent re-clear; the new ready-handler branch is driven by a prover-failing readiness test (503 + prover: failing). job_dispatcher wiring sits in the coverage-exempt dispatcher. fmt + the CI clippy commands (-D warnings, MVP + all-features) are clean locally; check --tests green. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(db): reset proof-dependent state to genesis (DEV + PRD prover recovery) (#208) * Promote: staging -> develop (#185) * perf(tests): shared Postgres container + per-test schema (Issue #181 Opt B) (#182) * perf(tests): shared Postgres container + per-test schema (issue #181 Opt B) Replaces the per-test `Postgres::default().start()` model with a single shared Postgres container that every test process attaches to via testcontainers' `with_reuse(ReuseDirective::Always)` and a stable container name (`zkcoins-test-shared-pg`). Each test still gets a fully isolated state via a UUID-named schema with `search_path` pinned to it; migrations are run per-schema. The reuse flag is load-bearing: `cargo nextest` defaults to one process per test, so a process-local `OnceCell<Postgres>` does not actually share state across tests — it degrades to one container per test. Verified on a local M5 Max (OrbStack): 6 db-tests finish in 1.5 s with exactly 1 container running, vs. ~24 s with 6 containers under the old per-process model. CI gains three `docker rm -f zkcoins-test-shared-pg` cleanup steps (one per test job, always-on) so the shared container does not leak across PR runs on the self-hosted runner. Coverage gate's `--ignore-filename-regex` is extended to skip `test_db.rs` — the new `#[cfg(test)]`-only test-infra module would otherwise drag its Drop-future uncovered lines into the 100% gate. `db_tests::connect_and_migrate_creates_all_tables` is rewritten to route through the real `db::connect_and_migrate` (via the `?options=-c search_path=<schema>` URL trick) so the success-path of that function stays covered. Expected wall on the M3 Ultra runner per issue #181: 47 min → ~37 min at `--test-threads=1` (Optimisation A — flipping the test isolation to multi-thread — is a follow-up that depends on this landing first; see #181 Recommendation section). Test files migrated to the shared helper: db_tests, state_tests, r2_probe_tests, username_tests, main_tests, runtime_tests, router_tests (incl. the jobs_test_state factory from #161), job_store_tests, account_node_tests, audit_tests, publisher_tests. * test(db): include jobs table in connect_and_migrate assertion Migration 0014 (introduced by #161, async Job-API) adds the jobs table to the production schema. The rebase of #182 onto staging left the hard-coded expected-tables list in connect_and_migrate_creates_all_tables unchanged, so the assertion sees an extra row ("jobs") it does not expect and fails fast under nextest's default fail-fast mode — masking the rest of the suite. Adds "jobs" at its alphabetic position and bumps the migration range in the comment from 0001-0013 to 0001-0014. * perf(tests): enable parallel execution (--test-threads=8) (#181 Opt A) (#183) With per-test schema isolation + shared-container reuse from #182, the suite is parallel-safe. This PR: - Flips `--test-threads=1` to `--test-threads=8` across the 3 CI test jobs (db-tests, prover-tests, test-and-coverage) and the matching CONTRIBUTING.md references. - Adds a `fs2` cross-process file lock around `init_shared_pg` in test_db.rs. testcontainers 0.27 does NOT atomicise its attach-or-create path: 8 concurrent nextest processes all see "container not present", all POST /containers/create, 1 wins and 7 fail with Docker 409 Conflict. The lock serialises the attach-or-create call; the container creation cost (~3 s once) amortises across the whole test run. - runtime_tests.rs: env mutation consolidated behind a `OnceLock`-backed `ensure_test_env()` so concurrent callers do not race on process-wide env. `PROOFS_DIR` removed from env entirely and passed as a parameter on `start_rest_node` (main.rs reads the env at the binary edge). - router_tests.rs: 2 hard-coded `/tmp/zkcoins-*-proofs` paths replaced with `tempfile::tempdir().keep()` so each parallel test gets a unique ProofStore directory and `next_id` cannot race. Empirical on an Apple M5 Max workstation (OrbStack): a wide DB + state + router + username + audit subset of 146 tests passes under --test-threads=8 in 183 s wall (CPU 1325 %, exactly one shared postgres:17 container live during the run). Expected on the M3 Ultra runner per #181: 47 min (pre-Opt-B) -> ~44 min (after #182, measured) -> ~10-12 min (after this PR). --------- Co-authored-by: TaprootFreak <142087526+TaprootFreak@users.noreply.github.com> * fix(db): reset proof-dependent state to genesis (DEV + PRD) DEV's mint prover started failing 100% with "prove failed" on 2026-06-05 with no deploy and an unchanged circuit_digest: persisted account proofs stopped recursing through the live circuit (the constraint-only / digest-unchanged staleness class that migration 0015 documents as detectable only by the canary, which the steady-state self-heal Keep-path does not run). This migration is the recovery for the already-stale state. Wipes the same proof-dependent table set as db::reset_proof_dependent_state_tx — accounts, smt_state, mmr_state, mmr_root_index, latest_block — plus the circuit_digest_meta singleton. Clearing the digest row (rather than rewriting it; SQL cannot compute the live circuit digest) puts the DB in the fresh-genesis shape the boot path already handles: no persisted digest -> canary on the now- empty accounts -> NoSample -> Baseline records the live digest. No new code path, reuses the integration-tested self_heal flow. usernames / append-only history / jobs / coin_proof_store are preserved exactly as the existing reset does. On-disk proof files are left as inert orphans (ProofStore::new resumes next_id at max_id+1 so ids never collide; the Jobs-API no longer writes the file store). Closed test env, no data to preserve, PRD genesis wipe explicitly authorized (CONTRIBUTING "Closed test environment"). sqlx applies it once per database: develop -> DEV, main -> PRD. Validated against postgres:17: full 0001..0016 chain applies clean, the six tables empty, usernames/history intact, re-apply is a no-op. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix(prover): detect systemic prove failures — /health/ready signal + boot self-heal arming (#209) * Promote: staging -> develop (#185) * perf(tests): shared Postgres container + per-test schema (Issue #181 Opt B) (#182) * perf(tests): shared Postgres container + per-test schema (issue #181 Opt B) Replaces the per-test `Postgres::default().start()` model with a single shared Postgres container that every test process attaches to via testcontainers' `with_reuse(ReuseDirective::Always)` and a stable container name (`zkcoins-test-shared-pg`). Each test still gets a fully isolated state via a UUID-named schema with `search_path` pinned to it; migrations are run per-schema. The reuse flag is load-bearing: `cargo nextest` defaults to one process per test, so a process-local `OnceCell<Postgres>` does not actually share state across tests — it degrades to one container per test. Verified on a local M5 Max (OrbStack): 6 db-tests finish in 1.5 s with exactly 1 container running, vs. ~24 s with 6 containers under the old per-process model. CI gains three `docker rm -f zkcoins-test-shared-pg` cleanup steps (one per test job, always-on) so the shared container does not leak across PR runs on the self-hosted runner. Coverage gate's `--ignore-filename-regex` is extended to skip `test_db.rs` — the new `#[cfg(test)]`-only test-infra module would otherwise drag its Drop-future uncovered lines into the 100% gate. `db_tests::connect_and_migrate_creates_all_tables` is rewritten to route through the real `db::connect_and_migrate` (via the `?options=-c search_path=<schema>` URL trick) so the success-path of that function stays covered. Expected wall on the M3 Ultra runner per issue #181: 47 min → ~37 min at `--test-threads=1` (Optimisation A — flipping the test isolation to multi-thread — is a follow-up that depends on this landing first; see #181 Recommendation section). Test files migrated to the shared helper: db_tests, state_tests, r2_probe_tests, username_tests, main_tests, runtime_tests, router_tests (incl. the jobs_test_state factory from #161), job_store_tests, account_node_tests, audit_tests, publisher_tests. * test(db): include jobs table in connect_and_migrate assertion Migration 0014 (introduced by #161, async Job-API) adds the jobs table to the production schema. The rebase of #182 onto staging left the hard-coded expected-tables list in connect_and_migrate_creates_all_tables unchanged, so the assertion sees an extra row ("jobs") it does not expect and fails fast under nextest's default fail-fast mode — masking the rest of the suite. Adds "jobs" at its alphabetic position and bumps the migration range in the comment from 0001-0013 to 0001-0014. * perf(tests): enable parallel execution (--test-threads=8) (#181 Opt A) (#183) With per-test schema isolation + shared-container reuse from #182, the suite is parallel-safe. This PR: - Flips `--test-threads=1` to `--test-threads=8` across the 3 CI test jobs (db-tests, prover-tests, test-and-coverage) and the matching CONTRIBUTING.md references. - Adds a `fs2` cross-process file lock around `init_shared_pg` in test_db.rs. testcontainers 0.27 does NOT atomicise its attach-or-create path: 8 concurrent nextest processes all see "container not present", all POST /containers/create, 1 wins and 7 fail with Docker 409 Conflict. The lock serialises the attach-or-create call; the container creation cost (~3 s once) amortises across the whole test run. - runtime_tests.rs: env mutation consolidated behind a `OnceLock`-backed `ensure_test_env()` so concurrent callers do not race on process-wide env. `PROOFS_DIR` removed from env entirely and passed as a parameter on `start_rest_node` (main.rs reads the env at the binary edge). - router_tests.rs: 2 hard-coded `/tmp/zkcoins-*-proofs` paths replaced with `tempfile::tempdir().keep()` so each parallel test gets a unique ProofStore directory and `next_id` cannot race. Empirical on an Apple M5 Max workstation (OrbStack): a wide DB + state + router + username + audit subset of 146 tests passes under --test-threads=8 in 183 s wall (CPU 1325 %, exactly one shared postgres:17 container live during the run). Expected on the M3 Ultra runner per #181: 47 min (pre-Opt-B) -> ~44 min (after #182, measured) -> ~10-12 min (after this PR). --------- Co-authored-by: TaprootFreak <142087526+TaprootFreak@users.noreply.github.com> * fix(prover): detect systemic prove failures — health signal + self-heal arming The 2026-06-05 DEV outage exposed two gaps around the digest-unchanged proof-staleness class that migration 0015 documents: 1. /health/ready lied. Its prover tag only reflected the one-shot boot warmup flag, so a node failing 100% of mint jobs with "prove failed" kept reporting prover: ready for ~100 minutes — invisible to the deploy smoke-test, Kuma, and any orchestration keyed on readiness. 2. The boot self-heal never re-checks in steady state. reset_decision consults the canary recursion only on the no-persisted-digest adoption branch; with a persisted digest equal to the live one it takes the Keep fast path. Constraint-only circuit changes (and any other event that stops persisted proofs from recursing while the digest stays byte-identical) therefore brick the node permanently — no restart heals it. New prover_health module: the job dispatcher counts CONSECUTIVE "prove failed" outcomes (the collapsed message is matched exactly, so request-level errors never move the streak; any successful prove resets it). At PROVE_FAILURE_THRESHOLD consecutive failures: * /health/ready reports prover: failing + 503 for the duration of the streak (gap 1) — the outage is now visible and gates traffic. * the dispatcher clears the persisted circuit digest via the new db::clear_circuit_digest (gap 2). This only ARMS the boot self-heal: the next restart finds no persisted digest, runs the canary recursion, and resets to genesis IFF the canary confirms the persisted proofs are stale — Compatible/NoSample just re-record the baseline, so a transient prover blip that is over by the restart causes no reset and no data loss. The destructive reset stays gated behind the authoritative canary; nothing is wiped at runtime. The steady-state boot keeps its O(1) digest comparison (the ~5 s canary still never runs on a healthy boot); the arming path is the only way a matching-digest boot reaches the canary. Coverage: prover_health is unit-tested exhaustively (threshold boundary, one-shot arming, streak reset); clear_circuit_digest gets a testcontainer round-trip incl. idempotent re-clear; the new ready-handler branch is driven by a prover-failing readiness test (503 + prover: failing). job_dispatcher wiring sits in the coverage-exempt dispatcher. fmt + the CI clippy commands (-D warnings, MVP + all-features) are clean locally; check --tests green. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: TaprootFreak <142087526+TaprootFreak@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…211) * docs: add Plonky3 migration plan with phased, executable task spec Adds MIGRATION_PLONKY3.md: a phase-by-phase, execution-ready work plan for the Plonky2 -> Plonky3 backend swap. Phase 0 is a hard recursion feasibility gate against p3-recursion (probed in Goldilocks to isolate the recursion/API risk from the field-migration risk). Subsequent phases port types/hash, Merkle gadgets, the state-transition circuit, recursion + aggregator, node integration, and parity/coverage/bench, each with exact files, acceptance criteria, and local verification commands. Field swap to KoalaBear/BabyBear is sequenced as a separate optional follow-up. Companion to ROADMAP.md and MIGRATION_RESEARCH.md. * docs: record Phase 0 result + defer cross-layer PI-threading choice to Phase-1-authorize The Phase 0 spike returned GO with one escalated finding: the high-level batch recursion does not propagate public inputs across layers (probe_d_multilayer_carry: air_public_targets = [0,0,0]), unlike Plonky2 cyclic recursion. This is a construction problem, not a p3-recursion capability gap, so it does not flip the gate to NO-GO. Record this in the Phase 0 gate section, and mark the cross-layer PI-threading construction as TBD — the choice between Option 1 (threaded outputs as AIR public values, fast), Option 2 (commit + Merkle/hash re-bind each layer, sound), and Option 3 (pinned probe catches a future upstream rev that propagates natively) is made at Phase-1-authorize time, not now. P5-T1 is written against the chosen option and its acceptance now requires the threaded prev_account value to be carried across transitions. * docs: resolve cross-layer threading to Option 2 + record Phase-5 budget risk Phase 0 closed the open threading question empirically, so the plan no longer defers it to Phase-1-authorize: - Cross-layer PI threading is RESOLVED to Option 2 (commit + hash/Merkle re-bind each layer). Option 1 (carry the value as an AIR public value) is dead — proven by probe_h_option1_air_public_values (injecting a non-existent public input is rejected) and probe_g_fanin_pi_passthrough (a real aggregation surfaces 0 per-leaf values to the outer); CircuitBuilder public inputs live in the committed Public table, never as AIR public values. Reflected in the §5 recorded result, the §6 Phase-1-authorize block (now "RESOLVED: Option 2"), P5-T1, and P5-T2 (per-leaf ProofData also needs Option-2 commit+re-bind, then §7.17 masking). - Record the Phase-5 warm-prove budget risk from probe_i_cost_projection: a recursion layer over a ~2^16-gate inner proof is ~3.2 s / ~1.4 GB — a single-layer lower bound on an arithmetic toy; the mandatory Option-2 re-bind adds Poseidon gates per layer, and the real constraints are Poseidon-heavy. Measure the real circuit + Option-2 at the START of Phase 5; a >5 s warm-prove is a NO-GO trigger. Option 3 (pinned probes catch a future upstream rev that restores native propagation) stays armed. * docs: gate is NO-GO — neither Option 1 nor Option 2 threads state across a batch layer Phase 0 closed the cross-layer threading question definitively: Option 1 (AIR public values) is dead (probe_h/g/d_multilayer_carry) and Option 2 (commit + hash re-bind) is also dead (probe_j + adversarial review — layer N+1 cannot read layer N's committed digest). There is no per-instance value channel across a batch-recursion layer, so zkCoins' prev_account/ProofData IVC carry is structurally unbuildable on this rev. Record NO-GO in the §5 result, the §6 Phase-1-authorize block ("DO NOT START Phase 1"; §6-§14 kept as a would-be plan for if an escape route opens), P5-T1 (BLOCKED), P5-T2 (BLOCKED), and the §10 budget note (moot under NO-GO). Escape routes: an upstream rev that exposes cross-layer public inputs (pinned probes auto-detect), a protocol redesign, or a fork (excluded by §16). Decision is the operator's per §16. * docs(plonky3): gate is GO via Path 1+5 (carrier tables) — overturn NO-GO, unblock Phases 4-5 Phase 0 reassessment: probe_q + probe_r_carrier_chain prove a custom AIR's public value (carrier table) threads state across a batch layer end-to-end (V_3==V_0+3, sound negatives, no fork). probe_r_cost: within ≤5s warm budget. Update P0-T6 memo block, §6 Phase-0 outcome, and P5-T1/P5-T2/budget-note to the carrier-table direction. Rationale: MIGRATION_PLONKY3_SOLUTIONS_RESEARCH.md; proof: PR #214.
…ier tables) (#212) * feat(plonky3): Phase 0 recursion feasibility spike — GO gate Add spikes/plonky3-recursion-spike, an isolated probe (its own workspace, excluded from the root workspace so the heavy Plonky3 git deps never enter the node/shared build or CI) that empirically proves Plonky3/Plonky3-recursion can express the three composition patterns the zkCoins circuit depends on, in Goldilocks, on the pinned revs: - Probe A (IVC/cyclic with base case): 4-layer chain; the verifier-circuit shape reaches a fixed point (true IVC, no growth) — the analogue of Plonky2 common_data_for_recursion. witness_count [25567, 104630, 107957, 107957]. - Probe B (fan-in-8, variable active count): 2-to-1 aggregation composes into a fixed-shape tree (fan-in-4 probed; fan-in-8 is one more level). No native conditional-verify primitive; inactive slots are padded with real proofs and masked downstream via an active bit. - Probe C (vk/PI binding): an inner proof's public inputs are bound in the verifier circuit; a mismatched claim is rejected in-circuit. Record P0-T5 cost (~4.65 s per stabilized layer, ~1 GB peak RSS) and the P0-T6 Go/No-Go memo (MIGRATION_PLONKY3_SPIKE_RESULT.md). Gate decision: GO. Pins: Plonky3-recursion 524665d0c2e1d294722c064786ae11dff8d9f33b, Plonky3 56952503e1401a62982ceaf952c5e4a829b61803. * docs(plonky3): tighten Phase 0 memo + probes to match §5 PASS criteria exactly Address logic-review findings — scope the spike's claims precisely against MIGRATION_PLONKY3.md §5 so the GO gate is not overstated: - Probe A: strengthen the fixed-point assertion (require the shape to have GROWN before stabilising, not just last-two-equal). Document that cross-layer counter PI THREADING (P0-T2 crit. 2) is not exercised — into_recursion_input carries empty table_public_inputs; the enabling primitive is proven in Probe C; explicit threading is Phase-5 work. - Probe B: relabel honestly — the probe proves 2-to-1 fan-in TREE COMPOSITION (4 identical real leaves), NOT variable active count / per-leaf PIs / active-bit masking. The masking strategy (§7.17) is Phase-5 construction on proven primitives. - Probe C: relabel as PUBLIC-INPUT binding (proven), with vk-equality connect-back as Phase-5 construction (the literal "wrong-vk proof" is not fed). Memo gate decision restated: the three mechanisms are proven; the three deferred items are in-repo Phase-5 construction, not upstream gaps. Decision stays GO. All 4 probes still green; fmt + clippy clean. * feat(plonky3): exercise the three §5 PASS items empirically (Probes D/E/F) The Phase-0 gate previously proved the three mechanisms but DEFERRED the three real §5 PASS items to Phase 5. This adds probes that exercise them end-to-end with real proving and positive+negative (+control) assertions: - Probe D (probe_d_pi_threading): cross-layer PI threading binding. An outer circuit verifying an inner uni-stark proof exposes air_public_targets and threads a value (next_start = inner.last + 1) bound to an outer public input; a wrong threaded value is rejected; a control without the bind accepts it. - Probe D part 2 (probe_d_multilayer_carry): the escalated finding. Verifying an inner BATCH proof of a CircuitBuilder circuit exposes NO inner public inputs (air_public_targets = [0,0,0]); the high-level chain does not propagate public inputs across layers (differs from Plonky2 cyclic recursion). Pinned by assert. - Probe E (probe_e_active_masking): variable-active-count masking (§7.17). An 8-slot fixed-shape consumer circuit, batch-stark-proved for real: active+correct with inactive-garbage accepted (masked); active-wrong rejected; flipping a garbage slot's active bit flips the verdict; flipping back re-masks. - Probe F (probe_f_vk_binding): vk-equality connect-back. Two ConstPrepAir instances (k=42/99) have different preprocessed commitments (= vks). A proof from vk_99 (internally valid against vk_99) is rejected SOLELY by the connect to vk_42; a control accepts it unbound. Rewrite MIGRATION_PLONKY3_SPIKE_RESULT.md: the §5 items are now exercised (not deferred); the gate is GO with ONE escalated finding (cross-layer public-input propagation), surfaced for operator judgment per §16. All 8 probes green; fmt + clippy clean. * docs(plonky3): mark probe_a row as P0-T2 crit. 1 (not PI threading) * feat(plonky3): close the cross-layer/cost gaps before Phase 1 (Probes G/H/I) Three integrated probes that resolve the previously-escalated open questions into hard, pre-Phase-1 constraints: - Probe H (probe_h_option1_air_public_values): Option 1 (carry the threaded value as an AIR public value) is DEAD. The honest empty-PI layer builds+proves; injecting a non-empty RecursionInput::BatchStark.table_public_inputs is rejected. Combined with probe_d_multilayer_carry (air_public_targets = [0,0,0]), both Option-1 avenues fail. - Probe G (probe_g_fanin_pi_passthrough): a real 2-to-1 aggregation's per-leaf values are NOT surfaced to the outer (air_public_targets = 0). The integrated fan-in-8 per-leaf-PI passthrough is blocked at the first cross-layer hop. => Together G+H decide the Phase-1-authorize choice: Option 2 (commit + hash re-bind) is MANDATORY for IVC threading AND aggregator per-leaf surfacing. - Probe I (probe_i_cost_projection): recursion-layer cost at real inner-proof scale. Sub-linear scaling; a layer over a ~2^16-gate proof is ~3.2s / ~1.4GB. Combined with the mandatory Option-2 overhead and the base prove, the 5s warm budget is at material risk -> measure on the real circuit early in Phase 5. Rewrite MIGRATION_PLONKY3_SPIKE_RESULT.md: status is now CONDITIONAL GO (Option 2 mandatory; warm-prove budget at material risk). The pinned probes (G/H/multilayer_carry) catch a future upstream rev that restores native public-input propagation. All 11 probes green; fmt + clippy clean. * docs(plonky3): scope Option-1 'impossible' to 'not achievable on this rev' * feat(plonky3): Probe J — Option 2 commit+rebind primitive works, but cannot compose The in-circuit Poseidon2 hash-bind primitive (add_hash_slice + connect) is real and binding: hash(V) binds to hash(V); a mismatched preimage is rejected. So Option 2's per-layer commit+rebind building block is expressible. But it needs layer N+1 to read layer N's committed digest, which is structurally impossible across a batch layer (probe_d_multilayer_carry / probe_g / probe_h: no per-instance value exposed, only whole-trace Merkle commitments). So multi-layer Option-2 threading is NOT achievable — confirming both Option 1 and Option 2 are dead and the cross-layer state IVC is unbuildable on this rev. * docs(plonky3): gate is NO-GO — Option 2 cannot compose across batch layers (Probe J) Probe J + an adversarial review of all six escape routes confirm that neither Option 1 nor Option 2 can thread a value across a batch-recursion layer. The per-layer commit+rebind primitive works (in-circuit Poseidon2 hash-bind), but layer N+1 cannot read layer N's committed digest (batch proofs expose only whole-trace Merkle commitments; FRI openings are FS-random; vk is per-circuit-static; NPO public_values are hardcoded empty with no public registration path). So zkCoins' cross-layer state IVC is structurally unbuildable on this rev. Memo status: CONDITIONAL GO -> NO-GO, with escape routes (upstream feature / protocol redesign / fork-excluded). * feat(plonky3): Probes L/N/O — multi-AIR coexistence, concurrency, soundness - Probe L (probe_l_multi_air): two heterogeneous AIRs (CounterAir state-transition-like + ConstPrepAir aggregator-like) co-verify in one verifier circuit, public inputs kept distinct + individually bound; cross-wiring A's PI to B's value is rejected. - Probe N (probe_n_concurrent): 4 independent prove+recurse+verify workloads on separate threads all succeed; peak RSS ~1.38 GB. Prover is concurrency-safe. - Probe O (probe_o_soundness): soundness spot-check — mismatched FRI private data (a different proof's Merkle paths) is rejected by the in-circuit verification, and a tampered public-input claim is rejected. Confirms the negatives in C/D/F/J/L are genuine rejections, not vacuous acceptances. - Probe M (probe_m_long_chain) added (50-layer IVC chain, fixed-point-holds-at-depth); runs slow, result folded into the memo separately. These validate recursion-mechanism robustness (multi-AIR, concurrency, soundness, depth) for a future re-evaluation; they do not change the NO-GO (cross-layer state threading is still unbuildable). * feat(plonky3): Probe P — proof serialization round-trip (node persistence) A recursion proof bincode-serializes to ~363 KB, round-trips byte-stable, and the deserialized proof still verifies; a truncated blob is rejected. Adds a verify_batch_proof helper. Relevant to Phase 6 proof-blob storage. Does not change the NO-GO. * docs(plonky3): record mechanism-robustness probes L-P (multi-AIR, depth-50, concurrency, soundness, serialization)
…recursion-reduction — GO, /api/send recovers w/o UX regression, port HOLD (#214) * feat(plonky3): Probe Q — a custom AIR's public value DOES cross a batch layer (overturns NO-GO) Replicates upstream test_batch_verifier_with_public_values (PR #407, in our pinned rev) in our crate: a custom PublicValueAir (num_public_values=1) proved with prove_batch and verified in-circuit via verify_batch_circuit surfaces its public value as a non-empty air_public_target (NOT [0,0,0]) and binds it soundly across the batch layer — correct value accepted, wrong value rejected. This overturns the scoped NO-GO: the [0,0,0] finding (probes D/G/H) held only for the PRIMITIVE tables and CircuitBuilder public inputs (which route to the committed Public table). A public-value-emitting AIR provides exactly the per-instance cross-layer value channel the IVC needs. The full IVC chaining via a custom carrier table is a public-API construction (~400-650 LOC), not an impossibility. * docs(plonky3): solution-space research — NO-GO overturned, 9 paths assessed Probe Q empirically overturns the scoped NO-GO: a custom AIR's public value DOES cross a batch-recursion layer (PR #407, in our pinned rev). Enumerate + assess all 9 solution paths with links/repo-pointers: (1+5) Plonky3 + custom public-value-emitting tables — viable, channel proven, IVC chaining is a public-API construction; (3) folding/Sonobe — native IVC, strong alternative; (2) self-authored upstream PR; (4) hybrid; (6) protocol redesign via off-circuit continuity (trusted node, §7.22 posture); (7) zkVMs; (8) fork (excluded §16); (9) Stwo/Triton/Halo2-accumulation. Recommend Path 1+5 behind a carrier-table IVC-chain spike (Probe R), with Sonobe benchmarked in parallel. * docs(plonky3): address review — add ProtoStar/Boojum/Lasso, gate-memo forward-pointer - Solutions doc: add ProtoStar/ProtoGalaxy + SuperNova (folding sub-schemes), Lasso (a component, not an IVC framework), and Boojum (Goldilocks STARK, EraVM-specific) — the three systems the brief named that were missing. - MIGRATION_PLONKY3_SPIKE_RESULT.md: add a top-of-file PARTIALLY-SUPERSEDED banner and correct escape-route #1 (the cross-layer capability was present all along via PR #407, not a missing upstream feature) — so a reader landing on the gate memo is pointed to the overturning result. - probe_q: simplify the self-referential air_public_targets shape assertion. * feat(plonky3): Probe R — carrier-table IVC chain threads a counter across 4 layers (GO) A real depth-4 IVC chain: each layer is a prove_batch proof of a custom CarrierAir with two public values [v_in, v_out] (AIR enforces v_out == v_in + 1, both bound to committed trace cells); each IVC link verifies both adjacent carrier proofs in-circuit via verify_batch_circuit (their public values surface as non-empty air_public_targets, not [0,0,0]) and connects prev.v_out == cur.v_in. The counter is provably carried layer-0 -> layer-3 (V_3 == V_0 + 3). Negatives: a wrong forwarded value is rejected (WitnessConflict on the thread bind; a control with the bind removed accepts it, isolating the cause); a carrier claiming an uncommitted public value is rejected (OodEvaluationMismatch). Does NOT use build_and_prove_next_layer, so upstream #436 is not hit. Public API only, no fork. This is the end-to-end empirical confirmation of Path 1+5: the cross-layer state IVC the original NO-GO deemed impossible is buildable via custom public-value-emitting tables. * docs(plonky3): gate memo banner — GO via Path 1+5 (carrier tables), Probe R confirms end-to-end * test(plonky3): Probe R-cost — carrier chain per-transition cost at 2^16 inner scale (within budget) * docs(plonky3): record Probe R-cost in gate memo — carrier chain within warm budget, add probe_q/r/r_cost rows * docs(plonky3): review polish — gate probe_r_cost verdict on STARK-prove class (not witness-gen floor), tag superseded Gate-decision heading * test(spike): add Probe S fair BabyBear vs Plonky2 prover bench * docs(spike): review polish — honest S-box degree-3-vs-7 magnitude (~1.5-2.5x, verdict robust), bump test count 20->21 + Probe S table row * test(spike): add Probe V degree-7 S-box bench on working HidingFriPcs recipe * test(spike): add Probe W real HidingFriPcs vs blowup-2 zk-proxy delta * docs(plonky3): add cutover playbook (Doc 1) + upstream maintenance plan (Doc 4) * docs(plonky3): correct Probe S optimism with Probe V/W — degree-7 1.67x + true-hiding 3x (~5x combined), production config slower at 2^16, net verdict pending Probe T * test(spike): add Probe T real-circuit Plonky3 prove-cost estimate Cost-faithful representative workload for the real zkCoins state-transition circuit under TRUE production crypto (degree-7 Poseidon2 + Keccak-hiding MMCS + HidingFriPcs, num_random_codewords=4). Models the real cost drivers (~4500 Poseidon2 hashes + ~50k non-hash gates) as a two-table batch, NOT the business logic. Sweeps the non-hash table height over 2^13..2^16 to bracket the unknown real layout. Finding: real multi-table prove_batch (p3-batch-stark) WORKS with HidingFriPcs + mixed degree-7/degree-3 instances; verify_batch succeeds. At the realistic layout (~2^13-2^14) Plonky3+BabyBear proves in ~312-449 ms warm p50 vs Plonky2 4350 ms = ~10-14x faster, ~2-3x lower RSS, near-zero circuit build (0.07 ms vs 8.2 s). Faster across the entire sweep including the 2^16 ceiling. * docs(plonky3): add wire/storage format migration (Doc 2) + carrier-table crypto-audit spec (Doc 3) * docs(plonky3): integrate Probe T — real circuit 10-14x faster under production crypto; V/W 2^16 was hash-saturation; full-prove verdict pending X+U * test(spike): add Probes X (aggregator recursion overhead), Y (cold-start), Z (verifier), AA (sustained-load soak) * docs(plonky3): Probe U e2e projection + integrate X/Y/Z/AA net verdict — send is wash/slower (recursion-dominated), mint ~2x, cold-start 38.7x, no leak * docs(plonky3): full migration audit summary — honest mixed verdict, decisive X-prime lever, operator decisions * docs(plonky3): redact internal host names from cutover playbook — role language only (review blocker) * test(spike): Probe X' batched-aggregator lever — same-vk verifier amortization Measure whether batching the 8 same-vk source proofs cuts the flat 8+1 aggregator cost Probe X reported (4.0s non-zk / 6.7s zk). Two framings, real STARK-prove via prove_all_tables: X'-a proves the 8 sources as one multi-instance BatchProof verified in-circuit once (lower bound: 0.98s non-zk / 1.66s zk, 4.1x reduction); X'-b proves 8 independent same-vk proofs as in the real protocol (3.97s non-zk / 6.69s zk, ~1.0x = flat). The recursion API verifies one BatchProof per verify_batch_circuit, so independent same-vk proofs cannot share the verifier — the batched floor is unreachable for /api/send. Realistic full send recomposes to 9.9s non-zk / 12.6s zk, a wash-or-loss vs Plonky2. Batching does not rescue the send case; MAX_IN_COINS reduction is the lever. * docs(plonky3): resolve batching lever via Probe X-prime — not reachable in-protocol, send case rests on MAX_IN_COINS; test count 29 * docs(plonky3): update Fair-Performance lead to the resolved mixed verdict (T/X/X-prime/U) * test(spike): Probe AB recursion-friendly levers — cheaper-inner-FRI 2.4x (64-bit), Poseidon2-MMCS already baseline, ZK-only-outer ~0 * test(spike): Probe AC MAX_IN_COINS sweep — aggregation ~linear in fan-in (~448ms/coin); N=4+cheaper-FRI cuts prove ~4x; e2e capped by node overhead * test(spike): Probe AD KoalaBear-vs-BabyBear field comparison — split verdict; KoalaBear transition ~1.26x faster (degree-3 leaf S-box) but dominant 8+1 aggregation ~2.1x SLOWER (20 vs 13 partial rounds in recursion verifier); recommend STAY on BabyBear * test(spike): probe AE — composed best-config full send-prove measurement * docs(plonky3): recursion-reduction research (AB-AE) — send speed case recoverable: MAX_IN_COINS=4 alone 1.9x, +64-bit inner FRI 3.32x; KoalaBear ruled out; 33 tests * docs(plonky3): apply resolutions — keep MAX_IN_COINS=8 (no UX regression), 64-bit inner FRI as port-phase auditor gate, port HOLD; recommended N=8+q48 = 2.25x send-prove
Keep the node repo limited to code, build, and standard project files. The protocol design drafts, the circuit spec, the roadmap, and the program-plonky2 session notes are archived verbatim in zk-coins/research (zkcoins-design/); the roadmap is also published at docs.zkcoins.app/roadmap. - delete root design markdowns: ARKADE_INTEGRATION, BITVM_BRIDGE, BRIDGE_MVP, LIGHTNING_ATOMIC_SWAP, MIGRATION_RESEARCH, MULTI_ASSET - delete SPEC.md (circuit/single-asset spec, archived to research) and ROADMAP.md (published at docs.zkcoins.app/roadmap) - delete program-plonky2 session notes (SESSION_STATE, STAGE_5D_NEXT_4_DESIGN, STEP4_REVIEW, STEP7_PREP) - slim CONTRIBUTING.md (936 -> 279 lines): keep dev setup, coding standards (incl. "No polling — events only" and the M3 Ultra target referenced by CI), and the PR flow; drop roadmap/migration narrative - rewire README, the program-plonky2 crate guide, and two circuit doc-comments to the docs site / research repo (no dangling references)
The Plonky3 recursion spike, its migration write-ups, and its benchmark results were merged to staging only (PRs #211/#212/#214) and must not be promoted to develop. Remove them here so the next staging -> develop auto-promote carries no Plonky3-migration artifacts. Everything removed is archived verbatim in zk-coins/research. - delete the plonky3-recursion-spike crate (36 files) - delete MIGRATION_PLONKY3.md / _SOLUTIONS_RESEARCH / _SPIKE_RESULT - delete docs/migration/PLONKY3_*.md (5 files) - delete scripts/bench/results/plonky3-*.md (5 files) - restore the workspace Cargo.toml to develop's form (drop the now-unused `exclude = ["spikes/plonky3-recursion-spike"]`)
…xDetail) (#218) The wallet's transaction-detail page needs more than the lean /api/history list row. Add a scoped detail endpoint that returns everything the node can derive for one account_history row without a schema change: - All HistoryItem core fields (txid/timestamp/direction/amount/status/ block_height/...), via the same history_row_to_item mapping so the two endpoints cannot drift. - The decoded account-state snapshot of the mutation: usable balance before/after (settled + coin_queue, mirroring balance_from_account_blob), the post-mutation num_sends (the wallet's authoritative BIP-32 child index), and the commitment public key (33-byte compressed hex). - The verifier circuit digest (proof-system identity) from circuit_digest_meta; a read failure degrades the field to null. - pending_inscriptions.commit_output_value when an inscription row exists (detail-only; the list query stays lean). Scoping: the row must match (id, address) AND have a user-facing source (mint/send/receive) — wrong-address or internal rows 404 identically, so ids cannot be enumerated across accounts. Malformed address or a non-integer/non-positive id is 422 (id parsed from the path as a string so the read surface keeps one validation contract; axum 0.7 would otherwise 400). Tests: handler-level unit tests for every branch (422 x5, 404 x2, 500 x2 incl corrupt-blob, 200 happy + digest), db-level tests for the scoped item query incl the inscription join, pure-fn tests for the decoders, api_remote live round-trip (mint -> list -> detail) + validation contract, openapi smoke (path + TxDetail schema).
Per the project model the node repo carries code/build/standard files only - no benchmark output. Delete scripts/bench/results/ (README + m5-max HTTP-mint-sweep CSV, probe_r2 JSON, m5-max-vs-m3-ultra write-up). The bench harness (node/src/bin/probe_r2.rs) stays; only the output moves. Archived verbatim in zk-coins/research benchmarks/node-runtime/.
* docs: Plonky3 migration plan + Phase-0 GO (carrier-table direction) (#211) * docs: add Plonky3 migration plan with phased, executable task spec Adds MIGRATION_PLONKY3.md: a phase-by-phase, execution-ready work plan for the Plonky2 -> Plonky3 backend swap. Phase 0 is a hard recursion feasibility gate against p3-recursion (probed in Goldilocks to isolate the recursion/API risk from the field-migration risk). Subsequent phases port types/hash, Merkle gadgets, the state-transition circuit, recursion + aggregator, node integration, and parity/coverage/bench, each with exact files, acceptance criteria, and local verification commands. Field swap to KoalaBear/BabyBear is sequenced as a separate optional follow-up. Companion to ROADMAP.md and MIGRATION_RESEARCH.md. * docs: record Phase 0 result + defer cross-layer PI-threading choice to Phase-1-authorize The Phase 0 spike returned GO with one escalated finding: the high-level batch recursion does not propagate public inputs across layers (probe_d_multilayer_carry: air_public_targets = [0,0,0]), unlike Plonky2 cyclic recursion. This is a construction problem, not a p3-recursion capability gap, so it does not flip the gate to NO-GO. Record this in the Phase 0 gate section, and mark the cross-layer PI-threading construction as TBD — the choice between Option 1 (threaded outputs as AIR public values, fast), Option 2 (commit + Merkle/hash re-bind each layer, sound), and Option 3 (pinned probe catches a future upstream rev that propagates natively) is made at Phase-1-authorize time, not now. P5-T1 is written against the chosen option and its acceptance now requires the threaded prev_account value to be carried across transitions. * docs: resolve cross-layer threading to Option 2 + record Phase-5 budget risk Phase 0 closed the open threading question empirically, so the plan no longer defers it to Phase-1-authorize: - Cross-layer PI threading is RESOLVED to Option 2 (commit + hash/Merkle re-bind each layer). Option 1 (carry the value as an AIR public value) is dead — proven by probe_h_option1_air_public_values (injecting a non-existent public input is rejected) and probe_g_fanin_pi_passthrough (a real aggregation surfaces 0 per-leaf values to the outer); CircuitBuilder public inputs live in the committed Public table, never as AIR public values. Reflected in the §5 recorded result, the §6 Phase-1-authorize block (now "RESOLVED: Option 2"), P5-T1, and P5-T2 (per-leaf ProofData also needs Option-2 commit+re-bind, then §7.17 masking). - Record the Phase-5 warm-prove budget risk from probe_i_cost_projection: a recursion layer over a ~2^16-gate inner proof is ~3.2 s / ~1.4 GB — a single-layer lower bound on an arithmetic toy; the mandatory Option-2 re-bind adds Poseidon gates per layer, and the real constraints are Poseidon-heavy. Measure the real circuit + Option-2 at the START of Phase 5; a >5 s warm-prove is a NO-GO trigger. Option 3 (pinned probes catch a future upstream rev that restores native propagation) stays armed. * docs: gate is NO-GO — neither Option 1 nor Option 2 threads state across a batch layer Phase 0 closed the cross-layer threading question definitively: Option 1 (AIR public values) is dead (probe_h/g/d_multilayer_carry) and Option 2 (commit + hash re-bind) is also dead (probe_j + adversarial review — layer N+1 cannot read layer N's committed digest). There is no per-instance value channel across a batch-recursion layer, so zkCoins' prev_account/ProofData IVC carry is structurally unbuildable on this rev. Record NO-GO in the §5 result, the §6 Phase-1-authorize block ("DO NOT START Phase 1"; §6-§14 kept as a would-be plan for if an escape route opens), P5-T1 (BLOCKED), P5-T2 (BLOCKED), and the §10 budget note (moot under NO-GO). Escape routes: an upstream rev that exposes cross-layer public inputs (pinned probes auto-detect), a protocol redesign, or a fork (excluded by §16). Decision is the operator's per §16. * docs(plonky3): gate is GO via Path 1+5 (carrier tables) — overturn NO-GO, unblock Phases 4-5 Phase 0 reassessment: probe_q + probe_r_carrier_chain prove a custom AIR's public value (carrier table) threads state across a batch layer end-to-end (V_3==V_0+3, sound negatives, no fork). probe_r_cost: within ≤5s warm budget. Update P0-T6 memo block, §6 Phase-0 outcome, and P5-T1/P5-T2/budget-note to the carrier-table direction. Rationale: MIGRATION_PLONKY3_SOLUTIONS_RESEARCH.md; proof: PR #214. * feat(plonky3): Phase 0 recursion feasibility spike — gate is GO (carrier tables) (#212) * feat(plonky3): Phase 0 recursion feasibility spike — GO gate Add spikes/plonky3-recursion-spike, an isolated probe (its own workspace, excluded from the root workspace so the heavy Plonky3 git deps never enter the node/shared build or CI) that empirically proves Plonky3/Plonky3-recursion can express the three composition patterns the zkCoins circuit depends on, in Goldilocks, on the pinned revs: - Probe A (IVC/cyclic with base case): 4-layer chain; the verifier-circuit shape reaches a fixed point (true IVC, no growth) — the analogue of Plonky2 common_data_for_recursion. witness_count [25567, 104630, 107957, 107957]. - Probe B (fan-in-8, variable active count): 2-to-1 aggregation composes into a fixed-shape tree (fan-in-4 probed; fan-in-8 is one more level). No native conditional-verify primitive; inactive slots are padded with real proofs and masked downstream via an active bit. - Probe C (vk/PI binding): an inner proof's public inputs are bound in the verifier circuit; a mismatched claim is rejected in-circuit. Record P0-T5 cost (~4.65 s per stabilized layer, ~1 GB peak RSS) and the P0-T6 Go/No-Go memo (MIGRATION_PLONKY3_SPIKE_RESULT.md). Gate decision: GO. Pins: Plonky3-recursion 524665d0c2e1d294722c064786ae11dff8d9f33b, Plonky3 56952503e1401a62982ceaf952c5e4a829b61803. * docs(plonky3): tighten Phase 0 memo + probes to match §5 PASS criteria exactly Address logic-review findings — scope the spike's claims precisely against MIGRATION_PLONKY3.md §5 so the GO gate is not overstated: - Probe A: strengthen the fixed-point assertion (require the shape to have GROWN before stabilising, not just last-two-equal). Document that cross-layer counter PI THREADING (P0-T2 crit. 2) is not exercised — into_recursion_input carries empty table_public_inputs; the enabling primitive is proven in Probe C; explicit threading is Phase-5 work. - Probe B: relabel honestly — the probe proves 2-to-1 fan-in TREE COMPOSITION (4 identical real leaves), NOT variable active count / per-leaf PIs / active-bit masking. The masking strategy (§7.17) is Phase-5 construction on proven primitives. - Probe C: relabel as PUBLIC-INPUT binding (proven), with vk-equality connect-back as Phase-5 construction (the literal "wrong-vk proof" is not fed). Memo gate decision restated: the three mechanisms are proven; the three deferred items are in-repo Phase-5 construction, not upstream gaps. Decision stays GO. All 4 probes still green; fmt + clippy clean. * feat(plonky3): exercise the three §5 PASS items empirically (Probes D/E/F) The Phase-0 gate previously proved the three mechanisms but DEFERRED the three real §5 PASS items to Phase 5. This adds probes that exercise them end-to-end with real proving and positive+negative (+control) assertions: - Probe D (probe_d_pi_threading): cross-layer PI threading binding. An outer circuit verifying an inner uni-stark proof exposes air_public_targets and threads a value (next_start = inner.last + 1) bound to an outer public input; a wrong threaded value is rejected; a control without the bind accepts it. - Probe D part 2 (probe_d_multilayer_carry): the escalated finding. Verifying an inner BATCH proof of a CircuitBuilder circuit exposes NO inner public inputs (air_public_targets = [0,0,0]); the high-level chain does not propagate public inputs across layers (differs from Plonky2 cyclic recursion). Pinned by assert. - Probe E (probe_e_active_masking): variable-active-count masking (§7.17). An 8-slot fixed-shape consumer circuit, batch-stark-proved for real: active+correct with inactive-garbage accepted (masked); active-wrong rejected; flipping a garbage slot's active bit flips the verdict; flipping back re-masks. - Probe F (probe_f_vk_binding): vk-equality connect-back. Two ConstPrepAir instances (k=42/99) have different preprocessed commitments (= vks). A proof from vk_99 (internally valid against vk_99) is rejected SOLELY by the connect to vk_42; a control accepts it unbound. Rewrite MIGRATION_PLONKY3_SPIKE_RESULT.md: the §5 items are now exercised (not deferred); the gate is GO with ONE escalated finding (cross-layer public-input propagation), surfaced for operator judgment per §16. All 8 probes green; fmt + clippy clean. * docs(plonky3): mark probe_a row as P0-T2 crit. 1 (not PI threading) * feat(plonky3): close the cross-layer/cost gaps before Phase 1 (Probes G/H/I) Three integrated probes that resolve the previously-escalated open questions into hard, pre-Phase-1 constraints: - Probe H (probe_h_option1_air_public_values): Option 1 (carry the threaded value as an AIR public value) is DEAD. The honest empty-PI layer builds+proves; injecting a non-empty RecursionInput::BatchStark.table_public_inputs is rejected. Combined with probe_d_multilayer_carry (air_public_targets = [0,0,0]), both Option-1 avenues fail. - Probe G (probe_g_fanin_pi_passthrough): a real 2-to-1 aggregation's per-leaf values are NOT surfaced to the outer (air_public_targets = 0). The integrated fan-in-8 per-leaf-PI passthrough is blocked at the first cross-layer hop. => Together G+H decide the Phase-1-authorize choice: Option 2 (commit + hash re-bind) is MANDATORY for IVC threading AND aggregator per-leaf surfacing. - Probe I (probe_i_cost_projection): recursion-layer cost at real inner-proof scale. Sub-linear scaling; a layer over a ~2^16-gate proof is ~3.2s / ~1.4GB. Combined with the mandatory Option-2 overhead and the base prove, the 5s warm budget is at material risk -> measure on the real circuit early in Phase 5. Rewrite MIGRATION_PLONKY3_SPIKE_RESULT.md: status is now CONDITIONAL GO (Option 2 mandatory; warm-prove budget at material risk). The pinned probes (G/H/multilayer_carry) catch a future upstream rev that restores native public-input propagation. All 11 probes green; fmt + clippy clean. * docs(plonky3): scope Option-1 'impossible' to 'not achievable on this rev' * feat(plonky3): Probe J — Option 2 commit+rebind primitive works, but cannot compose The in-circuit Poseidon2 hash-bind primitive (add_hash_slice + connect) is real and binding: hash(V) binds to hash(V); a mismatched preimage is rejected. So Option 2's per-layer commit+rebind building block is expressible. But it needs layer N+1 to read layer N's committed digest, which is structurally impossible across a batch layer (probe_d_multilayer_carry / probe_g / probe_h: no per-instance value exposed, only whole-trace Merkle commitments). So multi-layer Option-2 threading is NOT achievable — confirming both Option 1 and Option 2 are dead and the cross-layer state IVC is unbuildable on this rev. * docs(plonky3): gate is NO-GO — Option 2 cannot compose across batch layers (Probe J) Probe J + an adversarial review of all six escape routes confirm that neither Option 1 nor Option 2 can thread a value across a batch-recursion layer. The per-layer commit+rebind primitive works (in-circuit Poseidon2 hash-bind), but layer N+1 cannot read layer N's committed digest (batch proofs expose only whole-trace Merkle commitments; FRI openings are FS-random; vk is per-circuit-static; NPO public_values are hardcoded empty with no public registration path). So zkCoins' cross-layer state IVC is structurally unbuildable on this rev. Memo status: CONDITIONAL GO -> NO-GO, with escape routes (upstream feature / protocol redesign / fork-excluded). * feat(plonky3): Probes L/N/O — multi-AIR coexistence, concurrency, soundness - Probe L (probe_l_multi_air): two heterogeneous AIRs (CounterAir state-transition-like + ConstPrepAir aggregator-like) co-verify in one verifier circuit, public inputs kept distinct + individually bound; cross-wiring A's PI to B's value is rejected. - Probe N (probe_n_concurrent): 4 independent prove+recurse+verify workloads on separate threads all succeed; peak RSS ~1.38 GB. Prover is concurrency-safe. - Probe O (probe_o_soundness): soundness spot-check — mismatched FRI private data (a different proof's Merkle paths) is rejected by the in-circuit verification, and a tampered public-input claim is rejected. Confirms the negatives in C/D/F/J/L are genuine rejections, not vacuous acceptances. - Probe M (probe_m_long_chain) added (50-layer IVC chain, fixed-point-holds-at-depth); runs slow, result folded into the memo separately. These validate recursion-mechanism robustness (multi-AIR, concurrency, soundness, depth) for a future re-evaluation; they do not change the NO-GO (cross-layer state threading is still unbuildable). * feat(plonky3): Probe P — proof serialization round-trip (node persistence) A recursion proof bincode-serializes to ~363 KB, round-trips byte-stable, and the deserialized proof still verifies; a truncated blob is rejected. Adds a verify_batch_proof helper. Relevant to Phase 6 proof-blob storage. Does not change the NO-GO. * docs(plonky3): record mechanism-robustness probes L-P (multi-AIR, depth-50, concurrency, soundness, serialization) * feat(plonky3): carrier-table IVC (Path 1+5) + full migration audit + recursion-reduction — GO, /api/send recovers w/o UX regression, port HOLD (#214) * feat(plonky3): Probe Q — a custom AIR's public value DOES cross a batch layer (overturns NO-GO) Replicates upstream test_batch_verifier_with_public_values (PR #407, in our pinned rev) in our crate: a custom PublicValueAir (num_public_values=1) proved with prove_batch and verified in-circuit via verify_batch_circuit surfaces its public value as a non-empty air_public_target (NOT [0,0,0]) and binds it soundly across the batch layer — correct value accepted, wrong value rejected. This overturns the scoped NO-GO: the [0,0,0] finding (probes D/G/H) held only for the PRIMITIVE tables and CircuitBuilder public inputs (which route to the committed Public table). A public-value-emitting AIR provides exactly the per-instance cross-layer value channel the IVC needs. The full IVC chaining via a custom carrier table is a public-API construction (~400-650 LOC), not an impossibility. * docs(plonky3): solution-space research — NO-GO overturned, 9 paths assessed Probe Q empirically overturns the scoped NO-GO: a custom AIR's public value DOES cross a batch-recursion layer (PR #407, in our pinned rev). Enumerate + assess all 9 solution paths with links/repo-pointers: (1+5) Plonky3 + custom public-value-emitting tables — viable, channel proven, IVC chaining is a public-API construction; (3) folding/Sonobe — native IVC, strong alternative; (2) self-authored upstream PR; (4) hybrid; (6) protocol redesign via off-circuit continuity (trusted node, §7.22 posture); (7) zkVMs; (8) fork (excluded §16); (9) Stwo/Triton/Halo2-accumulation. Recommend Path 1+5 behind a carrier-table IVC-chain spike (Probe R), with Sonobe benchmarked in parallel. * docs(plonky3): address review — add ProtoStar/Boojum/Lasso, gate-memo forward-pointer - Solutions doc: add ProtoStar/ProtoGalaxy + SuperNova (folding sub-schemes), Lasso (a component, not an IVC framework), and Boojum (Goldilocks STARK, EraVM-specific) — the three systems the brief named that were missing. - MIGRATION_PLONKY3_SPIKE_RESULT.md: add a top-of-file PARTIALLY-SUPERSEDED banner and correct escape-route #1 (the cross-layer capability was present all along via PR #407, not a missing upstream feature) — so a reader landing on the gate memo is pointed to the overturning result. - probe_q: simplify the self-referential air_public_targets shape assertion. * feat(plonky3): Probe R — carrier-table IVC chain threads a counter across 4 layers (GO) A real depth-4 IVC chain: each layer is a prove_batch proof of a custom CarrierAir with two public values [v_in, v_out] (AIR enforces v_out == v_in + 1, both bound to committed trace cells); each IVC link verifies both adjacent carrier proofs in-circuit via verify_batch_circuit (their public values surface as non-empty air_public_targets, not [0,0,0]) and connects prev.v_out == cur.v_in. The counter is provably carried layer-0 -> layer-3 (V_3 == V_0 + 3). Negatives: a wrong forwarded value is rejected (WitnessConflict on the thread bind; a control with the bind removed accepts it, isolating the cause); a carrier claiming an uncommitted public value is rejected (OodEvaluationMismatch). Does NOT use build_and_prove_next_layer, so upstream #436 is not hit. Public API only, no fork. This is the end-to-end empirical confirmation of Path 1+5: the cross-layer state IVC the original NO-GO deemed impossible is buildable via custom public-value-emitting tables. * docs(plonky3): gate memo banner — GO via Path 1+5 (carrier tables), Probe R confirms end-to-end * test(plonky3): Probe R-cost — carrier chain per-transition cost at 2^16 inner scale (within budget) * docs(plonky3): record Probe R-cost in gate memo — carrier chain within warm budget, add probe_q/r/r_cost rows * docs(plonky3): review polish — gate probe_r_cost verdict on STARK-prove class (not witness-gen floor), tag superseded Gate-decision heading * test(spike): add Probe S fair BabyBear vs Plonky2 prover bench * docs(spike): review polish — honest S-box degree-3-vs-7 magnitude (~1.5-2.5x, verdict robust), bump test count 20->21 + Probe S table row * test(spike): add Probe V degree-7 S-box bench on working HidingFriPcs recipe * test(spike): add Probe W real HidingFriPcs vs blowup-2 zk-proxy delta * docs(plonky3): add cutover playbook (Doc 1) + upstream maintenance plan (Doc 4) * docs(plonky3): correct Probe S optimism with Probe V/W — degree-7 1.67x + true-hiding 3x (~5x combined), production config slower at 2^16, net verdict pending Probe T * test(spike): add Probe T real-circuit Plonky3 prove-cost estimate Cost-faithful representative workload for the real zkCoins state-transition circuit under TRUE production crypto (degree-7 Poseidon2 + Keccak-hiding MMCS + HidingFriPcs, num_random_codewords=4). Models the real cost drivers (~4500 Poseidon2 hashes + ~50k non-hash gates) as a two-table batch, NOT the business logic. Sweeps the non-hash table height over 2^13..2^16 to bracket the unknown real layout. Finding: real multi-table prove_batch (p3-batch-stark) WORKS with HidingFriPcs + mixed degree-7/degree-3 instances; verify_batch succeeds. At the realistic layout (~2^13-2^14) Plonky3+BabyBear proves in ~312-449 ms warm p50 vs Plonky2 4350 ms = ~10-14x faster, ~2-3x lower RSS, near-zero circuit build (0.07 ms vs 8.2 s). Faster across the entire sweep including the 2^16 ceiling. * docs(plonky3): add wire/storage format migration (Doc 2) + carrier-table crypto-audit spec (Doc 3) * docs(plonky3): integrate Probe T — real circuit 10-14x faster under production crypto; V/W 2^16 was hash-saturation; full-prove verdict pending X+U * test(spike): add Probes X (aggregator recursion overhead), Y (cold-start), Z (verifier), AA (sustained-load soak) * docs(plonky3): Probe U e2e projection + integrate X/Y/Z/AA net verdict — send is wash/slower (recursion-dominated), mint ~2x, cold-start 38.7x, no leak * docs(plonky3): full migration audit summary — honest mixed verdict, decisive X-prime lever, operator decisions * docs(plonky3): redact internal host names from cutover playbook — role language only (review blocker) * test(spike): Probe X' batched-aggregator lever — same-vk verifier amortization Measure whether batching the 8 same-vk source proofs cuts the flat 8+1 aggregator cost Probe X reported (4.0s non-zk / 6.7s zk). Two framings, real STARK-prove via prove_all_tables: X'-a proves the 8 sources as one multi-instance BatchProof verified in-circuit once (lower bound: 0.98s non-zk / 1.66s zk, 4.1x reduction); X'-b proves 8 independent same-vk proofs as in the real protocol (3.97s non-zk / 6.69s zk, ~1.0x = flat). The recursion API verifies one BatchProof per verify_batch_circuit, so independent same-vk proofs cannot share the verifier — the batched floor is unreachable for /api/send. Realistic full send recomposes to 9.9s non-zk / 12.6s zk, a wash-or-loss vs Plonky2. Batching does not rescue the send case; MAX_IN_COINS reduction is the lever. * docs(plonky3): resolve batching lever via Probe X-prime — not reachable in-protocol, send case rests on MAX_IN_COINS; test count 29 * docs(plonky3): update Fair-Performance lead to the resolved mixed verdict (T/X/X-prime/U) * test(spike): Probe AB recursion-friendly levers — cheaper-inner-FRI 2.4x (64-bit), Poseidon2-MMCS already baseline, ZK-only-outer ~0 * test(spike): Probe AC MAX_IN_COINS sweep — aggregation ~linear in fan-in (~448ms/coin); N=4+cheaper-FRI cuts prove ~4x; e2e capped by node overhead * test(spike): Probe AD KoalaBear-vs-BabyBear field comparison — split verdict; KoalaBear transition ~1.26x faster (degree-3 leaf S-box) but dominant 8+1 aggregation ~2.1x SLOWER (20 vs 13 partial rounds in recursion verifier); recommend STAY on BabyBear * test(spike): probe AE — composed best-config full send-prove measurement * docs(plonky3): recursion-reduction research (AB-AE) — send speed case recoverable: MAX_IN_COINS=4 alone 1.9x, +64-bit inner FRI 3.32x; KoalaBear ruled out; 33 tests * docs(plonky3): apply resolutions — keep MAX_IN_COINS=8 (no UX regression), 64-bit inner FRI as port-phase auditor gate, port HOLD; recommended N=8+q48 = 2.25x send-prove * chore: remove Plonky3 migration content from staging (#217) The Plonky3 recursion spike, its migration write-ups, and its benchmark results were merged to staging only (PRs #211/#212/#214) and must not be promoted to develop. Remove them here so the next staging -> develop auto-promote carries no Plonky3-migration artifacts. Everything removed is archived verbatim in zk-coins/research. - delete the plonky3-recursion-spike crate (36 files) - delete MIGRATION_PLONKY3.md / _SOLUTIONS_RESEARCH / _SPIKE_RESULT - delete docs/migration/PLONKY3_*.md (5 files) - delete scripts/bench/results/plonky3-*.md (5 files) - restore the workspace Cargo.toml to develop's form (drop the now-unused `exclude = ["spikes/plonky3-recursion-spike"]`) * feat(api): GET /api/history/{id} — per-transaction detail endpoint (TxDetail) (#218) The wallet's transaction-detail page needs more than the lean /api/history list row. Add a scoped detail endpoint that returns everything the node can derive for one account_history row without a schema change: - All HistoryItem core fields (txid/timestamp/direction/amount/status/ block_height/...), via the same history_row_to_item mapping so the two endpoints cannot drift. - The decoded account-state snapshot of the mutation: usable balance before/after (settled + coin_queue, mirroring balance_from_account_blob), the post-mutation num_sends (the wallet's authoritative BIP-32 child index), and the commitment public key (33-byte compressed hex). - The verifier circuit digest (proof-system identity) from circuit_digest_meta; a read failure degrades the field to null. - pending_inscriptions.commit_output_value when an inscription row exists (detail-only; the list query stays lean). Scoping: the row must match (id, address) AND have a user-facing source (mint/send/receive) — wrong-address or internal rows 404 identically, so ids cannot be enumerated across accounts. Malformed address or a non-integer/non-positive id is 422 (id parsed from the path as a string so the read surface keeps one validation contract; axum 0.7 would otherwise 400). Tests: handler-level unit tests for every branch (422 x5, 404 x2, 500 x2 incl corrupt-blob, 200 happy + digest), db-level tests for the scoped item query incl the inscription join, pure-fn tests for the decoders, api_remote live round-trip (mint -> list -> detail) + validation contract, openapi smoke (path + TxDetail schema). * chore: move benchmark output to research (#219) Per the project model the node repo carries code/build/standard files only - no benchmark output. Delete scripts/bench/results/ (README + m5-max HTTP-mint-sweep CSV, probe_r2 JSON, m5-max-vs-m3-ultra write-up). The bench harness (node/src/bin/probe_r2.rs) stays; only the output moves. Archived verbatim in zk-coins/research benchmarks/node-runtime/. --------- Co-authored-by: TaprootFreak <142087526+TaprootFreak@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automatic Release PR
Commits: 2 new commit(s)