* feat(shared): add spec-v1.1 protocol foundations (spec_v1 module)
Additive new shared::spec_v1 module implementing the frozen spec-v1.1 data
foundation the node protocol rebuild sits on:
- domain-tagged Poseidon hash catalogue (Hc) + SHA-256 boundaries
- core data structures: multi-asset AccountState, Coin, CoinTemplate,
ProofData (192B), SpendRecord (96B), x-only keys
- canonical serialization (serialize/parse) + Bech32m addresses
- NfLog (RFC-6962) + CoinHist SMT leaf/node hashing
Old-model code untouched; workspace compiles. Byte-exact against pinned
vectors V.1/V.2/V.3/V.5/V.6/V.11; generates the V.4 Poseidon values.
* fix(shared): canonical spec_v1 digest serialization + strict parsing
Address second-vendor review findings on P1-A:
- digest->bytes now reduces each Goldilocks limb via to_canonical_u64()
(spec 1.7.1); own spec_v1 encoder, old hash.rs untouched
- digest parse rejects non-canonical limbs (>= p) with SpecError (fail-loud)
- address decode is strict Bech32m only (rejects legacy Bech32)
- parse_account_state rejects non-ascending asset_id wire order (1.7.4)
- account_state_hash takes &AccountState, hashes canonical serialization
- generated-vectors test pins six anchor V.4 hex values as a regression oracle
Adds 6 regression tests; cargo test -p shared: 50 passed.
* feat(shared): P1-B host nullifier accumulator + CoinHist SMT
- NfLog RFC-6962/9162 inclusion (PATH) and consistency (PROOF/SUBPROOF)
proofs over the Poseidon log, with position-binding and the correct
right-associative old-root reconstruction; boundary suite k=0..12 and
multi-set-bit prefix tests (m with 3-4 set bits)
- stateful in-memory Path-A accumulator: first-occurrence fold, Pk->(pos,R)
index, NAV (size, mth), canonical-value check, double-spend classification,
reorg canonical replay (truncate-and-refold), activation_height origin
- NetworkParams pinned tuple with canonical encoding + SHA-256 identifier
- per-account CoinHist SMT: admit (0->1), spend (1->2), non-inclusion
cargo test -p shared: 79 passed; clippy -p shared --no-deps clean.
* fix(shared): P1-B accumulator review fixes
Second-vendor review findings on the consensus-bearing accumulator:
- fold() now takes a full ChainPosition and rejects out-of-canonical-order
entries (defense-in-depth on the total-order property, spec 3.6/3.7)
- reorg_replay signals a finality-breaking reorg (snapshots the final prefix,
reports displacement) so /health/ready can drop to 503 deep_reorg (spec 3.9)
- size_final pins finality_confirmations = 6 and uses checked_sub (no unchecked
subtraction / release-mode wrap)
- NetworkParams fields are private, constructed only through validating new();
canonical_encoding errors on an over-long tag instead of truncating
- verify_subproof rejects a consistency proof with prepended garbage nodes
- coinhist level>256 and admit-on-spent now fail loud with precise errors
Adds 13 regression tests; cargo test -p shared: 92 passed; clippy --no-deps clean.
* feat(program-plonky2): P1-C.a in-circuit SHA-256 and wide-u128 gadgets
In-circuit crypto primitives for the compliance circuit, built from
plonky2 1.1.0 primitives (no external ecosystem gadget crates):
- SHA-256 (FIPS 180-4) + BIP-340 tagged-SHA-256 gadget
- wide multi-limb u128 arithmetic: range-checked limbs, carry-propagated
addition over >=132-bit width, exact >= comparison
Tests: in-circuit SHA-256 byte-exact vs host sha2 (empty/abc/192-byte/
tagged) + wrong-preimage rejected; u128 conservation holds and a
mod-2^128 wraparound is rejected. cargo test gadgets: 7 passed.
* chore: lock sha2 dev-dependency for program-plonky2 gadget tests (P1-C.a)
* feat(program-plonky2): P1-C2 in-circuit RFC-6962 NfLog inclusion + consistency gadget
In-circuit verification of RFC-6962 inclusion (PATH) and log-consistency
(PROOF/SUBPROOF) proofs over the NfLog accumulator, using domain-tagged
in-circuit Poseidon that matches the host shared::spec_v1::nflog byte-for-byte:
- verify_nflog_inclusion (position-bound, constant-size H_MAX=64)
- verify_nflog_consistency (dual mth_a/mth_b reconstruction, m=0/m=n/general)
- host<->circuit field-element parity cross-check
- the spec 1.7.8 D-05 boundary suite: symbolic O(log n) subtree-root fixtures
accepting k=0..=63, plus NL-B1/NL-B2 mutation rejections
shared added as a dev-dependency (test-only host cross-check). cargo test
nflog_consistency: 5 passed.
* feat(program-plonky2): P1-C.b1 in-circuit nonnative secp256k1 Fp/Fn field arithmetic
8x u32-limb foreign-field arithmetic over secp256k1 base field Fp and
scalar field Fn, from plonky2 1.1.0 primitives (no ecosystem crate):
- add_mod / sub_mod / mul_mod (schoolbook product + witnessed-quotient
reduction constrained by a*b == q*m + r and r < m, fully range-checked)
- inverse_mod (witnessed inverse constrained by a*inv == 1 mod m)
- canonical range checks (< p / < n)
Tested against host num-bigint for both fields; rejects non-canonical
witnesses and wrong product reductions. cargo test nonnative: 8 passed.
* feat(program-plonky2): port plonky2_u32 custom gates to plonky2 1.1.0
Vendored + ported the plonky2_u32 u32-arithmetic custom gates
(add_many_u32, arithmetic_u32, comparison, range_check_u32,
subtraction_u32) + gadgets from plonky2 0.1.2 to 1.1.0 (Gate-trait API
migration: eval_unfiltered(_circuit/_base_batch), generators, serialization).
A u32 multiply now uses 1 gate vs ~30 from bit-decomposition primitives --
the efficiency foundation for a viable in-circuit secp256k1 (a from-primitives
scalar_mul measured 24.5M gates). 21 ported tests pass (gate-constraint
good/bad, canonicity, low-degree).
* feat(program-plonky2): rebuild nonnative field arithmetic on ported plonky2_u32 gates
Replaced the from-primitives nonnative (thousands of gates per mul_mod,
24.5M for one scalar_mul, non-viable) with plonky2-ecdsa's biguint +
nonnative ported onto the efficient u32_lib custom gates. One nonnative
mul_mod is now 114 gates. Fp (Secp256K1Base) + Fn (Secp256K1Scalar)
add/sub/mul/inverse verified against host num-bigint; rejects non-canonical
witnesses and wrong product reductions. 10 tests pass.
* feat(program-plonky2): secp256k1 curve + GLV scalar mul on efficient nonnative
Ported plonky2-ecdsa curve gadgets (projective add/double, windowed scalar
mul, GLV endomorphism) onto the efficient nonnative. Measured (--release):
scalar_mul = 113765 gates (2^17), build 2.1s + prove 3.0s -- vs 24.5M gates
and un-buildable (>67min) from-primitives, a ~215x reduction. In-circuit
secp256k1 is now feasible; no proof-system version bump needed. Verified vs
host secp256k1: add/double/lift_x KATs, off-curve rejection, GLV scalar_mul,
mutated-witness rejection.
* feat(program-plonky2): in-circuit BIP-340 verify + sign-to-contract (P1-C.b3)
The transition-signature check (spec 2.1 clause 2 / 3.2), composing the
efficient GLV scalar-mul + the SHA-256 gadget:
- in-circuit lift_x_even_y (BIP-340 even-Y point recovery)
- BIP-340 verify: s*G == R + e*P (x-only, tagged-SHA-256 challenge e)
- sign-to-contract opening R = R' + t*G, t = SHA256(bytes(R') || H(ProofData))
Verified against the V.8 fixture (--release): valid signature verifies;
tampered s / Pk / rx are all rejected; a wrong H(ProofData) yields a
different R and is rejected (S2C genuinely binds the proof). Measured:
361528 gates (2^19), build 76s + prove 102s. Completes the in-circuit
foreign-field crypto (P1-C).
* fix(program-plonky2): constrain foreign-field soundness gaps (cross-vendor review)
- F1 (potential BLOCKER): GLV sign flags k1_neg/k2_neg were allocated with
add_virtual_bool_target_unsafe (unconstrained), which via conditional
negation + an MSM with no on-curve check could admit an off-curve point
and break BIP-340 soundness. Swapped all boolean selectors to _safe
(assert_bool); 0 unsafe remain. Inherited from upstream plonky2-ecdsa;
missed by the honest-witness functional tests. Regression test
glv_decomposition_rejects_non_boolean_sign_flag confirms a non-boolean
flag now fails to prove.
- F2: assert signature scalar s != 0 (BIP-340 requires 0 < s < n).
- F4: range-check div_rem_biguint quotient limbs to u32.
Honest cases still pass (V.8 valid-verify, scalar_mul vs host, tampered
s/Pk/rx rejected); +4 gates. Hardens P1-C.
* feat(program-plonky2): P1-D.1 compliance circuit skeleton
In-circuit AccountState/Coin/ProofData targets, variable-length
serialize(AccountState) -> ash matching host shared::spec_v1 byte-for-byte
(overwrite-mode absorption handled via per-count candidate select, verified
on a partial 3/32 active-slot account), output-coin construction + ocr
(CoinsRoot), and the 40-element public-input layout (ProofData 28 +
consumed_pubkey 8 + network_id 4) with compile-time network binding. Host
parity verified for ash/coin.identifier/ocr/address; wrong network rejected.
Skeleton: 50256 gates, build 1.2s + prove 1.1s. Foundation for clauses 1-10.
* fix(program-plonky2): config-aware u32 range-check batching for pinned config
range_check_u32_circuit now splits a range check into <=7-limb batches when
the CircuitConfig's num_wires cannot hold a wider U32RangeCheckGate (8 limbs
= 136 wires > the 135 of the SS1.7.9-pinned standard_recursion_zk_config),
so the whole secp256k1 nonnative stack builds+proves under the pinned config,
not only standard_ecc_config. Under ecc-config (136 wires) no split triggers,
so existing layouts are unchanged. Adds a pinned-config proving test
(curve_glv_scalar_mul_fits_standard_recursion_zk_config; scalar_mul 116765
gates). Also corrects the div_rem noncanonical-quotient test to assert
verify()-rejection (plonky2 validates witness consistency in verify(), not
prove(); the quotient range-check was already sound). No constraint logic changed.
* feat(program-plonky2): P1-D.2 compliance signature binding (clauses 2/4/2a-c)
Wires into the compliance circuit, under the SS1.7.9 standard_recursion_zk_config:
- clause 2: BIP-340 verify of txn_sig over the per-network m_state by Pk_i
(== prev.current_pubkey), sign-to-contract R = R' + t*G with
t = SHA256(R' || H(ProofData)), and npk_commit binding
- clause 4: nk_commit == Hc(NkCommit, nk); per-input nf = Hc(Nullifier, nk||id);
pairwise-distinct nf; inr = NullifiersRoot Merkle root
- clause 2a/2c: input recipient == owner; in-circuit coin.identifier recompute
Verified (--release): valid transition proves+verifies; every negative case
rejects (wrong sig / Pk_i / npk_commit / nk / duplicate nf / wrong input
coin.identifier); host parity for nf/inr/nk_commit/H(ProofData). 13 compliance
tests pass. Circuit (skeleton+signature): 440983 gates, ~58s build+prove.
* feat(program-plonky2): P1-D.3 conservation + mint v1/v2 + state folding + coin-history
Extends the compliance circuit with:
- clause 3: per-asset wide-u128 conservation In(a)+Mint(a) >= Out(a) (exact
non-modular; a u128 wraparound is rejected); amounts range-checked
- mint (SS6.5): v1 + token-standard-2 (creator binding, AssetId/AssetIdV2 +
terms_hash derivations, v2 cap check amount<=cap_total, v2 genesis binding)
- clause 7: new-account-state balance folding new=prev-In+Self (+Recv later),
no underflow, zero-entry removal, send_counter++, key rotation, ash recompute
- clause 8: in-circuit CoinHist SMT update (0->1 admit, 1->2 spend, sequential
two-root over witnessed paths, replay guard) matching host byte-for-byte
31 compliance tests pass (valid v1/v2 mints + conservation/wraparound/
underflow/bad-mint/cap/spend-absent/readmission-replay negatives + host parity).
* feat(program-plonky2): P1-D.4 cyclic PCD recursion + NAV + predecessor anchoring (clause 1)
Makes compliance circuit C recursive:
- cyclic recursion: C verifies its own previous proof (bootstrapped
CommonCircuitData fixed-point + a hand-written zk-safe dummy circuit, since
plonky2's vendored dummy_circuit asserts !zero_knowledge and cannot be used
under the mandated standard_recursion_zk_config); InitialProof uses the base
dummy branch, AccountUpdateProof verifies the real prev_proof
- conditional-NAV: nav_commitment == Hc(NavCommit, Hc(NfLog/Root,size||mth)||rand);
prefix(prev.nav in w.nav) via in-circuit RFC-6962 consistency (nflog_consistency)
- predecessor-nullifier anchoring: (Pk_prev,R_prev) RFC-6962 inclusion in w.nav,
R_prev S2C-opens H(prev.ProofData), Pk_prev == prev_proof.consumed_pubkey
Verified: a genuine 2-hop cyclic chain (InitialProof->AccountUpdateProof)
proves; 6 anchoring negatives reject (forged prev proof / wrong nav_commitment
/ non-prefix nav / key substitution / wrong S2C opening / pos out of range);
39 compliance tests pass. Recursive circuit: 1048576 gates (2^21),
build ~18-26min + prove ~10-11min under the zk config.
* compliance: add clause 10 received-coin admission — full circuit C
Complete the compliance predicate C with clause 10 (the receive path):
per-received-coin cyclic recursion over each creating_proof (MAX_RX_COINS=4),
coin binding to the creating proof's output_coins_root, cross-account NAV
prefix, and creating-nullifier key+leaf anchoring (Pk_create==consumed_pubkey,
R_create S2C-opens H(creating.ProofData), leaf included at pos_create<size).
Received coins feed the clause-7 balance fold (prev+self_output+received ==
new+input) and the clause-8 admission (coin-history 0->1); they do not enter
clause-3 conservation. Inactive slots are packed at the tail and contribute
zero to every value-bearing constraint.
Full C: 1,403,783 gates, degree_bits=21; all 10 clauses wired.
40/40 release tests pass (valid receive proves, six clause-10 negatives reject).
* balance: add C_balance balance-attestation circuit (§5.7)
Add the non-cyclic balance-attestation circuit C_balance. It verifies one
compliance proof under C's constant verifier data, pinning the C proof's
cyclic verifier-data public-input tail to C's canonical verifier data
in-circuit (the equivalent of check_cyclic_proof_verifier_data), and proves
the §5.7 statement: S.owner==subject, one asset's balance, ash(S) bound to
pi.new_account_state_hash, the sign-to-contract anchor on R_anchor,
Pk_anchor==pi.consumed_pubkey, prefix(nav ⊑ nav_ceiling), and network_id.
Reuses C's account hashing, NAV consistency, and S2C gadgets via minimal
pub(crate) accessors; C's constraints and circuit_digest are unchanged
(gate count identical at 1,403,783 on rebuild).
Public inputs: 60 elements per §2.5 (subject, asset_id, balance, nav_ceiling,
size_ceiling, anchor{txid,block_hash,height,Pk_anchor,R_anchor}, network_id).
size_ceiling uses a canonical (< ORDER) 64-bit decomposition bound to the same
target committed in nav_ceiling.
C_balance: 193,437 gates, degree_bits=18. Balance tests: the valid attestation
proves and verifies; eight negatives reject (six normative statement checks
plus a tampered verifier-data tail and a non-canonical size_ceiling).
* prover: add production prover bridge for C / C_balance
Add script-plonky2 prover_bridge: a host-facing API that assembles the
compliance (C) and balance-attestation (C_balance) witnesses from spec_v1
host structures, produces genuine proofs, and verifies them under the
mandatory acceptance obligations.
prove_transition proves an InitialProof or AccountUpdateProof from a
TransitionWitness, recursively verifying the predecessor and each received
coin's creating proof, and cross-checks the proved ProofData, consumed_pubkey
and network_id against the host-derived values. verify_transition performs
both obligations: data.verify plus check_cyclic_proof_verifier_data (verify
alone is insufficient for a cyclic proof). prove_attestation and
verify_attestation cover C_balance. Doc-comments state the out-of-circuit host
preconditions (canonical NAV, first-occurrence anchor) the node must enforce.
Exposes a NonNativeTarget::value accessor (no constraint change). Circuit gate
counts unchanged: C=1,403,783, C_balance=193,437. Prover-bridge end-to-end test
proves and verifies a genesis/mint, a send, and a balance attestation, and
rejects a tampered cyclic verifier-data tail; the existing compliance (36) and
balance suites pass.
* prover: add host BIP-340+S2C signature preflight to prove_transition
Replace a comment that falsely claimed a pre-proving network/signature check
with a real host-side BIP-340 + sign-to-contract verification, so an invalid
or wrong-network signature fails fast instead of wasting a full proving run
(the circuit C remains the authoritative in-circuit verifier).
verify_transition_signature reconstructs R = R' + t*G (t = SHA-256(R' ||
H(ProofData))), requires the even-y S2C nonce to match the signature's R, and
checks the BIP-340 equation s*G == R + e*P with e over m_state. It reuses the
same canonical x-only even-y lift the circuit applies to R', so it accepts
exactly the signatures C accepts. Promotes field_bytes/is_odd/tagged_hash to
module scope and makes sha2 a normal dependency. Prover-bridge end-to-end
release test passes (1646s).
* engine: add in-memory state-transition engine (P1-E.2)
Add script-plonky2 state_engine: the in-memory zkCoins state model (per-account
AccountState + CoinHist, the global NfLog accumulator) and the §2.3 two-phase
transition lifecycle (request -> awaiting_signature -> finalise) for mint, send
and receive, driving the P1-E.1 prover bridge.
begin_mint/begin_send/begin_receive build the transition witness and surface the
six ProofData fields; finalise proves via the bridge and applies the new state
transactionally. verify_incoming_transition runs both acceptance obligations plus
the canonical-NAV / size_final check against the local NfLog.
Hardening (a correctness review found these): multi-input sends and batched
receives build sequential intermediate CoinHist roots; NfLog apply is staged and
committed only after all checks pass (no partial mutation on error); finalise
binds the pending envelope (owner/mode/nav_opening/prev-state) to the witness
before proving; wrapper ProofData is re-extracted from the proof and compared
before it is trusted; token-standard-2 mint is rejected loudly until the request
API carries an explicit non-owner recipient. Circuit gate counts and digests
unchanged; a benign pub(crate) re-extract helper is added to the bridge.
Tests: single-input send end-to-end proves and applies; overspend, envelope
mismatch, transactional rollback on duplicate, forged-wrapper rejection,
multi-input sequential roots and initial receive all pass (8/8).
* half-agg: add NISSHAC half-aggregation and AggregateStateNullifierV3 codec (P1-F.1)
Add script-plonky2 half_agg: the NISSHAC half-aggregation crypto (§1.7.10 /
§3.3) over BIP-340/secp256k1 and the AggregateStateNullifierV3 inscription
payload codec (§3.5).
aggregate_sig derives z and the per-index coefficients a_j and returns
s_agg = sum(a_j * s_j) while retaining every (Pk_j, R_j). aggregate_verify runs
the single multi-scalar relation s_agg*G == sum(a_j * (R_j + e_j*Pk_j)).
comm_retrieve / comm_verify implement the receiver's sign-to-contract opening
R == R' + t*G with the unreduced-tweak rejection. verify_single is the plain
BIP-340 check. Canonical x-only/scalar encodings are enforced everywhere;
off-curve, infinity, non-canonical, or over-order values are rejected.
AggregateStateNullifierV3 serialize/deserialize follows the §3.5 layout
(42-byte header + format 0x00 raw / 0x01 half-aggregated body) with fail-closed
parsing (rejects wrong version/format, count over/underrun, trailing bytes,
truncation, and format 0x00 count != 1). The Taproot envelope and the
block_anchor-vs-inclusion bound are the scanner's job (P1-F.2 / P1-G).
Reuses the prover bridge's BIP-340/S2C EC helpers (promoted to pub(crate)).
Tests: NISSHAC completeness (k=1,2,3), tamper/non-canonical rejection,
comm_verify round-trip, payload round-trip and malformed-payload rejection, and
a measured payload-size report (k=1:138B, k=10:714B, k=100:6474B). 7/7 pass.
* inscription: add Taproot commit/reveal construction and payload extraction (P1-F.2)
Add script-plonky2 inscription: builds the §3.5 Taproot commit/reveal pair that
carries an AggregateStateNullifierV3 payload in an OP_FALSE OP_IF envelope, and
the fail-closed payload-extraction primitive scanners use to read it back.
build_envelope_script splits the payload (marker 0x42 0x42) into minimal pushes
of at most 520 bytes inside a single OP_FALSE OP_IF ... OP_ENDIF leaf.
extract_payload_from_input implements §3.5 exactly: it ignores the annex, rejects
key-path spends, decodes the control block and verifies it commits the executed
Tapscript to the prevout, then concatenates the envelope pushes and returns the
payload iff it begins with the marker. Fail-closed: a non-minimal push, an
over-520-byte push, a non-data opcode in the body, or a second marker envelope in
one leaf makes that input carry zero nullifiers. The block_anchor and
first-occurrence checks remain the scanner's job (P1-G).
Adds rust-bitcoin 0.32.5. Tests (in-memory, no node): commit/reveal round-trip
extracts and deserializes the aggregate; a large payload splits across bounded
pushes and round-trips; every fail-closed shape is rejected (8/8).
* publisher: add half-agg batch publishing over a live Bitcoin node (P1-F.3)
Collects state-nullifier signatures, half-aggregates them into an
AggregateStateNullifierV3, inscribes the payload into a Taproot
commit/reveal pair and broadcasts both to bitcoind over cookie-authenticated
RPC.
The commit output's internal key is the BIP-341 NUMS point, so its key path
is provably unspendable and the envelope leaf is the only way to spend it.
Funding is restricted to segwit inputs: build_inscription emits a commit
transaction with an empty funding witness while the reveal already references
commit_txid, so a legacy scriptSig would change that txid when signed and
invalidate the pre-built reveal. Fees are sized in two passes against the
measured vsize of the signed transactions, with a drift assertion before
broadcast.
Covered by unit tests for the guards and by a live regtest round trip that
broadcasts, confirms and reads the payload back out of the mined reveal.
* publisher: close six review findings before freeze
Critical: an oversized batch could broadcast a commit whose reveal exceeds
Bitcoin's weight limit. Since the commit output's key path is the NUMS point
and its only leaf would then be unminable, that value is permanently
unspendable. Both transactions are now checked against MAX_STANDARD_TX_WEIGHT
before anything is broadcast, and max_half_agg_members_for_standard_reveal
reports the batch size that fits. Batches are never auto-split: composition
decides which nullifiers land together and stays with the caller.
The aggregate is now verified with aggregate_verify against the publisher's
own network m_state before any transaction is built. Previously only the
canonical point and scalar encodings were checked, so a member signed for a
different network was inscribed and paid for while the scanner discarded the
whole batch.
The block anchor is now the oldest member's verified build tip, per the
section 3.5 rule that it must be an ancestor of, or equal to, that tip.
Members carry their proof-time tip in BatchMember. A stale tip fails loudly
rather than being silently replaced with a fresher one.
fetch_reveal_payloads is removed in favour of fetch_reveal_payload_details:
the short form dropped per-input extraction errors, making a fully malformed
reveal indistinguishable from one carrying no inscription.
Fee sizing is now a fixed-point iteration over fees and change topology
instead of two rigid passes, so a batch whose change falls below dust after
recalculation still publishes. Funding is restricted to input types with a
predictable witness size (v0 P2WPKH, v1 P2TR key path), and selection walks
to the next candidate when one cannot cover the final measured fees.
* publisher: close five confirmation-review findings
Every member's build tip is now validated against this node's chain, not just
the selected one: each height must not exceed the tip and each hash must be the
canonical block at that height, with the lookups cached so a large batch does
not issue one call per member. The doc-comment now states plainly that
build_tip is a caller assertion the publisher cannot cryptographically bind,
since NISSHAC does not commit it and the payload has no per-member tip field.
A dishonest submitter can overstate freshness; the scanner cannot observe that
from the aggregate, so there is no consensus divergence, and closing the gap
needs proof-layer tip supply.
connect now compares bitcoind's reported chain against the configured network
and fails loudly on mismatch. Previously a testnet-configured publisher would
verify and broadcast against a regtest node while a conformant regtest scanner
discarded the whole aggregate.
Publication uses an effective anchor gap of 94 rather than 100, leaving six
blocks of margin for inclusion delay, and the tip is re-checked immediately
before the first broadcast. At the old bound a single block of delay between
selection and inclusion made a batch carry zero valid nullifiers while its
fees stayed spent.
Funding candidates are now rejected only against their measured requirement.
The provisional estimate seeds the iteration and no longer gates admission, so
a UTXO that funds the real transactions is no longer discarded against an
inflated estimate, and the error no longer claims a measurement that did not
happen.
The fee fixed point records the states it has already seen, so a repeating
fee/topology pair is detected instead of relying on branches that could not
fire.
* publisher: close six freeze-review findings
Testnet is bound to Signet exactly, as the spec pins it. The previous mapping
also accepted testnet3 and testnet4, so a publisher could broadcast where no
conformant testnet scanner was watching.
The build-tip documentation is corrected. It previously claimed the aggregate
signature covers block_anchor and that proof-time height is cryptographically
bound in the per-account proof; neither is true — aggregate_verify ignores
block_anchor, the coefficients cover only (R, Pk) members, and ProofData has no
height field. It also understated the consequence: a submitter claiming a
newer tip than the proof was built against turns a batch that the truthful
anchor would have made ineligible under the gap bound into an admitted one.
Scanners agree with each other, but the rule is not enforced here, so this is a
trust assumption on the caller.
The pre-broadcast guard now re-verifies the anchor's identity rather than only
its height gap, bypassing the lookup cache. A same-height reorg during fee
convergence previously passed the guard, both transactions went out, and the
scanner rejected the aggregate for a non-ancestor anchor.
The inclusion-delay margin moves into the configuration, validated against the
maximum gap, and its effect is documented: with margin m, inclusion up to m + 1
blocks after the check still satisfies the bound, and beyond that the batch
carries no nullifiers while its fees stay spent. The documentation also records
that the commit transaction cannot be fee-bumped, since bumping it would change
commit_txid and invalidate the pre-built reveal, so an adequate fee rate chosen
up front is the only lever.
Member count and reveal weight are validated before any per-member chain
lookup, so an unpublishable batch costs a constant number of calls instead of
one per member. Funding candidates that cannot cover the theoretical minimum
are skipped arithmetically before anything is built or signed, and the number
of constructed attempts is bounded and reported.
Rejections are classified as measured or unmeasured, so a candidate rejected
before any transaction was built is no longer reported as failing to cover
measured fees.
* publisher: bound anchor-selection RPC cost, correct the fee-bump note
Anchor selection applied the height window only after looking up every
member's claimed height. A weight-valid batch of thousands of genuinely
correct historical heights therefore issued one getblockhash per height and
was rejected only afterwards for staleness — finite, but hours of synchronous
work against a slow node. The window is now applied arithmetically on the
caller-supplied heights before any lookup, so a rejected batch costs a single
tip query and an accepted one at most as many lookups as the window is wide,
regardless of member count. Chain-existence and future-ness validation of
every surviving member is unchanged.
The commit-fee note claimed an adequate upfront fee rate was the only lever.
The commit does signal RBF, and because the reveal carries no signature a
replacement commit can be paired with a newly built one; the reveal can also
serve as a replaceable CPFP child. The note now states that no fee-bump path
is implemented here rather than implying none exists.
* scanner: rebuild the nullifier accumulator from Bitcoin (P1-G)
Implements the five mandated steps of section 3.6 over confirmed blocks:
discover script-path inputs whose executed leaf carries a zkCoins envelope,
parse and bound-check the payload, verify signatures against the per-network
m_state, order the survivors by (height, tx_index, vin_index, member_index),
and fold them into the accumulator by first occurrence.
The anchor bound is enforced in all three parts: strictly below the inclusion
height, gap at most 100, and the claimed hash must be the canonical block at
that height, which is what makes strict ancestry real rather than a height
comparison. A signature failure discards the whole payload rather than the
offending member, as step 3 requires. Malformed inputs carry zero nullifiers
and are recorded with a reason instead of aborting the block.
Reorgs are handled by canonical replay against the existing accumulator, and
a broken finality assumption is surfaced to the caller rather than absorbed.
The RFC-6962 log and the accumulator itself already exist in shared and are
consumed unchanged; this module is the Bitcoin-side bridge to them.
Covered by unit tests for the bound predicate and the discovery pre-filter,
and by live regtest tests that publish real inscriptions and scan them back,
including a double spend of one account state across two batches and a real
chain reorganisation.
* scanner: separate data failures from infrastructure failures
The accumulator must be a pure function of the confirmed chain, so a failure
that is not derivable from chain data must never influence it. The module
conflated two kinds: a malformed envelope or a failed signature is a
deterministic property of block content that every honest node sees alike,
while an RPC timeout or a lagging index is a property of this node's
environment. Both were recorded as rejections and the block was then
checkpointed, so a node that hit a transient error permanently omitted a
nullifier its peers admitted.
Data failures and infrastructure failures are now distinct types, and a
rejection can only be constructed from the former. An infrastructure failure
aborts the scan and leaves the checkpoint untouched, so the work is retried.
connect now requires txindex to be enabled and fully synchronized, and rejects
an activation height that disagrees with the pinned network value rather than
substituting it. A scanner that cannot resolve prevouts, or that starts from
the wrong origin, cannot produce a correct accumulator, so it refuses to start
instead of producing a wrong one.
Forward scanning verifies that each block links to the previously processed
one. A reorg during a scan previously mixed blocks from two forks into one
accumulator and left it undetected, and the outcome depended on how the scan
was split into calls.
Reorg replay is atomic: the replacement range is collected in full before any
state is touched, so a failure partway leaves the scanner unchanged instead of
losing a canonical nullifier on retry.
Replacement-block rejections and duplicate positions are reported rather than
discarded, and parent transactions and anchor hashes are cached per scan run,
with the anchor cache dropped on reorg.
* scanner: hold the reorg path to the forward path's guarantees
The forward scan was hardened last round and the replacement path was not, so
it revalidated payloads without the guarantees the forward path had gained.
Both now share one block-collection implementation, so a future fix cannot
land in only one of them.
The anchor-hash cache was cleared only after replacement collection, so
replacement blocks were validated against getblockhash answers from the fork
that had just been orphaned. A payload anchored to the new block at that
height was rejected while one naming the orphaned hash was accepted, and since
the signature does not bind the anchor the latter is trivial to construct. The
cache is now invalidated before any replacement validation begins.
Replacement collection now verifies block linkage as the forward path does. A
second reorg during collection could otherwise mix blocks from two forks into
one replay stream, and a canonical final block left the checkpoint consistent
with the live chain so nothing detected it afterwards.
A reorg occurring after the last forward block could return a tip the
accumulator did not reflect, which matters because membership answers are only
meaningful relative to a stated tip. The reported tip is now the one the log
actually corresponds to.
Reorg outcomes are merged across a call with sticky finality_broken and summed
displaced counts, so a shallow reorg can no longer erase an earlier finality
break from the report.
Replacement admissions and duplicates are reported like forward ones, and the
live test now reorgs onto a replacement block that carries both a new winner
and a duplicate.
Atomicity is per block rather than per call: a block is fully processed and
checkpointed or not processed at all, an infrastructure failure resumes at the
failing block, and the error carries the completed prefix's report so nothing
observed is lost. The previous per-call claim was not true.
* scanner: decide anchor admissibility from the inclusion block's ancestry
Anchor validation resolved getblockhash against whatever chain was active at
that moment, while validating a transaction from a specific inclusion block.
If the chain moved to a competing fork and back during a scan, the anchor was
checked against the other fork: a valid payload was rejected, or one naming an
orphaned hash admitted, and because the chain was canonical again by the final
check nothing detected it and the wrong decision was checkpointed. Two nodes
then disagreed permanently on entries, positions and the root.
Clearing the cache when a reorg is detected could not fix this, because that
interval leaves nothing to detect. Section 3.5 asks whether the anchor is an
ancestor of the inclusion block, which is a property of that block's own
ancestry rather than of the current tip, so the check now answers exactly that
question: against the verified chain the scanner walked, and otherwise by
walking back from the inclusion block, at most as far as the maximum gap. Walk
results are keyed by the descendant they were derived from, so an entry cannot
be reused under a different fork.
Activation height is taken from the supplied network parameters instead of a
compile-time constant, so mainnet and signet scanners can start at all; their
pinned value is observed at deployment and published, and only regtest is
fixed at zero.
Duplicate handling keeps a public-key to winner-position index, replacing a
linear scan over all prior survivors that a single aggregate could repeat into
quadratic work.
Every error path after the first committed block now carries the partial
report through one exit, so a failure in a post-scan tip query can no longer
discard the reports of blocks already folded and checkpointed.
* scanner: verify the parameter set against its published identity
The activation origin was compared against another value from the same caller,
so it confirmed nothing. A scanner configured with a wrong height and a
matching hand-made parameter set connected successfully while a peer using the
published values started elsewhere, which is exactly the divergence the scan
origin rule exists to prevent.
The parameter set is content-addressed, so its identifier is the external
truth: connect now requires the operator to supply the published identifier and
compares it against the set's own. Any difference in any field, including the
activation height, changes that identifier, so a node can no longer diverge by
accident. The set's network tag must also correspond to the configured
network, and regtest stays pinned to zero.
The ancestry walk resumes from the deepest cached ancestor of the inclusion
block instead of restarting from the block each time. Because anchor resolution
precedes signature verification, one block carrying payloads with increasing
anchor gaps could otherwise force thousands of synchronous block queries
without holding a single valid signature. The walk cache is bounded with a
deterministic eviction policy, since an evicted entry only costs a re-walk
while an unbounded one costs memory an attacker chooses.
Prevout lookups return the single output they need rather than cloning the
whole parent transaction on every cache hit.
* accumulator: maintain the log root incrementally
Folding recomputed the Merkle tree head over the whole log after every
admission, so building N entries cost quadratic hashing and a reorg replay
repeated it. The chain scanner is exactly the bulk caller the old doc-comment
warned away from, since it folds every nullifier published from the activation
height onward, so the accumulator could not have synced a real chain.
The root is now carried as the set of perfect-subtree roots and updated in
logarithmic time per append. nflog_mth is untouched and remains the normative
reference, which is also what the equivalence test compares against: the
incremental head matches it byte for byte for every size from zero to three
hundred and at the power-of-two boundaries where split and peak-bagging
behaviour changes. Inclusion and consistency proofs verify against the
incremental head unchanged, and a replay ends in the state a fresh sequential
fold produces.
Measured on twenty thousand entries: 298 ms incremental against a projected
980 s for the previous behaviour.
* nflog: keep protocol sizes in u64 end-to-end
The log verifiers took u64 sizes and narrowed them with `as usize` after the
guards had already run on the untruncated values. On a 32-bit target that
narrowing is silent: verify_inclusion(leaf, 0, [], 2^32+1, leaf) collapses the
size to 1 and accepts, and verify_consistency(1, X, 2^32+1, X, []) collapses
both sizes to 1 and accepts a non-prefix claim.
This is not a hypothetical target. wasm32 has a 32-bit usize, and these
primitives are the ones a wallet would compile there.
Carry u64 through split_point, verify_path_range and verify_subproof instead of
rejecting out-of-range values, so the defect class disappears rather than the
instance. The remaining `as usize` conversions are bounded by entries.len().
No emitted digest changes; the generated vector files are byte-identical.
* vectors: generate the spec's <REGEN> values from the reference implementation
The specification pins its Poseidon-derived values as <REGEN> placeholders: no
one may hand-author them, because a wrong vector would lead two implementations
to agree on something invalid. This generates all of them from the live
primitives, plus the two circuit digests the spec carries as protocol constants.
Blocks:
- Poseidon values (22), extended with detect_tag over the pinned V.10 fixture
- circuit digests for C and C_balance, one per network tag per §589,
encoded via digest_to_bytes (§1.7.1) rather than the bincode form
- V.11 log vectors: mth@n, nav_root@n, twelve inclusion and five consistency
paths over the spec's pinned sample-leaf sequence
- V.5 transition signatures (BIP-340 + sign-to-contract) and the V.6
aggregate scalar s_agg
s_agg reproduces the value V.8 already pins, which makes it the one vector in
this set with an independent reference to check against.
The V.11 boundary suite now feeds one fixture generator into all three
consumers -- the independent RFC-6962 reference, the host verifiers, and the
in-circuit gadget -- as §1.7.8 requires, covering every size bit k = 0..63 for
both accept and reject. Previously it exercised only the host verifiers, and
its peak-bagging block compared the reference against itself. Negative cases
mutate exactly one property instead of reseeding the fixtures; cases that
cannot be built that way are listed rather than dropped.
The independent reference lives behind a test-fixtures feature so a normal
build neither compiles nor exports a second RFC-6962 derivation path.
Every value is computed; none is written by hand.
* nflog gadget: represent log sizes canonically
The gadget carried a log size or position as one Goldilocks target and
decomposed it with split_le(x, 64). Goldilocks has p = 0xffffffff00000001,
which is below 2^64, so that decomposition is not injective: the bit patterns
of 1 and of p + 1 = 0xffffffff00000002 satisfy the same constraint.
A malicious prover could exploit the alias directly. Bind a root computed for
size p + 1, then let the verifier read the same target as 1, enter the n == 1
base case, and accept an empty inclusion path. Both decompositions are
satisfiable, so an honest witness generator choosing canonical bits does not
close it -- soundness is about what a prover can satisfy.
Route every size-carrying target through a canonicity check, so a second
representative is unsatisfiable. Sizes stay 64-bit; §2.5 sets H_MAX = 64 and
narrowing the supported range would not have been a fix.
The soundness test injects the p + 1 bit assignment onto the wires directly,
which the high-level witness API cannot express, and asserts the proof fails.
This changes the shape of C and C_balance and therefore their digests. That is
possible only because those digests are still <REGEN> in the specification:
once pinned, §1.7.8 makes any change to the frozen circuit surface a new
protocol version.
Host-side vectors are unaffected and byte-identical.
* nflog boundary suite: count only what actually ran, and build the missing negatives
The peak-bagging block incremented host_accept and gadget_accept without
calling either implementation, and compared the reference against itself. All
192 peak cells were credited to three layers while one layer had run. A count
that overstates coverage is worse than a reported gap, because it stops anyone
from looking.
Peaks now reach host and gadget through the consistency path that consumes
them. What genuinely cannot reach all three is counted separately as ref_only
rather than folded into a number that reads as three-layer agreement.
The suite also skipped 131 negatives as unconstructible because swapping is a
no-op on a single peak or chunk. NL-B2 also permits a root to be dropped or
duplicated, which changes the bagging property alone and rejects cleanly, so
those cases exist now -- including every adjacent (2^k, 2^k+1) consistency case
with one mth_a chunk.
Honest counts, per layer actually executed:
accept ref=631 host=631 gadget=631 (+191 reference-only, reported)
reject ref=2260 host=2260 gadget=2260 (+132 reference-only, reported)
Remaining skips are 71, each genuinely unconstructible at its size. No layer
disagreed, and the new cases surfaced no regression from 302fe30.
* circuit: carry protocol sizes as two u32 limbs end to end
The canonicity check added in 302fe30 closed the split_le alias by proving the
reconstructed integer is below p. That makes the decomposition unique, but it
also makes every value in p .. 2^64-1 unwitnessable -- 2^32-1 valid protocol
values. §2.5 sets H_MAX = 64 and the log supports sizes through 2^64-1, so a
size of exactly p became a legitimate value with no constructible proof. One
soundness hole traded for a liveness hole.
The test suite could not see it: its largest size is 2^63+1, and p is above
2^63, so the entire affected band lay beyond what 36 minutes of tests reached.
Introduce U64LimbsTarget { lo, hi } with both limbs range-checked to 32 bits,
and keep protocol numerics in that form through comparison, borrowing
subtraction, the split-point derivation, the bit-driven recursion and the
big-endian byte encoding. Deriving limbs from a single field target after the
fact is not enough -- the moment a u64 passes through one Goldilocks element,
either the alias or the narrowing returns.
Public input layout is unchanged: C stays at 108, C_balance at 60. Sizes,
positions and counters are private; size_ceiling was already two public u32 and
now has the limbs as its representation rather than a derived view.
New tests cover the previously invisible band -- n at p-1, p, p+1 and 2^64-1,
with positions and adjacent consistency pairs -- and a non-canonical limb
outside u32 still rejects.
Cheaper as well as more correct: the gadget drops from 6669 to 1825 gates for
inclusion and 6849 to 2005 for consistency, degree bits 13 to 11, because two
32-bit range checks cost less than a 64-bit split plus a comparison against p.
smt.rs and main.rs carry the same defect class and are deliberately untouched;
neither is reached by C or C_balance. Host-side vectors are byte-identical.
* vectors: regenerate the circuit digests after the limb redesign
The digest is a function of the circuit's shape, so the canonical size
representation changed all six values. The previous set described a circuit
that no longer exists.
C drops from 1403783 to 1382481 gates and C_balance from 193437 to 191268;
degree bits are unchanged at 21 and 18. The saving exceeds the isolated
gadget's 4844 because the inclusion and consistency gadgets are instantiated
several times per proof.
Build: 3652s.
* node: add v1.1 persistence and a flag-gated path to the StateEngine
First cutover block. The StateEngine was in-memory only, while the running node
persists the retired model: global SMT and MMR, account blobs carrying the
legacy Proof, commitment payloads in pending_inscriptions. The v1.1 model needs
different state entirely -- the NfLog accumulator, per-account CoinHist, and
ComplianceProof blobs -- and the old global structures have no successor by
design.
Everything here is additive. New tables alongside the existing ones, nothing
dropped, nothing migrated; a v1.1 node runs against a fresh database. The
legacy path stays the default and is unchanged.
Selected by ZKCOINS_PROVER=v11; unset, empty and 'legacy' all resolve to the
legacy path, tested without mutating the environment. If the v1.1 path is
selected and a component it needs is missing, it fails loudly rather than
falling back -- a node that silently proves with the wrong circuit would look
healthy while producing proofs no v1 verifier accepts.
The nullifier index is keyed by Pk rather than by (Pk, R), matching §3.6
first-occurrence folding and the NfLogAccumulator rather than a generic
seen-set.
Restart identity is what the persistence tests assert: after a full reload, the
NfLog root, its size, and every account's CoinHist root are byte-identical. A
test that only round-trips blobs would not establish it.
ProverBridge::new becomes lazy so persistence tests need no circuit build;
digests and proofs are still constructed on demand. circuit_digest_bytes() is
exposed on the bridge for self-heal parity, using the canonical §1.7.1 encoding
rather than the legacy bincode form.
* node: make the shadow flag honest and close four persistence defects
A review found the flag claimed more than it did. ZKCOINS_PROVER=v11 reported
'Prover mode: v11' and then unconditionally built the legacy Prover, loaded
legacy state, ran legacy self-heal and exposed legacy REST proving. Deferring
the prover swap to stage 3 is the plan; claiming it had happened is not. The
flag is now ZKCOINS_V11_SHADOW and both its name and its startup message say
what it does: v1.1 state is maintained alongside the legacy path, proving stays
legacy.
Boot now validates the pinned consensus parameters. It previously accepted any
non-negative activation height without checking the parameter set against its
published identity -- the same shape of empty check the scanner already avoids
by carrying an independently pinned, content-addressed identifier. It uses that
mechanism rather than a second one.
Reads are snapshot-consistent. The write path already replaced all tables in
one transaction, but a concurrent load could observe old and new rows together
and reconstruct a state that never existed. Loading now runs at REPEATABLE READ,
so a reader sees either the whole old or the whole new state.
A database with v1.1 rows but no meta row used to load as an empty engine. That
is a silent fallback producing a plausible wrong state, so it now fails loudly;
genuinely empty still loads empty.
The tip cursor stores the block hash beside the height, so two forks at the same
height are distinguishable after a reload. Migration 0019 is unreleased and was
edited in place.
Found while fixing: from_engine would have zeroed the tip hash on every adapter
persist, so reloads could drop it.
* style: apply rustfmt to the rest of the workspace
Formatting only, no semantic change. Earlier blocks were committed without a
formatting pass, so 'cargo fmt --all --check' failed across files unrelated to
the current work. Keeping this separate leaves the preceding commit readable.
* node: run the v1.1 publisher and scanner as an exclusive alternative stack
Two publishers existed side by side and the binary used only the legacy one. It
writes a bincode commitment, and the scanner callback folds that commitment into
the global SMT, so the double-spend enforcer is first-write-into-SMT. The v1.1
stack publishes AggregateStateNullifierV3 with NISSHAC half-aggregation, and its
enforcer is the NfLog accumulator with §3.6 first-occurrence folding.
Those are different on-chain objects with different double-spend semantics, so
the two must never reach the same accumulator or the same database. That is
enforced structurally rather than by convention: a stack marker is persisted,
and a node refuses to boot when the marker and the selected stack disagree --
legacy data with the v1.1 stack, or v1.1 data with the legacy stack, both fail
with an explicit refusal.
Behind the shadow flag the node now publishes via script-plonky2 and scans into
the NfLog. The fold is tested against a shuffled multi-member inscription
including a duplicate Pk, where first occurrence must win.
Deliberately still open, to be closed by later blocks: the prove path still
produces commitments under the flag, so publishing is refused rather than
silently downgraded until stage 3 wires it; receive remains legacy bookkeeping
(G3); wallet signing is untouched (G4).
Operational note the plan missed: the node reads the chain through Esplora,
while the v1.1 scanner and publisher speak bitcoind cookie RPC. A v1.1 node
therefore needs a bitcoind, which is a deployment requirement rather than a code
gap -- the publisher has needed a wallet-capable node since it was built.
* node: bind the stack separation to the database and survive a restart across a reorg
A review found the separation was advisory. The boot check knew two tables,
four writers bypassed it, and the marker was claimed in a transaction of its
own before validation -- so the window between checking and writing was exactly
where the bad states arose. The marker is now validated inside the same
transaction as every write of v1.1 scan state, and a missing marker is an
unconditional refusal whenever either stack's data exists; only a genuinely
empty database may claim a stack.
Publishing refusal was not total either. create_and_broadcast_inscription was
guarded but resume_pending_inscriptions was not, so a v1.1-claimed database
holding an old or injected pending row could still broadcast a bincode
commitment through Esplora. Every publishing entry point is guarded now,
recovery included.
The worst defect was invisible to any test that did not look for it. Each boot
built a fresh scanner at activation_height with an empty folded_keys set, so a
reorg that happened while the node was down went unnoticed: the new canonical
stream was folded into an NfLog still carrying the old fork's first-occurrence
winners, diverging from consensus with nothing raising an alarm. Boot now
reconciles the persisted tip hash against the chain before folding anything,
and replays from the last common ancestor rather than continuing. The test
asserts the restarted node reaches the same accumulator a continuously running
node would hold.
finality_broken is honoured rather than ignored, and readiness reflects the v1.1
scan state when that stack is claimed.
The full suite passes at 479 tests once the environment it needs is set
(PUBLISHER_KEY, IS_MAINNET, ESPLORA_URL, ESPLORA_WS_URL, USERNAME_DOMAIN). The
13 failures reported earlier are pre-existing: lazy statics panic without those
variables and poison the tests that follow.
connect_v11_publisher still has no call site in the binary; wiring it is stage 3,
after which bitcoind with txindex and a wallet becomes a deployment requirement.
* node: fail-stop on unrecoverable reorgs, and guard broadcast structurally
A third review round found the reconciliation was cosmetic. Boot compared only
the canonical hash at the persisted tip height and, on any mismatch, replaced
everything, cleared the replace flag and reported ready. It never resolved the
persisted hash itself, never found a common ancestor, never measured depth, and
never noticed displaced final positions -- so an offline reorg of six blocks or
more was silently repaired. §3.9 requires fail-stop for displaced finality, not
recovery.
Boot now resolves the persisted tip, walks back to the last common ancestor and
measures the depth. Below activation height, beyond the recoverable limit, or
with any previously final position displaced, it refuses: no fold, not ready,
explicit error. Only a shallow non-final reorg replays from the ancestor.
Emptiness was decided over one table while three others carry durable legacy
state, so a legacy database could look empty and be claimed by v1.1. It is now
decided over every durable table of both stacks, in the same transaction that
claims the marker.
Two more public broadcast paths were unguarded. Rather than patch entry points
a third time, the guard moved to the internal choke point every path traverses
before the client broadcasts, so a new caller cannot omit it.
The reorg test's oracle was circular -- it built its 'continuous node' by
calling the replace path under test. It now folds the canonical stream
sequentially by first occurrence instead, so the two sides can disagree.
Full suite: 484 passed.
* node: close the claim race, make an unguarded client unobtainable, stop over-refusing
Three defects from the fourth review round.
The claim could still race a legacy writer: the emptiness check saw both stacks
empty, a concurrent legacy write committed SMT/MMR state, and the claim then
committed mode=v11 over a database that was no longer empty. All three legacy
writers -- persist_state_tx, persist_state_and_mark_complete_tx and
insert_root_index -- now carry the same capability check inside their own
transaction, so no interleaving can produce a mixed database.
The broadcast guard was not a choke point for the third time: boot recovery and
the recover_inscription binary both reached the client directly. Rather than
guard two more call sites, the client itself is now the guard. connect(url) is
the only public construction and checks before building; the inner Esplora
client is private with no escape hatch. Possessing a broadcast-capable client
therefore implies the check already happened, including from a separate binary.
The reconciliation had become too strict in two places, which is the failure
mode that closing a soundness hole tends to produce. A one-block reorg of the
activation block has its ancestor at activation_height - 1 and was refused as
'below activation', although below activation the NfLog is empty by §3.6 and a
rescan is exactly a replay from activation. And an RPC node whose tip sits
below the queried height was treated as divergence -- 'I do not know yet' read
as 'the chain says otherwise'. Behind is now distinguished from diverged: an
incomplete view refuses rather than guessing in either direction.
Still refused, deliberately: depth beyond §3.9's limit, an unresolvable or
pruned tip, no common ancestor down to genesis, an ambiguous cursor, and any
RPC failure.
Full suite: 488 passed.
* node: separate transient from fatal, and put the raw client out of reach
Fifth review round on this block.
Reconciliation and the first scan observed different chains: reconciliation ran
before the scanner's first pass, so a reorg landing between the two was
invisible -- the first report carried no reorg and the node took the forward
path. Both are now bound to one observation.
The recovery binary still obtained an unguarded client, and esplora-client
remained a direct dependency with raw clients built in production code. While
the raw type is reachable, a wrapper is a convention rather than a boundary, so
the dependency now lives behind a module that exposes only the guarded type.
The test exercises the binary's own path instead of setting the flag by hand.
Two mutators bypassed the claim invariant: claim_stack_scan_mode, a public
'test helper' compiled into production that inserted the marker without
checking either stack, and reset_proof_dependent_state_tx, which deleted all
four legacy tables with no capability check.
The response to a lagging bitcoind was wrong even though the reasoning was
right. Treating an incomplete view as 'not divergence' is correct, but the node
then errored out and marked finality broken, so a node whose bitcoind was still
syncing could not start at all and a transient lag looked like a finality
violation. Reconciliation now returns a typed outcome: Ready for fresh, still
canonical and shallow reorg; RetryableIncompleteView for an RPC behind or
unreachable, which stays unready and backs off without touching the deep-reorg
flag; and a hard error only for an unresolvable tip, a missing ancestor, depth
beyond §3.9, or RPC infrastructure failure.
Found while fixing: self-heal under the v1.1 claim no longer wipes legacy state,
it only updates the digest.
Full suite: 493 passed.
* node: reconcile against immutable ancestry, and enforce the client boundary by compilation
Sixth review round on this block.
The boot observation had an ABA race. The first scan captured chain A, then
reconciliation queried the mutable live chain height by height, so an A→B→A
sequence passed every check: persisted state from B, scan on A, reconciliation
seeing B and reporting still-canonical, the chain returning to A before the
final pin. Stale B fold keys were then seeded and A survivors appended, and
because the checkpoint already read A no later scan reported a reorg -- the
mixed accumulator was permanent.
This class was solved once before here. The scanner's anchor validation had the
same defect, and two fixes based on better cache handling both failed because
A→B→A leaves no detectable reorg. The answer then was to validate against the
immutable ancestry of a fixed block rather than the live tip, and it is the
answer now: classification runs purely over the captured scan tip's ancestry,
using getblockcount and getblockheader by hash. Sampling the live tip twice is
not an ABA defence and is kept only as a secondary 'tip moved' retry.
Self-heal under a v1.1 claim preserved exactly what a reset exists to clear.
Skipping the legacy SMT/MMR wipe is correct and stays; keeping stale
proof-bearing account rows was not, and came from my own instruction last round
being too broad. A v1.1 reset now clears legacy account and proof state while
leaving the structures v1.1 does not use untouched.
A genuinely behind node was classified fatal: the persisted hash was resolved
before checking whether the live node had reached that height, so a restored
bitcoind that does not know the hash yet failed before it could report being
behind. The order is reversed; an unknown hash is fatal only once the node is at
or beyond that height.
The client boundary was organizational. esplora-client stayed a normal
dependency of the node package, so every binary target could construct a raw
client, and the boundary test searched text rather than types. The dependency
now lives in a separate esplora-bound package that exports only the guarded
wrappers, and a compile-fail test asserts the raw type is not in scope.
Full suite: 493 passed.
* node: require a witness to construct a broadcast-capable client
Seventh round on this block, and the last open defect from it.
Hiding the raw esplora-client type stopped anyone naming it, but the facade
still exported an unguarded connect() while the claim check sat in the node-side
wrapper. Each earlier round had moved the type one level deeper and left the
check where it was, which relocates a hole rather than closing it.
The capability is now part of construction. The facade's connect() requires a
witness value whose constructor is gated behind a feature only node enables, and
ensure_legacy_publisher_allowed returns that witness after validating the claim.
Possessing a broadcast-capable client therefore implies the check ran, enforced
by the compiler rather than by convention, and a compile-fail test asserts a
client cannot be built without one.
One residual remains, reported rather than hidden: inside the node crate itself
a caller could invoke the witness constructor directly instead of going through
the claim check. Production paths do not, and closing it fully would mean
merging the witness into the claim-check module at the cost of the package
boundary that keeps raw esplora-client out of reach. The trade is worth stating
before it is decided.
Both known follow-ups are untouched: boot reconciliation still rejects every
reorg of depth six or more while the live path differs, and a bitcoind tip below
activation height is still treated as fatal.
Full suite: 493 passed.
* node: move the broadcast policy into a shared crate, drop the witness
Final defect on this block. The witness closed the cross-crate hole but was
public and feature-gated, so a caller inside node could construct it directly
instead of going through the claim check -- possession no longer implied the
check had run.
Policy and construction are now co-located. A new zero-dependency stack-policy
crate holds the process mode registry and ensure_legacy_publisher_allowed, and
esplora-bound calls that check inside its own broadcast-client constructor. The
witness type and its feature are gone. Every construction of a broadcast-capable
client therefore runs the same check, from any crate including node itself,
while raw esplora-client stays confined to esplora-bound. There is nothing left
to forge and nothing to forget.
The policy crate stayed small -- the process claim is plain mutex state, so no
dependency cycle appeared.
Two things surfaced about the test suite while verifying, neither related to
this change. The shared postgres container had accumulated roughly 4140 leftover
schemas and 485k relations from aborted runs, and the resulting catalog thrash
cost minutes per test; recreating it fixed that. And api_remote is an end-to-end
suite against a deployed node, which CI runs only in the deploy workflows and
excludes elsewhere -- included by mistake it fails 49 times with HTTP 502.
Excluding it is now part of the documented invocation.
Suite: 506 passed, 0 failures outside api_remote.
* node: make the process claim monotonic and unstall the test harness
The claim was not monotonic in production: clear_process_stack_mode_for_test was
public and compiled in, so a caller could withdraw a claim, obtain a
broadcast-capable client under no claim, and re-set it -- leaving that client
valid while components disagreed. The reset now exists only behind a
test-support feature, and a compile-fail test asserts it is unreachable from a
production build.
The test harness stalled for anyone running the full suite: after 193 seconds a
reviewer saw 6 tests passed, 18 hung and 482 not started. The shared postgres
container had accumulated roughly 4140 schemas and 485k relations from aborted
runs, and the attach-or-create path had no timeout. Container readiness now
times out at 90s, pools are smaller, teardown is reliable, and failures say
exactly which docker command fixes them instead of hanging.
All 507 tests complete; zero leftover schemas afterwards. One test fails under
load and passes alone in 1.5s -- scanner_ws watchdog timing. It predates this
change and is being treated as a defect in its own right, not written off as
noise.
Wall-clock is 95.6 minutes under default parallelism. With the stall gone, the
remaining cost is Plonky2-heavy tests rather than harness overhead.
* scanner_ws: fix the watchdog race the flaky test was reporting
The test passed alone and failed under load, which is usually writt…
github-actions
Bot
added
the
ci:full
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Automatic Promote PR
Commits: 1 new commit(s)