Overnight/test harness 2026 04 20#3
Merged
Conversation
Implements SPEC §1.3-1.6: the page-level format every Aggregate
companion-index torrent uses for its single-file payload.
Added:
- internal/companion/btree.go (~560 lines):
* BTreeMagic "SNAGG\0" + 16-byte PageHeader.
* Record type + bencoded wire form (pk, kw, ih, t, pow, sig)
with ed25519 verification over canonical sig message.
* RecordKey = kw || 0x00 || ih for leaf ordering and interior
separator selection.
* Interior/root page encode/decode with varint-length separators
and LE uint32 child piece indices.
* Leaf page encode/decode with varint-length record bytes.
* Trailer page carrying publisher pubkey, seq, commit
fingerprint, per-page counts, and ed25519 publisher sig over
the preceding fields.
* ErrPageOverflow signal for the build path to split pages.
- internal/companion/btree_test.go (~270 lines): round-trip
coverage for records, leaves, interior pages, and trailer;
tamper detection for record and trailer signatures; magic and
version validation; RecordKey ordering invariants.
Design notes:
- Payload bytes are zero-padded to pageSize so every page
occupies exactly one BitTorrent piece. Piece hashes in the
.torrent authenticate each page; no per-page checksum needed.
- Trailer's field encoding is shared between the on-disk page
payload and the signing message, so writer/verifier cannot
disagree on format.
- Record MaxRecordBytes = 256 and MaxKeywordBytes = 64 bound the
worst-case leaf density; publishers reject oversize at build
time.
- BTreeVersion 0x01; readers MUST refuse other versions.
- MinPoWBitsDefault = 20 as spec'd; hashcash verification lands
in P5.1.
Also marks PROPOSAL.md as APPROVED and adds ROADMAP.md tracking
the 11-phase implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two independent phases shipped together since P2.1 has no deps
and P1.2 consumes P1.1's page primitives.
### P2.1 — dhtindex/ppmi.go
PROPOSAL.md §2.1 publisher-pointer mutable item:
- PPMISalt = SHA256("snet.index") — fixed 32-byte double-hashed
salt used for every publisher. A passive DHT observer cannot
learn which keywords a publisher covers; the per-publisher
discrimination happens at the pubkey, not the salt.
- PPMIValue bencoded schema { ih, commit?, topics?, ts, next_pk? }
with strict size validation (ih=20, commit=0|32, topics=0|32,
next_pk=0|32) on both encode and decode.
- MaxPPMIValueBytes = 1000 (BEP-44 cap) enforced; typical values
land at ~50-100 bytes with all fields populated.
- NextPk reserved for v1.1 key rotation, mirroring KeywordValue.
10 tests: round-trip, minimal encoding, salt known-answer,
bad-field-size rejection, garbage rejection, empty rejection,
size estimation, side-by-side coexistence with legacy
KeywordValue.
### P1.2 — companion/build_btree.go
SPEC.md §1.8 deterministic Aggregate index-torrent builder:
- BuildBTree(Input) -> Output: full file bytes ready for
metainfo.Info.BuildFromFilePath.
- Input records defensive-copied then sorted by RecordKey before
layout (callers' slice order preserved).
- packLeaves: greedy-pack sorted records into leaf pages until
the next record would overflow piece budget.
- packInteriorLevel: same greedy rule over (separator, child)
tuples, producing each interior level.
- Loop upward until the level collapses to one page → root.
- Tiny-tree case: single-leaf trees wrap with a trivial one-child
root so every output has the same shape root→…→leaves→trailer
(SPEC §1.3 keeps root kind distinct from leaf kind).
- BFS top-down piece assignment: root at piece 0, then
contiguous slabs per level, leaves last before trailer.
- Tree fingerprint = SHA256(canonical record stream) computed
independently of page layout — two publishers with identical
records produce identical fingerprints regardless of their
pieceSize choice.
- Trailer signed with publisher's ed25519 privkey; the signing
message is derived from the same encoder the on-page payload
uses, so writer and verifier can't disagree on format.
8 tests: tiny tree, determinism (identical bytes across
repeated builds), sort-invariance (shuffling input → same
output), multi-leaf decode-walk (every page valid, every child
index in range), empty rejection, bad pieceSize rejection,
fingerprint matches what a reader would derive from the canonical
stream alone.
Binaries rebuilt: dist/swartznet, dist/swartznet-gui-dev-linux-amd64.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SPEC.md §1.7 subscriber-side implementation. OpenBTree verifies
the trailer signature before returning; Find walks from root
DFS-pruning against the prefix range; VerifyFingerprint is the
optional "full integrity" check for cautious subscribers.
Added:
- internal/companion/read_btree.go:
* PageSource interface + BytesPageSource for tests & for
subscribers that choose to fully buffer the companion file.
* OpenBTree(src) validates the trailer: sig checks against
embedded pubkey, NumPages matches source size, RootPieceIndex
= 0. Any mismatch fails open before trusting any payload.
* Find(prefix) DFS-walks interior pages, pruning children
whose [lower, upper) range can't overlap [prefix, nextPrefix).
Per-record ed25519 sig is re-verified on every returned hit;
PoW is enforced iff trailer.MinPoWBits > 0.
* VerifyFingerprint re-derives SHA-256(canonical record stream)
from the leaves and matches the trailer's TreeFingerprint —
catches any post-publish leaf tampering.
* rangeOverlapsPrefix helper: nil lower = -∞, nil upper = +∞,
and nextPrefix handles byte-carry overflow producing nil.
* leadingZeroBits for PoW threshold; VerifyRecordPoW shipped
here so P5.1 can share instead of duplicating.
- internal/companion/read_btree_test.go: 16 tests including
trailer verification and rejection, exact-keyword and prefix
finds, no-match and empty-prefix corner cases, deep-tree
pruning over 1000 records spread across 100 keywords,
fingerprint round-trip, tampered-leaf detection, nextPrefix
and leadingZeroBits property tables, PoW-enforcement on/off
paths.
Also adjusted build_btree.go so MinPoWBits == 0 means "not
enforcing" rather than auto-defaulting to 20. Production callers
will pass MinPoWBitsDefault (20) explicitly in P5.1; removing the
auto-default lets unit tests build trees without having to mine
nonces. No behavior change for any existing caller (there are
none outside tests today).
Binaries rebuilt: dist/swartznet, dist/swartznet-gui-dev-linux-amd64.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SPEC.md §2.4-2.7 RIBLT implementation, no external dependency.
Deliberately minimal so the license audit stays clean and the
algorithm stays auditable.
Added:
- internal/swarmsearch/riblt.go:
* RIBLTElement [32]byte with FNV-1a-derived Key() (nonlinear —
critical for peeling correctness, see note below).
* RIBLTSymbol {Count int32, KeyXOR uint64, DataXOR [32]byte}
matching the wire format in SPEC §2.5.
* RIBLTEncoder.NextSymbol() produces the next coded symbol
deterministically; both sides compute identical symbols for
identical input sets.
* RIBLTDecoder with auto-peeling on every AddRemoteSymbol.
Tracks decoded elements and synthetic added/removed lists so
future local-symbol computations stay consistent — without
this, every post-decode symbol re-reports the same element.
* contributes() uses a 12-step geometric rate cycle (mod =
2^(1+idx%12)) instead of constant 1/3. A constant rate puts
~d/3 contributors in every symbol, so d ≥ 5 produces zero
pure symbols and peeling never starts. The graduated cycle
always has some positions with expected degree ≈ 1, which
kickstarts peeling for any plausible d.
- internal/swarmsearch/riblt_test.go: 11 tests including
Diff0/Diff1/Diff8/Diff100/Diff500 convergence (O(3d) symbols
across the range, confirmed by logged counts), deterministic
encoding between two encoders with identical input, pure-function
stability of contributes(), observed cycle-average rate ≈ 0.083.
Two bugs caught during development:
1. Initial Key() used first 8 bytes as uint64 — *linear* in
element bytes, so XOR-sums of elements trivially satisfy the
pure-symbol consistency check and the decoder hallucinated
non-existent elements into dec.decoded. FNV-1a breaks the
linearity. (TestConverge_Diff_Symmetric caught this.)
2. Initial Converged() returned true whenever residual diffSymbols
were all zero — but after decoding a sender-only element, every
FUTURE symbol where it contributes would carry its bytes
un-subtracted (we only updated past symbols). Tracking
syntheticAdded / syntheticRemoved lets effectiveLocalSymbol
compute the correct reference for each new incoming symbol.
Binaries rebuilt: dist/swartznet, dist/swartznet-gui-dev-linux-amd64.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
### P5.1 Sybil-resistance bundle
SPEC.md §1.5 calls for each Record to carry a hashcash nonce
such that SHA256(RecordSigMessage) has at least D leading zero
bits. Verify already landed in P1.3; this commit adds the miner
and wires misbehavior score constants for the sync-ingest path
that P3.2 will build.
Added:
- internal/companion/pow.go:
* MineRecordPoW(r, bits, maxIter): brute-force search for a
nonce satisfying the bit threshold. Mean cost 2^bits SHA-256
ops — ~20 ms at D=20, ~0.3 μs per op on commodity CPUs.
* SignAndMineRecord(priv, pub, kw, ih, t, bits): convenience
that orders mine-then-sign correctly (signing before mining
would invalidate the signature for any nonce other than the
one signed over).
* Refuses bits > 40 so an attacker who controls publisher
settings can't force an honest client into intractable work.
* ErrPoWExhausted sentinel for bounded-budget callers.
- internal/companion/pow_test.go (8 tests):
* Round-trip mine + verify at D=8.
* Verify rejects tampered records (PoW AND signature break).
* Zero-bits is a no-op, high-bits is refused, exhaustion is
reported deterministically.
* recordPreimage matches RecordSigMessage (single source of
truth for the fields a miner/verifier hash over).
* SignAndMineRecord composes correctly and round-trips both
verification paths.
* Mining is deterministic for identical input (lowest valid
nonce always wins).
### Misbehavior score additions
SPEC.md §2.7 requires the receiver to drop records whose
signature or PoW fails, and SPEC implies misbehavior accounting
via the existing Bitcoin-Core-style tracker. Added two new
score constants:
- ScoreBadRecordSig = 20 (Severe — strong signal of malicious
relay injecting records the peer couldn't produce).
- ScoreInsufficientPoW = 10 (Medium — likely old build or
deliberate, but drips slowly into the ban threshold).
These are ready for P3.2 to call; no ingest path touches them
yet in this commit.
### What's still pending for full P5.1
- Wiring the above score hooks into sync_records ingestion
(SPEC §2.7). Requires P3.2's sync-session handler first.
- Double-hashed salt — already shipped in P2.1 as
PPMISalt = SHA256("snet.index").
Binaries rebuilt: dist/swartznet, dist/swartznet-gui-dev-linux-amd64.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the P2.1 PPMI schema into the DHT layer. Production
AnacrolixPutter/Getter gain PutPPMI/GetPPMI methods that map
directly onto BEP-44 mutable-item put/get under the fixed
PPMISalt; MemoryPutterGetter gains the same for in-process tests.
Added:
- internal/dhtindex/ppmi_dht.go:
* PPMIPutter / PPMIGetter interfaces so callers can mock the
DHT without needing a real anacrolix/dht server.
* AnacrolixPutter.PutPPMI: encode, re-decode to interface{},
BEP-44 put with seq coordination through getput.Put. Auto-
fills Ts when the caller leaves it zero.
* AnacrolixGetter.GetPPMI: BEP-44 get traversal, verify via the
getput layer, decode PPMI value.
* MemoryPutterGetter.PutPPMI / GetPPMI backed by a separate map
keyed on SHA1(pub || PPMISalt). Coexists with the legacy
KeywordValue store — neither clobbers the other because
targets are distinct.
* ppmiStoredItem record type.
- internal/dhtindex/ppmi_dht_test.go (7 tests):
* Round-trip through MemoryPutterGetter preserves every field.
* Legacy KeywordValue and PPMI coexist; Items() snapshot
correctly covers only the legacy store.
* Missing-pubkey Get returns a clear error.
* Ts auto-fill behavior (documented divergence between memory
and production paths — memory stores the struct verbatim,
production auto-fills before encode).
* Bad-field validation rejects wrong-length IH.
* Per-pubkey isolation: PPMI under pub A doesn't shadow pub B.
* Compile-time interface satisfaction check for AnacrolixPutter,
AnacrolixGetter, and MemoryPutterGetter.
Small change to dht.go: MemoryPutterGetter gains a ppmiStore
map; NewMemoryPutterGetter initialises both maps. Existing
Put/Get paths unchanged.
Binaries rebuilt: dist/swartznet, dist/swartznet-gui-dev-linux-amd64.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the existing Lookup to optionally do PPMI resolution
before the legacy per-keyword fan-out. Publishers who have a
PPMI are recorded in the new LookupResponse.PPMIsResolved field
and SKIPPED on the legacy path (we trust the Aggregate pointer
and don't double-query their per-keyword items). Publishers
without a PPMI fall through to the existing path unchanged.
When no PPMIGetter is attached, Query behaves exactly as before.
The migration per PROPOSAL §6 Phase 1 is dual-write on the
publisher side; this commit adds the matching dual-read.
Added:
- LookupResponse:
* PPMIsResolved []ResolvedPPMI — successful publisher pointers.
* PPMIMissing int — indexers whose PPMI fetch failed. Useful
for "most of your indexers haven't migrated" UI.
- ResolvedPPMI struct with PubKey, Label, and the fetched PPMIValue.
- Lookup.SetPPMIGetter / PPMIGetter accessor methods.
- resolvePPMIs helper that fans out GetPPMI in parallel.
- legacyQuery helper factored out of Query so the Aggregate
path and the legacy path can share encoding / merge / score /
source-tracking without duplication.
Refactored:
- Lookup.Query now orchestrates the two paths: fan out PPMIs
first (when getter is set), exclude resolved publishers from
the legacy fallback set, run the legacy path on the
fallback-only subset. IndexersAsked reports the total (PPMI
+ legacy).
- internal/dhtindex/lookup_ppmi_test.go (5 tests):
* Without a PPMI getter, Query is identical to pre-P2.3.
* Mixed migration state: A has PPMI, B is legacy — A
resolves via Aggregate path, B returns a legacy hit, total
IndexersAsked = 2.
* All publishers migrated → PPMIsResolved populated, Hits
empty (no legacy fallback).
* No publishers migrated → PPMIMissing equals total, all
legacy fall-through.
* SetPPMIGetter(nil) cleanly disables the Aggregate path.
All 70+ existing dhtindex tests continue to pass unchanged.
Binaries rebuilt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SPEC.md §2 msg_types 4-8 implementation. All five wire types
decode/encode with size validation; a SyncSession state machine
holds one end of an RIBLT exchange and produces/consumes frames
on demand. Callers wire frames through LTEP themselves — keeping
the session I/O-free makes it trivially unit-testable and
reusable for future transports (if we ever add, say, a localhost
HTTP debug endpoint for it).
Added:
- internal/swarmsearch/sync_wire.go (~220 lines):
* MsgTypeSyncBegin (4), SyncSymbols (5), SyncNeed (6),
SyncRecords (7), SyncEnd (8) constants.
* Status / budget constants: SyncStatusConverged /
LimitExceeded / Aborted; DefaultSyncMaxSymbols (2000),
DefaultSyncMaxBytes (1 MiB), MaxSymbolsPerMessage (100),
MaxRecordsPerMessage (500), MaxNeedIDsPerMessage (1000).
* SyncFilter {Pubkeys, Since, Prefix}.
* Five message structs matching SPEC §2.3-2.8 byte schemas.
* Encode / Decode pair per type with per-field size validation
(element 32B, sig 64B, ih 20B, pubkey 32B, id 32B).
* Decoders reject payloads carrying the wrong msg_type
discriminator, mirroring existing v1 message decoders.
- internal/swarmsearch/sync_session.go (~340 lines):
* SyncRole (initiator / responder), SyncSessionPhase machine
with six states (Idle → Begun → SymbolsFlowing → Needed →
Fulfilled → Ended).
* LocalRecord: wire-friendly view of a signed Aggregate record.
Deliberately duplicated from companion.Record so the session
can stay import-cycle-free (the signature + PoW re-
verification happens at the outer handler layer).
* NewSyncSession indexes records by RIBLT element ID (SHA-256
over pk || kw || ih || t_LE per SPEC §2.4), bootstrapping
both the encoder and decoder's local set.
* Begin / ApplyBegin negotiate the session.
* ProduceSymbols emits up to MaxSymbolsPerMessage-capped
batches, honoring the maxSymbols budget with
ErrSymbolBudgetExceeded.
* ApplySymbols ingests peer batches, running auto-peeling.
* NeedIDs / RemovedIDs expose decoded differences.
* NeedFrame / ApplyNeed handle the pull request.
* BuildRecordsFrame / ApplyRecords handle the bulk delivery.
* Finish / ApplyEnd close the session with observed counts.
* RecordByID is the responder-side lookup hook.
- internal/swarmsearch/sync_wire_test.go (10 tests): round-trip
every message type, reject empty / oversized symbol batches,
reject bad DataXOR / pk / sig / id sizes, reject mismatched
msg_type discriminators.
- internal/swarmsearch/sync_session_test.go (10 tests):
* End-to-end roundtrip: sender {a,b,c,d} + receiver {a,b,e} →
decodes Added=2 (c,d), Removed=1 (e), delivers records.
* One-sided 40-record diff converges in ≤300 symbols.
* Phase guards reject calls from wrong phase.
* TxID mismatch surfaces as protocol error.
* Budget enforcement clamps produced symbols and raises
ErrSymbolBudgetExceeded on subsequent calls.
* Empty NeedFrame ("I'm done") advances to Needed phase.
* Finish / ApplyEnd both land in Ended phase.
* RecordByID round-trips against stored ID.
* localRecordID is deterministic for identical input.
Not in this commit: LTEP dispatch integration in handler.go.
The session is fully driven by test-controlled frame flow today;
plumbing it through the connection-level handler — with rate-
limit enforcement and misbehavior scoring from P5.1 on bad
sigs/PoW — lands in the Final integration phase alongside the
wire-compat matrix regression gates.
Binaries rebuilt: dist/swartznet, dist/swartznet-gui-dev-linux-amd64.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SPEC.md §3 orchestration for a fresh subscriber. Three
independent channels populate the Lookup indexer set:
- Channel A (RunAnchors): fetch each anchor pubkey's PPMI and
admit with AnchorReputation seed on success.
- Channel B (CandidateFromCrawl): admit a pubkey harvested via
BEP-51 + snet.pubkey metainfo inspection, subject to sig
validation and Bloom-policy gate.
- Channel C (IngestEndorsement): accumulate distinct endorsers
for each candidate; admit on EndorsementThreshold reach OR
Bloom-policy gate.
Admission bookkeeping:
- admitted set + MaxTrackedPublishers cap (default 100)
so a spammy gossip flood can't blow out the indexer set.
- AnchorReputation (0.8) and CandidateReputation (0.1)
seed distinct trust levels; EigenTrust-style anchoring.
- tracker/bloom optional; bloomPolicy currently default-denies
when no Bloom filter is attached (conservative — channel C
still admits via pure endorsement threshold).
Added:
- internal/daemon/bootstrap.go (~280 lines):
* DefaultAnchorPubkeys []string (empty pending production
release), BootstrapOptions, Bootstrap struct.
* NewBootstrap constructor with hex-anchor validation and
defaults.
* RunAnchors: parallel PPMI fetch across anchor set, returns
(successCount, perAnchorErrors).
* IngestEndorsement / CandidateFromCrawl admission paths.
* IsAdmitted / IsPending / AdmittedCount / AnchorKeys
inspectors.
* Internal admit() registers to Lookup + seeds tracker.
- internal/daemon/bootstrap_test.go (10 tests):
* Anchor hex validation (non-hex, wrong-length, empty-string
handling).
* RunAnchors: successful anchors admitted to Lookup, failed
anchors surfaced in errs slice, anchor labels populated.
* RunAnchorsNoPPMIGetter fast-errors cleanly.
* Endorsement threshold: 2 distinct endorsers → admit;
duplicate endorser doesn't cross threshold; idempotent on
already-admitted.
* CrawlCandidate: invalid sig rejected; bloom-default-deny
without Bloom filter.
* MaxTrackedPublishersCap refuses admission past the cap.
* AdmitSetsLookupLabel: admitted pubkeys appear in
lookup.Indexers() with non-empty labels.
Not in this commit:
- The actual BEP-51 crawler (triggers CandidateFromCrawl) —
requires a live DHT; the Final phase will wire it through
the engine.
- HTTPS anchor fallback (P5.2) for when channels A/B/C all
return nothing.
Binaries rebuilt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
### P5.2 HTTPS anchor fallback (SPEC.md §3.5)
Last-ditch channel for subscribers whose A/B/C channels all
returned nothing. Fetches a JSON anchor list from a project-
operated endpoint, parses hex pubkeys, and extends the
Bootstrap's anchor set. Serves ONLY pubkeys, never records —
compromising the endpoint can delay but not poison bootstrap.
Added:
- internal/daemon/bootstrap_https.go:
* DefaultBootstrapURL (empty pending production release).
* HTTPSFallbackClient interface for test injection + a
production http.Client adapter with sane timeout.
* FallbackToHTTPS(ctx, url, client): parse JSON array of
64-char hex pubkeys, dedupe against existing anchors,
extend in place. Returns count of newly-added anchors.
* MaxAnchorBootstrapBytes (64 KiB) ceiling blocks hostile
megabytes from reaching the parser.
- internal/daemon/bootstrap_https_test.go (8 tests): new
anchors admitted, duplicates deduped, malformed entries
silently skipped, empty URL refused, transport errors
propagated, bad JSON rejected, oversized responses
rejected, post-fallback anchor list survives RunAnchors.
### Final — end-to-end Aggregate integration
internal/daemon/aggregate_e2e_test.go demonstrates the full
publisher → DHT → subscriber flow:
- TestAggregateEndToEnd:
* Publisher signs 30 records across {linux, ubuntu, debian}.
* companion.BuildBTree produces a piece-aligned B-tree with
trailer signed by the publisher's ed25519 key.
* Publisher's PPMI stored in the in-memory DHT via
MemoryPutterGetter.PutPPMI.
* Bootstrap admits the publisher as an anchor, succeeding
RunAnchors call adds it to Lookup.
* Lookup.Query("ubuntu") resolves via the PPMI path: the
committed fingerprint matches the builder's output.
* companion.BTreeReader opens the tree via BytesPageSource
(stand-in for a seeded torrent), verifies trailer sig,
and Find("ubuntu") returns exactly the ubuntu records.
* Every returned record's ed25519 sig re-verifies.
* VerifyFingerprint re-derives SHA-256 over the canonical
record stream and matches the trailer.
- TestAggregateMixedMigration:
* Publisher A has PPMI; Publisher B is on legacy per-keyword.
* Lookup.Query falls through per-publisher: A resolves via
PPMI, B returns legacy Hits. PPMIMissing = 1 (B).
Small shim added: MemoryPutterGetter.PubKey() accessor so
tests can fetch the public key it signs puts under, without
reaching into unexported fields.
### Status
All 12 roadmap phases complete. ~7500 lines shipped across
companion/, dhtindex/, swarmsearch/, daemon/ with ~120 new
test functions. Cross-package race sweep green. Binaries
rebuilt.
Next steps (post-roadmap, not in this commit):
- LTEP dispatch integration for the sync msg_types 4-8
(deferred from P3.2).
- BEP-51 crawler wiring to CandidateFromCrawl (deferred from
P4.1). Requires a live anacrolix/dht server.
- docs/05-integration-design.md §4.3 amendment to reflect the
PPMI inversion now that it's implemented.
- CHANGELOG.md entry for the v0.5.0 milestone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picks up the LTEP dispatch deferred from P3.2. Now two
sn_search peers can actually exchange sync frames over TCP,
subject to the new capability gate.
### New services bit
- BitSetReconciliation = 1 << 9 in internal/swarmsearch/services.go.
Peers that don't advertise this bit in their peer_announce
MUST NOT receive sync frames. Reserved-bit allocation follows
the existing rule: never reuse, always next-available.
### Handler dispatch
- HandleMessage gains five new cases (msg_types 4-8) routing to
handleSyncFrame, which:
* Checks peer's Services mask for BitSetReconciliation.
* If not set: reply reject code 2, charge ScoreUnexpectedMessage,
return without state change.
* If set: decode the specific frame, dispatch to onSync* handler.
- Per-peer session map added to Protocol: map[addr]map[txid]*SyncSession.
- onSyncBegin: creates a responder session and immediately
Finish()-es with SyncStatusConverged (no records source yet;
engine integration will plumb a LocalRecord provider).
Wire behavior is correct — SPEC §2 permits zero-symbol sessions
to complete immediately.
- onSyncNeed: builds a records frame with every requested ID
reported as missing (same rationale: no records source yet).
- onSyncSymbols / onSyncRecords / onSyncEnd: route to the
matching session; ignore if unknown txid.
- registerSyncSession / lookupSyncSession / releaseSyncSession
helpers manage the per-peer state with fine-grained locking.
### Tests (internal/swarmsearch/sync_handler_test.go — 6 tests)
- TestSyncCapabilityGateRejects: peer without bit 9 → reject
code 2, no session registered.
- TestSyncCapableBeginReplysWithEnd: peer with bit 9 → sync_end
reply converged, session released afterward.
- TestSyncSymbolsUnknownSession: unknown txid → silent drop,
no reply, no panic.
- TestSyncNeedAllMissing: known session + missing IDs → sync_records
with empty Records and full Missing list.
- TestSyncBadPayloadCharges: malformed sync frame charges
ScoreBadBencode.
- TestBitSetReconciliationBitOps: Has/With/Without round-trip
for the new bit.
### Docs
- CHANGELOG.md: new Unreleased "Aggregate redesign" section
summarising the 10-commit series + remaining engine plumbing.
- docs/06-bep-sn_search-draft.md: capability table gets bit 9
entry; message-types table gains rows for msg_types 4-8 with
a pointer to SPEC §2.
Binaries rebuilt. Full swarmsearch race sweep green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously onSyncBegin always shipped a zero-record converged
reply because no hook existed to load matching local records.
This commit introduces a RecordSource interface, makes the
handler query it on sync_begin, and produces a real first batch
of RIBLT symbols when records exist.
### New surface
- RecordSource interface:
LocalRecords(filter SyncFilter) ([]LocalRecord, error)
Any provider that can enumerate the node's local signed-record
set satisfies it.
- Protocol.SetRecordSource(src) / RecordSource() accessors.
- recordSource field added to Protocol under the existing mu.
### Handler behavior change
- onSyncBegin:
* Queries RecordSource.LocalRecords(m.Filter); a nil source or
LocalRecords error is treated as "no records" (still correct
wire-level behavior; quiet telemetry).
* Constructs the SyncSession with the loaded records so
subsequent ApplyNeed lookups resolve locally.
* Zero-record path still emits sync_end converged and releases
the session.
* Non-zero path calls ProduceSymbols(MaxSymbolsPerMessage) and
replies with a SyncSymbols frame. The session stays registered
for subsequent sync_need / sync_symbols traffic.
- onSyncNeed:
* Calls sess.ApplyNeed(m) which looks up each requested ID
against the session's pre-indexed records; known IDs come
back as LocalRecords, unknown land in missing.
* Ships BuildRecordsFrame(records, missing) via
sendSyncRecords.
- sendSyncSymbols helper added (mirror of sendSyncRecords /
sendSyncEnd).
### Tests (internal/swarmsearch/record_source_test.go — 6 tests)
- TestRecordSourceAccessors: Get/Set round-trip, detach
with nil.
- TestSyncBeginStreamSymbolsWhenRecordSourcePresent: sync_begin
with a 5-record source now replies SyncSymbols (not SyncEnd)
and the session remains registered.
- TestSyncBeginZeroRecordsFromSource: empty source falls through
to sync_end converged.
- TestSyncBeginRecordSourceError: source.LocalRecords error
treated as "no records"; handler ships sync_end converged
without crashing.
- TestSyncNeedReturnsRecordsFromSource: sync_need with known IDs
receives a SyncRecords reply carrying those records; Missing
is empty.
- TestSyncBeginAppliesFilter: filter.Prefix on SyncBegin is
honored — ubuntu records filtered out when prefix="lin";
subsequent sync_need for an ubuntu ID returns it as Missing.
Binaries rebuilt. Full swarmsearch race sweep green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Keeps docs/ in sync with the v0.5.0 Aggregate redesign per
CLAUDE.md's "docs and CHANGELOG track the code" rule.
- docs/05-integration-design.md §4.3 gains:
* A supersession callout at the top pointing readers at
PROPOSAL.md and SPEC.md for the new PPMI layout.
* A new §4.3.1 "Aggregate (v0.5.0) — per-publisher PPMI
layout" describing the target/salt/v schema, the
O(publishers) cost collapse, the double-hashed-salt
privacy property, commit-bound trailer verification, the
v0.5/v0.6/v0.7 dual-write/dual-read migration staging, the
three-channel bootstrap, and the BitSetReconciliation
peer-wire complement.
- docs/07-bep-dht-keyword-index-draft.md:
* Same supersession notice at the top of the Abstract.
* Draft text kept verbatim for the dual-read migration window;
per-keyword items remain a valid legacy format any BEP-44
DHT node will keep serving.
No code changes. Build still green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In-process Go benchmarks for the two SPEC-named regression gates.
The testbed/Docker bootstrap scenario is its own integration
track; this commit handles the unit-level ones.
### internal/companion/btree_bench_test.go
- BenchmarkPrefixQuery: narrow prefix query over a 50k-record
3-level B-tree. Measures the pruned-walk hot path.
Measured: ~16.8 ms/op on an i9-14900K — well under SPEC's
<50 ms target. Scaling to 5M records adds at most one more
interior level, so the O(log n) walk stays under 50 ms.
- BenchmarkPrefixQueryWide: prefix that matches every record.
Measures the worst-case where every leaf overlaps; dominated
by per-record ed25519 signature verification. Useful lower
bound for "broad keyword" workloads.
- BenchmarkBuildBTree: construction cost over 10k records. Catches
regressions in the sort/pack/sign pipeline.
### internal/swarmsearch/riblt_bench_test.go
- BenchmarkRIBLTConverge_Diff{0,10,100,500}: parameterised
convergence measurement. Records both symbols/op (the
algorithmic signal) and bytes/op (48-byte symbol assumption)
so CI can gate on bandwidth directly.
Measured on an i9-14900K:
Diff=0 → 20 symbols / 960 bytes
Diff=10 → 118 symbols / 5.7 kB
Diff=100 → 732 symbols / 35 kB
Diff=500 → 3244 symbols / 156 kB
SPEC §7 targets "<5 kB/peer/hr steady state" for a 10k-peer
network producing ~1000 new records/hr. That translates to
per-meeting diffs around 50 — sitting between our Diff=10 and
Diff=100 benchmarks, implying ~10-20 kB actual per-meeting
steady state. Within the same order of magnitude as the SPEC
aspiration; the <5 kB target was optimistic for the chosen
rate distribution. Future tuning (denser peer meshes, different
cycle lengths in contributes()) may close the gap.
- BenchmarkRIBLTEncoderNextSymbol: isolates raw symbol-production
cost — independent of decode — for catching regressions in
contributes() or the XOR accumulator.
Benchmarks run only with -bench flag; regular `go test ./...`
is unchanged. No production code touched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User-facing CLI surface for the v0.5.0 Aggregate redesign. Both
subcommands operate on the single-file payload inside an
Aggregate companion torrent (the bytes companion.BuildBTree
produces) — no daemon required, no HTTP API dependency, no
live DHT. Useful for operators validating their own index build
before publishing and for subscribers inspecting companion
torrents they've downloaded.
### cmd_aggregate.go
cmdAggregate dispatches to one of:
- inspect <index-file>
Opens the B-tree via companion.OpenBTree (which verifies
the trailer signature), prints file size, page count,
record count, publisher pubkey, sequence, created ts,
min PoW bits, and tree fingerprint.
- find [--piece-size=N] [--verify] <index-file> <prefix>
Runs a prefix-query via BTreeReader.Find, prints each
matching record as "<infohash-hex> <keyword> t=<ts>".
--verify also runs VerifyFingerprint (scans every leaf —
slower, catches any post-publish tampering that escaped
the trailer-sig check).
- help
Prints usage.
Both subcommands take --piece-size so users who published at a
non-default piece size (256 KiB) can still inspect their files.
### cmd_aggregate_test.go (9 tests)
- inspect prints records, fingerprint, and core fields.
- inspect rejects missing files with an exit code and a
read-error message on stderr.
- inspect rejects malformed (zero-filled) files — the bad-magic
path in OpenBTree fires correctly.
- find returns prefix matches, filters out non-matching
keywords.
- find returns "0 records" for an empty result, still exit OK.
- --verify flag drives VerifyFingerprint over a known-good tree.
- Unknown subcommand returns exitUsage.
- help returns exitOK and mentions 'aggregate' in output.
- No-subcommand case prints usage + exits with exitUsage.
Top-level main.go routes "aggregate" to cmdAggregate and lists
it in `swartznet help`.
Full repo race sweep clean (cmd, companion, dhtindex, swarmsearch,
engine, daemon, indexer, gui, httpapi, etc.). Binaries rebuilt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the v0.5.0 CLI loop: operators can now build, inspect, and query Aggregate companion-index files end to end without a running daemon. Pairs with the inspect/find subcommands landed in 3416638. ### Input schema (JSONL, one record per line) {"kw": "ubuntu", "ih": "<40-char hex sha-1>", "t": <unix ts>} The `t` field is optional (defaults to 0). Empty kw or wrong-length ih are rejected early with a line number so users can debug their own extraction pipelines without guessing. ### cmd_aggregate_build.go Flags: --in FILE JSONL input ("-" for stdin), default "-". --out FILE Output path (required). --key FILE Identity file to sign with; defaults to the node's ~/.local/share/swartznet/identity.key. --seq N Trailer sequence number (monotonic per publisher). --piece-size N Piece size in bytes (must match .torrent metainfo). --pow-bits N Hashcash difficulty (0=off, 20=production, >40 refused). Flow: load key → parse JSONL → sign (+ optional mine) each record → companion.BuildBTree → write output. Pure offline — no DHT, no daemon, no network. Print summary: records, pages, bytes, fingerprint, output path. ### cmd_aggregate_build_test.go (7 tests) - TestCmdAggregateBuildInspectFindChain: end-to-end — build 3 records, inspect reports 3, find "ubu" returns both ubuntu entries. Validates the full CLI loop in one pass. - TestCmdAggregateBuildRequiresOut: missing --out is exitUsage. - TestCmdAggregateBuildRefusesHighPoW: --pow-bits=50 refused (cost-prohibitive guard from companion.MineRecordPoW). - TestCmdAggregateBuildRejectsBadJSONL: "not json at all" surfaces with line 1 in stderr. - TestCmdAggregateBuildRejectsBadInfohash: short ih caught at record-parse, before any signing. - TestCmdAggregateBuildRejectsEmptyInput: empty JSONL → exitUsage. - TestCmdAggregateBuildWithSmallPoW: --pow-bits=4 completes quickly and min_pow_bits=4 is readable in the trailer via `aggregate inspect`. Live smoke tested: build 3 records, inspect reports the trailer correctly, find "ubu" returns 2 matches with the expected infohashes and timestamps. Binaries rebuilt. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In-memory cache keyed on RIBLT element ID, implements the RecordSource interface introduced in 9e6f613. The engine can now populate it as the local publisher signs records; sync responders see fresh records immediately on the next sync_begin without any poll cadence. ### internal/swarmsearch/record_cache.go RecordCache: thread-safe map[ID]LocalRecord with: - Add(r): insert (idempotent — same pk/kw/ih/t → same ID). - Remove(id): delete by RIBLT element ID. - RemoveByRecord(r): compute ID then delete. - Get(id): point lookup, returns (record, ok). - Len(): size of the cache. - Snapshot(): independent copy of every record (mutation-safe). LocalRecords(filter SyncFilter) ([]LocalRecord, error): implements RecordSource. Filters by: * Pubkeys — record.Pk must be in the supplied set; invalid- length entries silently ignored. * Since — record.T >= filter.Since. * Prefix — record.Kw starts with filter.Prefix. All conditions conjunct; zero-value filter returns every record. cacheRecordID is the deterministic 32-byte key derivation — matches localRecordID (sync_session.go) bit-for-bit so a record added to the cache is findable via the session's ApplyNeed without an ID-translation layer. ### internal/swarmsearch/record_cache_test.go (15 tests) - TestCacheIDMatchesSessionID: explicit invariant that cacheRecordID == localRecordID for the same input. Guards against future drift where a refactor might touch one but not the other. - Basic Add/Get/Remove/Len/RemoveByRecord round-trips. - Idempotent Add: re-adding same record leaves Len unchanged. - LocalRecords: * Empty filter returns every record. * Pubkeys filter narrows to listed publishers. * Since filter drops records below threshold. * Prefix filter drops records whose kw doesn't start with it. * Combined filters conjunct correctly — the intersection test covers 4 records, 3 conditions, 1 expected hit. * Invalid-length pubkeys silently ignored. - Compile-time interface satisfaction check (RecordSource). - Concurrency: parallel writer + reader goroutines, -race clean, final Len == N. - Snapshot returns independent slice (mutation doesn't leak). - TestRecordCacheDrivesSyncBegin: end-to-end — wire a RecordCache as RecordSource on a real Protocol, send sync_begin, assert the reply is SyncSymbols (not the zero- record SyncEnd path). Proves cache.Add → sync responder. Scope note: engine wiring (calling cache.Add when the local publisher mints a record) is the next step and lives in internal/engine. This commit ships the standalone component so it can be unit-tested in isolation; the engine-side call sites land separately. Binaries rebuilt. Full swarmsearch race sweep green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lightweight JSON endpoint exposing v0.5 Aggregate-track state.
Lets CLI/UI consumers see what's going on in the new layer
without cross-cutting into swarmsearch internals.
### internal/httpapi/aggregate.go
GET /aggregate response shape:
{
"ppmi_enabled": true|false,
"known_indexers": int,
"indexers": [{"pk": hex64, "label": "..."}],
"record_source_kind": "cache" | "custom" | "",
"record_cache_size": int,
"services": hex16 (uint64 ServiceBits)
}
- PPMIEnabled reflects lookup.PPMIGetter() != nil.
- Indexers snapshot comes straight from Lookup.Indexers();
order is not sorted — UI can sort if display matters.
- RecordSourceKind identifies the source type:
* "cache" → *swarmsearch.RecordCache (size reported in
RecordCacheSize).
* "custom" → any other impl (size not reported; we don't
leak the underlying type name).
* "" → no source attached.
- ServicesAdvertised is the uint64 bitmask big-endian
hex-encoded (16 chars). Bit 9 = 0x200 = BitSetReconciliation.
### Server wiring
server.go registers `GET /aggregate → handleAggregateStatus`
next to the other read-side endpoints. Nil swarm/lookup collabs
surface as zero-valued fields (matches the
"all Options fields optional" pattern).
### internal/httpapi/aggregate_test.go (6 tests)
- Empty daemon (no swarm, no lookup) → zero-valued response,
status 200.
- Lookup populated with one indexer but no PPMI getter:
Indexers[0] carries the hex pubkey + label;
PPMIEnabled=false.
- PPMI getter attached: PPMIEnabled=true.
- Swarm with a RecordCache source: kind="cache", size matches,
ServicesAdvertised is 16 hex chars.
- Swarm with a non-cache RecordSource (anonymous struct):
kind="custom", size=0.
- Response Content-Type: application/json.
Binaries rebuilt. Full httpapi race sweep green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hits GET /aggregate alongside /status and appends a block for the
v0.5 Aggregate track. Keeps the command usefully informative in
the dual-read migration window when both legacy and Aggregate
paths may be active.
### cmd_status.go
- fetchAggregateBlock: best-effort GET /aggregate. A 404 (older
daemon), network error, or decode failure returns nil; the
renderer quietly omits the block. This keeps the command
backward-compatible — users running a daemon from before
fac200d still see the normal output.
- emitStatusText gains an `agg *AggregateStatusResponse` arg.
After the Publisher block, when agg != nil, calls
emitAggregateBlock which prints:
PPMI enabled, known indexers, record source kind,
record cache size, services advertised (hex).
Lines are conditionally omitted when their values are
zero/empty so the block stays tight.
- The "(no keywords published yet)" branch no longer
return-early-short-circuits the function; both paths fall
through to the optional Aggregate block.
- --json output now wraps both payloads in a single JSON
object: {"status": ..., "aggregate": ...}.
- Existing cmd_status_test.go call sites updated to pass nil
for the new agg argument.
### cmd_status_aggregate_test.go (4 tests)
- TestStatusTextOmitsAggregateWhenNil: nil agg → no Aggregate
block rendered.
- TestStatusTextRendersAggregateBlock: populated response
produces every expected line (PPMI, indexers, record source,
cache size, services hex).
- TestStatusTextAggregateBlockWithoutPPMI: PPMIEnabled=false
still renders the block, just with the flag off; record-source
line is omitted when kind is empty.
- TestStatusTextAggregateBlockEmptyCache: custom kind + 0 size
→ kind line renders, size line omitted.
Binaries rebuilt. Full cmd/swartznet race sweep green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the CLI status-tab addition from ec7a791. The embedded web UI's Status panel now carries an "Aggregate (v0.5)" card next to the existing Local/Swarm/DHT/Publisher cards, populated from GET /aggregate. Older daemons that 404 on /aggregate silently skip the card — pattern copied from the existing Bloom/Reputation conditional cards. ### Changes - app.js refreshStatus: adds /aggregate to the Promise.all fetch with .catch(() => null) so older daemons don't break the whole status refresh. - renderStatus gains an `agg` parameter; when non-null, appends a new card with PPMI enabled (yes/no), known indexers count, optional record source kind, optional cache size. - Conditional row rendering: record-source line omitted when kind is empty; cache-size line omitted when size is 0 — matches the CLI renderer's conventions from ec7a791. ### internal/httpapi/web/aggregate_card_test.go (3 tests) Since the embedded JS has no browser-level test harness, tests assert on the bundle content directly — cheap, deterministic, catches regressions where a refactor drops the card or its fetch. - TestAggregateCardPresentInBundle: the 9 critical substrings (/aggregate, card title, field reads) all present. - TestStatusCardsPresentInBundle: the older Local/Swarm/Publisher cards still present — guards against accidental removal during future refactors. - TestAggregateFetchTolerantOfMissingEndpoint: the fetch's .catch(...) idiom is preserved — otherwise older daemons would break the whole status panel. Binaries rebuilt. httpapi/web race sweep green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the web-UI Aggregate card from 160b5e6 and the CLI status block from ec7a791 into the Fyne-based native GUI. Shows the four v0.5 Aggregate-track signals an operator wants to see on-screen: PPMI enabled, known indexers, record source kind, cache size. ### internal/gui/status.go - New card widget "Aggregate (v0.5)" with 4 label rows. Slots into the adaptive grid next to DHT Routing and DHT Publisher. - refresh() introspects the live subsystems: * Lookup.PPMIGetter() != nil → "yes"/"no". * Lookup.Indexers() → count. * SwarmSearch().RecordSource() → "cache" (with Len()) / "custom" / "-". - Nil-safe throughout: a daemon without Aggregate wiring (no Lookup, no SwarmSearch, no RecordSource) renders all zeros and "no" rather than crashing. Follows the same pattern as the DHT routing-table card landed in 5ab4d71: plain widget.Card, labelRow helper, fyne.Do for the UI-thread update from the poll goroutine. Binaries rebuilt (CLI + GUI). internal/gui race sweep green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the last wiring gap in the Aggregate track. Every engine constructor path now creates an empty *swarmsearch.RecordCache and attaches it to the swarm protocol via SetRecordSource. Publishers feed it via Engine.RecordCache().Add(...); sync responders serve from it automatically; the /aggregate endpoint, CLI status block, web card, and GUI card all flip from showing "no record source" to reporting "cache" with a real Len(). ### Changes in internal/engine/engine.go - Engine gains a `recCache *swarmsearch.RecordCache` field, always non-nil after engine.New. - engine.New constructs the cache immediately after the swarm protocol and calls swarm.SetRecordSource(recCache). This is unconditional — cheap empty map, consistent observable state across headless-test and production constructor paths. - New Engine.RecordCache() accessor next to the other getters. Documented as the entry point for publisher hot-paths that mint signed records. ### Tests (internal/engine/record_cache_wiring_test.go) - TestEngineNewAttachesRecordCache: engine.New produces a non- nil cache, swarm.RecordSource() returns it, and it is the same *RecordCache instance the accessor exposes. Catches the class of bug where an engine constructor silently replaces the source with a different impl. - TestEngineRecordCacheAddVisibleToSource: adding a record via Engine.RecordCache().Add(r) makes it visible to the swarm's attached RecordSource via LocalRecords. Confirms the wiring is live, not just wired-at-construction-then-disconnected. Full repo race sweep clean (15 packages). Binaries rebuilt. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Running daemons with an identity now push one LocalRecord per tokenised keyword into the RecordCache whenever a torrent's metadata arrives — closes the loop from PROPOSAL.md: metadata → mint → sync responder serves records over sn_search. ### Changes in internal/engine/engine.go - autoIndex fires a new Engine.MintAggregateRecords(ih, name) call alongside the existing Layer-D publisher submit. The minting logic is extracted into an exported method so tests can exercise it without spinning up a torrent-add fixture. - MintAggregateRecords tokenises `name` via dhtindex.TokenizeAll (same analyser the legacy publisher uses, so both paths agree on keyword shape). For each token: SignAndMineRecord under the engine's identity, convert to swarmsearch.LocalRecord, Add to the cache. - PoW is 0 (disabled) in v0.5 dual-read window — no reader below a threshold today; bumping to MinPoWBitsDefault is a follow-on schema bump. - Silent-skip when identity or cache is nil — headless test setups still exercise every other autoIndex code path without crashing. - Debug log at end reports token count and resulting cache size for regression observability. ### Tests (internal/engine/mint_records_test.go — 4 tests) - TestMintAggregateRecordsPopulatesCache: real engine-with- identity, Mint a torrent named "Ubuntu Linux 24.04 LTS" → cache contains records for "ubuntu" AND "linux"; every stored record's ed25519 sig re-verifies under the publisher's pubkey. - TestMintAggregateRecordsSilentWithoutIdentity: engine built with IdentityPath="" (explicit opt-out — config.Default() would otherwise point at the user's real identity file) does NOT populate the cache. Documents the no-identity guard. - TestMintAggregateRecordsIdempotent: successive mints at different timestamps grow the cache rather than shrink; documents that "idempotent" here means safe-to-call-repeatedly, not Len-stays-put. - TestMintAggregateRecordsEmptyName: empty name → zero tokens → zero new records. Binaries rebuilt. Full repo race sweep clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the last deferred item from the original roadmap. A
daemon started with DHT on now:
- constructs a *daemon.Bootstrap with the engine's Lookup,
PointerGetter (which satisfies PPMIGetter), known-good
Bloom filter, and reputation tracker;
- kicks off channel A (anchor PPMI fetch) in a background
goroutine so daemon.New doesn't block on the parallel
anchor fetch;
- exposes the bootstrap via d.Bootstrap so HTTP/CLI/GUI
surfaces can introspect admission state.
Channels B (BEP-51 crawl) and C (peer_announce endorsement
gossip) stay pluggable hooks — the Bootstrap supports them
today; the engine still needs to deliver the respective signals.
### Changes in internal/daemon/daemon.go
- Daemon struct gains `Bootstrap *Bootstrap` field alongside
the existing subsystems. Nil when Lookup unavailable.
- New branch between companion-subscriber setup and session
restore constructs Bootstrap when eng.Lookup() != nil.
Errors are warn-logged but never fatal — a daemon can still
start without the bootstrap, it just runs without an anchor
fetch.
- RunAnchors fires in a goroutine so daemon.New stays fast; the
log line reports (succeeded, errors) once it completes.
### Tests (internal/daemon/bootstrap_wiring_test.go)
- TestDaemonBootstrapNilWithoutDHT: DisableDHT=true → engine
has no Lookup → d.Bootstrap stays nil. Covers the "DHT-off"
daemon path.
- TestDaemonBootstrapAttachesWithDHT: Regtest=true + DHT on →
Bootstrap attaches; fresh AdmittedCount is 0. Confirms the
wiring is live without depending on hardcoded anchors.
Binaries rebuilt. Full 15-package race sweep clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plumbs the Bootstrap orchestrator's counters (anchor pubkeys,
admitted publishers) through the /aggregate endpoint and the CLI
status block so operators can observe channel-A fetch progress
live without log tailing.
### httpapi
- New BootstrapProbe interface in aggregate.go — AnchorCount +
AdmittedCount. Kept deliberately minimal so the httpapi layer
doesn't need to import internal/daemon (would create an
import cycle).
- AggregateStatusResponse gains a nullable `Bootstrap` field of
type *AggregateBootstrap{Anchors, Admitted int}. Omitted when
the server has no probe attached.
- Server struct + Options gain a `Bootstrap BootstrapProbe`
slot; wired into NewWithOptions.
### daemon
- daemon.Bootstrap gains an AnchorCount() method returning
len(b.anchorKeys) so it natively satisfies BootstrapProbe
alongside the already-existing AdmittedCount().
- daemon.New passes d.Bootstrap into httpapi.Options.Bootstrap
when non-nil. Order matters: the Aggregate-bootstrap block
runs before the HTTP-API block in daemon.New, so the probe
is set by the time the Server sees it.
### CLI status block
- emitAggregateBlock renders two new rows when
agg.Bootstrap != nil:
"bootstrap anchors: N"
"bootstrap admitted: M"
- Rows omitted when the probe is nil — matches the pattern for
the other optional rows (record source, cache size).
### Tests (6 new across two files)
- httpapi/aggregate_test.go (+2):
* TestAggregateEndpointReportsBootstrap: fake probe with
anchors=5, admitted=12 surfaces both values in the JSON.
* TestAggregateEndpointOmitsBootstrapWhenNil: no probe →
Bootstrap field omitted from response.
- cmd_status_bootstrap_test.go (+2):
* TestStatusTextRendersBootstrapBlock: populated probe →
both lines render.
* TestStatusTextOmitsBootstrapWhenNil: nil Bootstrap →
no "bootstrap" lines in output.
Binaries rebuilt. Full repo race sweep clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comprehensive summary of the 26 commits on this branch. Intended as source material for the PR description, release notes, and an operator-facing overview of what shipped. Covers: - New byte formats (PPMI, B-tree index torrent, hashcash). - New wire protocol (sn_search msg_types 4-8, BitSetReconciliation capability bit, RIBLT). - New runtime components (BTreeReader/BuildBTree, RecordCache, SyncSession, PPMIPutter/Getter, Bootstrap). - Integrations (Engine attaches cache, mints records on torrent-add; daemon wires Bootstrap; sync handler queries RecordSource). - Observability (/aggregate endpoint, CLI/web/GUI status cards). - Ops tooling (swartznet aggregate build/inspect/find). - Regression gates (benchmarks, end-to-end integration test). - Docs updates (05/06/07 + CHANGELOG). Also enumerates: - A worked-example CLI exercise showing the offline build/find flow and the running-daemon introspection flow. - Seven explicitly deferred items (empty anchor list, BEP-51 wiring, endorsement gossip extraction, production PoW difficulty, per-record reputation, OHTTP/PIR, Dandelion++) with pointers to PROPOSAL sections that still cover them. - Migration notes for operators — dual-write/dual-read, no action required on upgrade. - The two rough-edge bugs caught in-development (RIBLT graduated rate, FNV-1a key hash). - Chronological commit table with every hash + subject. No code changes. Docs only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SPEC §3.3 channel-C wiring: peer_announce frames now carry an
optional `endorsed` list of 32-byte publisher pubkeys the sender
vouches for. The handler routes each entry through a new
EndorsementSink interface which the daemon implements via an
adapter over Bootstrap.IngestEndorsement.
### swarmsearch/wire.go
- PeerAnnounce gains `Endorsed [][]byte` (bencode key
"endorsed", omitempty). Capped at MaxEndorsedPerAnnounce
(= 10, matching SPEC §3.3) on both encode and decode.
- EncodePeerAnnounce refuses oversized lists and malformed
(non-32-byte) entries up front.
- DecodePeerAnnounce silently truncates over-cap lists and
filters wrong-length entries — malformed entries are local
validation faults, not protocol violations, so the rest of
the frame still decodes.
### swarmsearch/protocol.go
- New EndorsementSink interface with a single
NoteEndorsement(endorser, candidate) method.
- Protocol struct gains an endorsementSink field + fine-
grained locking alongside the existing indexerSink.
- SetEndorsementSink / EndorsementSink getter/setter.
### swarmsearch/handler.go
- PeerAnnounce dispatch extended: after the existing indexer-
sink routing, iterate pa.Endorsed and call
sink.NoteEndorsement for each valid entry. Drops:
* Entries with the wrong length (already filtered by
Decode, defensive here).
* Self-endorsement (candidate == sender pk).
* All-zero pubkeys.
- Endorsements from a peer that did NOT announce its own
publisher pk are DROPPED — protects the admission policy
from pubkey-less peers influencing threshold counts.
### daemon/daemon.go
- New bootstrapEndorsementSink{boot *Bootstrap} adapter
implements swarmsearch.EndorsementSink.
- After Bootstrap construction, daemon.New installs the
adapter via eng.SwarmSearch().SetEndorsementSink. Keeps
the swarmsearch package free of any internal/daemon
import (would have been an import cycle).
### Tests (9 new in swarmsearch/endorsement_test.go)
- Wire: round-trip preserves 32-byte entries; encode refuses
oversize lists and wrong-length entries; decode filters
malformed entries and truncates overruns.
- Handler: routes valid endorsements when sender has pk;
drops everything when sender has no pk; filters self + zero
candidates.
- Protocol: SetEndorsementSink / EndorsementSink round-trip.
Binaries rebuilt. Race sweep clean across the 8 packages
touched; the pre-existing GUI -race flakes observed on CI are
unrelated to this commit (none of internal/gui touched).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fyne v2.7.3's theme.DefaultTheme() uses lazy-init; the first two
concurrent callers race on the internal theme cache. Our helper
tests (TestSwartzThemeColours, TestCopyableValue, etc.) that call
DefaultTheme indirectly were marked t.Parallel() and regularly
tripped the -race detector. Observed failure rate ~20-30% across
tail runs; non-deterministic which tests trip.
### Diagnosis
Race captured from the detector:
Read at 0x000003bccf60 by goroutine 58:
fyne.io/fyne/v2/theme.DefaultTheme() theme.go:44 +0x675
gui.(*swartzTheme).Color() theme.go:54 +0x6ef
gui.TestSwartzThemeColours() helpers_test.go:306
Previous write at 0x000003bccf60 by goroutine 61:
fyne.io/fyne/v2/theme.DefaultTheme() theme.go:45 +0x5a
fyne/test.(*configurableTheme).Icon() theme.go:224 +0xaf
theme.safeIconLookup() icons.go:1373 +0x41
theme.ContentCopyIcon() icons.go:958 +0x224
gui.copyableValue() app.go:316 +0x22f
gui.TestCopyableValue() helpers_test.go:401
The race window is strictly the first-call init — once the
theme is materialised, subsequent reads are safe.
### Fix
internal/gui/testmain_test.go adds a TestMain that calls
theme.DefaultTheme() and theme.BackgroundColor() serially
before os.Exit(m.Run()). Forces the lazy init chain (theme →
icons → colour palette) to complete in a single goroutine,
eliminating the race window. Parallel tests afterward see a
stable pointer.
Cost: one theme instantiation, ~microseconds.
### Verification
Before: 2-3 fails per 10-iteration batch under -race.
After: 30/30 clean, 10/10 clean (two separate batches).
Also confirmed the full repo race sweep across all 15 packages
runs clean.
No production code touched — fix is test-harness-only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Every v0.5+ node now automatically advertises the Aggregate sync
capability in its peer_announce. Two running engines see each
other's bit across the LTEP wire — the capability gate in the
sync handler will pass on inbound msg_types 4-8 in real
peer-to-peer traffic, not just unit tests.
### swarmsearch/services.go
DefaultServices() bitmask now includes BitSetReconciliation
(bit 9, 0x200) alongside the existing:
BitShareLocal | BitFileHits | BitContentHits |
BitCompanionPublisher | BitCompanionSubscriber |
BitSnippetHighlight
Resulting bitmask: 0x000000000000002ed.
Before this change, every v0.5 daemon shipped BUT refused to
participate in sync sessions — each sync_begin frame was
gated on Services.Has(BitSetReconciliation), which was only
set when tests manually registered the bit. This commit
flips the default so production traffic works too.
### testlab/aggregate_services_scenario_test.go (2 tests)
- TestScenarioBitSetReconciliationAdvertised: spins up a
2-node cluster, wires the mesh, waits for handshakes, and
asserts every node sees BitSetReconciliation on at least one
known peer. Proves the bit actually travels from one
peer_announce emission to the other side's PeerState.
- TestDefaultServicesIncludesBitSetReconciliation: belt-and-
braces regression guard — a future refactor that accidentally
drops the bit from DefaultServices fails this test cheaply.
Observed on run:
node 0: peer 127.0.0.1:35830 advertises BitSetReconciliation
(services=00000000000002ed)
node 1: peer 127.0.0.1:34316 advertises BitSetReconciliation
(services=00000000000002ed)
Binaries rebuilt. Full 15-package repo race sweep clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the initiator-side half of the Aggregate sync protocol. Before this commit, every node could RESPOND to a sync_begin but had no way to ORIGINATE one — the capability was responder-only, making peer-to-peer sync strictly asymmetric and impossible to use in production. ### New exports - Protocol.StartSync(peerAddr, filter, localRecords) (*SyncSession, error) Creates an initiator-side session, calls Begin(filter) to produce the sync_begin frame, registers the session (so inbound sync_symbols/records/end route back via the existing lookupSyncSession path), encodes + sends the frame, returns the session handle. Rolls back session registration on send error so the txid slot is free for retry. - Protocol.SendSyncNeed(peerAddr, sess, ids) Encodes + ships a sync_need frame. Caller-convenience wrapper so users don't have to reach for the Sender themselves. - Protocol.CloseSync(peerAddr, sess, status) Emits sync_end, marks session Ended, releases from the per-peer map. Idempotent. - Protocol.WaitSyncConverged(sess, timeout) bool Poll-convenience for tests and ops tools that don't want to roll their own convergence loop. ### Guards - ErrSyncPeerUnknown / ErrSyncCapabilityMissing / ErrNoSender — StartSync refuses early when: * The peer is not known to the swarm (no PeerState). * The peer's advertised services lack BitSetReconciliation. * No Sender is attached (no way to transmit anyway). ### Tests (7 new in sync_start_test.go) - TestStartSyncUnknownPeer, TestStartSyncPeerMissingCapability, TestStartSyncNoSender — each guard fires on the right condition. - TestStartSyncHappyPath: session created with role=initiator phase=Begun, sync_begin frame sent with the session's txid, lookupSyncSession finds the same session instance afterward. - TestStartSyncSenderFailRollsBack: Send error propagates AND releases the session so the txid slot is free. - TestCloseSyncReleasesSession: sync_end frame matches session txid, lookup returns nil post-close. - TestSendSyncNeed: sync_need frame carries the supplied IDs. recordingSender helper shared across tests captures every outbound Send for assertion. Keeps the tests free of live transport concerns — same pattern as the existing Query tests. Full repo race sweep clean on second run. The first-pass flake on engine.TestSessionRoundTrip (10 s timeout under concurrent package load) reproduces neither in isolation nor on retry; unrelated to this commit. Binaries rebuilt. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Locks the default-Status branch in EncodeSyncEnd: a SyncEnd constructed without a Status field gets SyncStatusConverged promoted in. This is the defence against an operator forgetting to set Status — a frame on the wire always carries a non-empty status string. swarmsearch package coverage 95.7% → 95.8%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two tests for the dark-theme implementation:
- Color: walks every named color the switch handles
explicitly + an unknown name to hit the default-fallback
arm. Color was at 25% because only a handful of cases
fire when a real Fyne app renders.
- Font / Icon / Size: pass-through helpers that delegate
to theme.DefaultTheme(); just verify they return non-nil
/ non-zero values.
gui package coverage 8.3% → 9.4%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two small assertions on the windowForObject helper:
- Runtime test: windowForObject(nil) returns nil. The
function takes a fyne.CanvasObject and is called by
every tab to find its host window; the nil-input guard
prevents a downstream panic in app.Driver().CanvasForObject.
- Compile-time: var _ func(fyne.CanvasObject) fyne.Window
= windowForObject pins the signature so future Fyne API
drift causes a build failure instead of a silent shift.
Function coverage stays at 25% (full traversal needs a real
Fyne canvas with active app + driver, which costs more setup
than the contract value here justifies). The test exists for
the contract lock, not the coverage number.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four tests for the wire-level guards in DecodeInterior:
- bad kind (leaf page passed in): only PageKindInterior /
PageKindRoot are accepted.
- payload-length-exceeds-page: synthetic page claims
PayloadLength=60000 in 24 bytes — defends against
truncated wire frames.
- payload-too-short: 1-byte payload (less than the 2-byte
child-count header) is rejected.
- short separator: 1 child with a 1000-byte varint
separator length but no separator bytes — "short
separator or child index" guard fires.
Helper makeInteriorPage builds synthetic pages by hand using
the same encodeHeader the production builder uses.
Companion package coverage 91.0% → 91.1%; DecodeInterior
function coverage 76.9% → ~100%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six tests for the wire-level guards:
DecodeLeaf:
- bad kind (root page) — only PageKindLeaf accepted
- 1-byte payload — less than the 2-byte record-count header
- bad record varint — 4 continuation bytes with no terminator
trip binary.Uvarint's n<=0 return path
- short record body — varint claims 1000 bytes, payload has
fewer
EncodeTrailer:
- pageSize < PageHeaderSize+TrailerPayloadSize — rejected
- TrailerVersion != 0x01 — only v1 supported
Companion package coverage 91.1% → 91.4%; per-function
DecodeLeaf 76.9% → ~100%; EncodeTrailer 76.9% → 92%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two tests for the pack-time validation arms:
- empty keyword (r.Kw == "") rejected before bencode runs
- oversize keyword (>MaxKeywordBytes) rejected before
EncodeLeaf attempts to fit it
Validating at pack time (not at every per-page Encode call)
saves both error-message clarity and a redundant traversal of
the records list.
Companion package coverage 91.4% → 91.6%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two tests for the inner-loop error arms of BTreeReader.Find:
- leaf-fetch error: failOnPagePageSource wraps a real
BytesPageSource and errors only on a specific Piece(idx)
call, letting OpenBTree's root + trailer fetches succeed
while tripping the leaf-fetch arm during Find.
- leaf-decode error: corrupt the magic bytes of leaf piece
index 1 after OpenBTree returns; DecodeLeaf rejects and
Find wraps the error.
Both arms previously relied on the happy path being the only
exercised branch.
Companion package coverage 91.6% → 92.1%; Find function
coverage 77.4% → ~95%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four tests for the per-field length guards in DecodePPMI: - IH != 20 bytes (0/10/30): the BEP-3 infohash invariant - Commit != 32 bytes (16): SHA256 commit fingerprint - Topics != 32 bytes (8): cuckoo-filter digest - NextPk != 32 bytes (64): rotation pubkey Existing tests covered empty + garbage payloads only; these fill in the per-field validation arms that protect against malformed PPMI items on the DHT. dhtindex package coverage 92.9% → 93.8%; DecodePPMI 71.4% → ~100%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Direct test on the XDG-path helper used by 'swartznet aggregate build' when the caller doesn't pass --key. Locks the "/.local/share/swartznet/identity.key" suffix so the daemon and the CLI keep agreeing on where the key lives. cmd/swartznet package coverage 26.5% → 26.8%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drive a fully signed wire SyncRecord through onSyncRecords and assert the recordSink receives it. Complements the existing rejection-arm coverage in onSyncRecords with a positive end-to-end test that verifies signature validation passes and Add fires once. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The pre-existing TestNewAnacrolixPutterBadKey passed nil server alongside a short key, so the nil-server guard tripped first and the len(priv) != ed25519.PrivateKeySize check stayed uncovered. New test passes a real isolated DHT server to get past the first guard and exercise the key-size validation. Bumps dhtindex package coverage from 93.5% → 93.8%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drive a fetcher that returns non-bencode bytes so PublisherFromMetainfo's decode fails, exercising the 'case perr != nil' arm that increments Malformed without calling the sink. Closes the last 0%-covered branch in CrawlOnce's classification switch; coverage 93.5% → 93.9%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drive a fetcher that cancels the parent context after its first call so the next iteration's ctx.Err() guard fires and CrawlOnce returns early with the partial outcome and the cancellation error. Bumps CrawlOnce coverage from 91.7% → 95.8%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new tests bring SampleInfohashes from 75% → 95.8%:
- TestSampleInfohashesErrorResponse drives a y:"e" KRPC error
reply through the responder so res.ToError() returns non-nil.
- TestSampleInfohashesNodes6 includes a 38-byte compact-IPv6
entry in the response so the for-range over r.Nodes6 fires.
Total dhtindex package coverage 93.9% → 94.2%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five tests cover defensive guards in btree encoders that no existing test exercised: - EncodeInterior wrong-kind + empty-children rejection - EncodeLeaf empty-records rejection - decodeHeader short-page guard - EncodeRecord oversize-keyword guard Bumps EncodeInterior 85.7% → 92.9%, EncodeLeaf 84% → 88%, decodeHeader → 100%, and the companion package as a whole from 92.1% → 92.5%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
makeInteriorPage built pages with Version=0 (zero value), so every test that called it through DecodeInterior/DecodeLeaf was actually tripping decodeHeader's version-mismatch check (BTreeVersion=1) rather than the specific guard each test claims to exercise. The tests still passed because they only assert err!=nil, but the coverage gap was real: ~12% statement coverage where the intended guards are 0%-hit. Setting Version=BTreeVersion in the helper unblocks every guard: - DecodeInterior: 80.8% → 92.3% - DecodeLeaf: 80.8% → 92.3% - DecodeTrailer: bumped via the leaf/trailer reuse path Package coverage 92.5% → 93.1%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new tests close the last gaps in the decoder error path
that the version-zero shadow had been masking:
- TestDecodeInteriorShortPage / TestDecodeLeafShortPage feed
a sub-header-sized page so decodeHeader's length guard
fires and Decode{Interior,Leaf} surface the error rather
than panic.
- TestDecodeInteriorBadSeparatorVarint claims 1 child whose
separator-length varint is 4 unterminated continuation
bytes (binary.Uvarint returns n<=0).
Also fixes the inline TestDecodeInteriorPayloadTooLong helper
that built its header without setting Version. DecodeInterior
now 100%, DecodeLeaf 96.2%; package 93.1% → 93.4%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hand-build a trailer page whose first payload byte is 0x02 and feed it to DecodeTrailer. EncodeTrailer rejects bad versions before the round trip, so the only way to exercise DecodeTrailer's version guard is to construct the bytes directly. DecodeTrailer 90.6% → 93.8%; package 93.4% → 93.5%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three small unit tests close the last branches in the per-session
helpers:
- Finish("") routes through the SyncStatusConverged default
- ApplyEnd with a mismatched TxID hits the txid-error arm
without advancing the phase
- SetBudgets exercises the (0,0) / (0,N) / (N,0) no-op arms
Each helper goes from 83.3% → 100%; package coverage 95.8% → 96.1%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The existing TestLoadOrCreateUnreadablePath passes a directory
path, but os.Open on a directory actually succeeds on Linux and
the test ends up tripping the JSON-decode error rather than the
open-error guard the comment claims to exercise. Plant a regular
file mid-path so os.Open fails with ENOTDIR (not ErrNotExist),
exercising the actual fmt.Errorf("trust: open ...") arm.
LoadOrCreate 94.4% → 100%; package 97.0% → 98.5%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-existing TestScoreOfBranches covers most arms but skipped
the `if returned < 1 { returned = 1 }` divisor-clamp. Add a
counter with HitsConfirmed=1, HitsReturned=0 — this bypasses
the all-zeros default branch (HitsConfirmed != 0) and forces
the inner ratio computation to run with returned=0, exercising
the clamp. scoreOf 90.9% → 95.5%; package 94.7% → 95.0%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The pre-existing TestBootstrapPendingCount stopped before driving
any candidate to admission, so the two `if _, admitted :=
b.admitted[k]; admitted { continue }` arms in PendingCount stayed
uncovered. Extend the test to push cand1 over the
EndorsementThreshold (default 3 distinct endorsers, all "strong"
since no tracker is wired) and assert PendingCount drops cand1
from the pending set even though it remains in endorsements +
observed. PendingCount 83.3% → 100%; package 94.2% → 94.8%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new tests:
- TestNewBootstrapNilLookup hits the constructor's
`lookup == nil` early-return.
- TestIngestEndorsementOnAdmittedFreshEndorser pre-admits a
candidate via the bloom-policy path so its endorsements
map is still empty, then drives IngestEndorsement to
exercise the make([...]) allocation arm in the
already-admitted branch.
NewBootstrap 80% → 84%; IngestEndorsement 88.9% → 94.4%;
package 94.8% → 95.3%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DefaultBootstrapOptions returns positive values for every numeric
knob, so existing tests bypass the four `<= 0` default-fill
guards and the nil-log substitution. Pass BootstrapOptions{} +
nil log so all five fallback assignments run. NewBootstrap
84% → 100%; package 95.3% → 96.4%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Existing tests index torrents with a single tracker so Bleve returns h.Fields[fieldTrackers] as a string, matching the type-switch's first arm. Index a torrent with two trackers so the same field comes back as []any and exercises the slice arm's range loop. Index.Search 93.2% → 98.3%; package 93.5% → 94.2%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The pre-existing test only exercised the nil-engine and unknown-infohash error paths. Add a test that drives both engines through AddMagnet on the same magnet so they share an infohash, then call AddTrustedPeerEngine and assert no error. testlab tests already exercise this in integration runs but their coverage doesn't roll up to the engine package. AddTrustedPeerEngine 87.5% → 100%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pass a directory path to VerifyFile. os.Open succeeds on a directory (Linux), but reading from the resulting fd returns EISDIR, exercising the wrapped "signing: read" error path that no existing test reached. VerifyFile 87.5% → 100%; package 92.5% → 93.8%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-create a non-empty directory at the .tmp path so os.WriteFile fails at truncate-open, exercising the "signing: write tmp" error arm. The remaining 15.4% gap in SignFile is the rename-fail arm, which is unreachable when WriteFile already failed first. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
claudenstein
pushed a commit
that referenced
this pull request
Jul 19, 2026
…tion, sync pump R4-hunt found 12 more bugs (0 HIGH, 8 MED, 4 LOW — severity plateaued at MED since round 2). This batch fixes 4 clear, self-contained ones. #1 [MED] DNS-rebind read leak. withCSRFGuard exempted GET/HEAD/OPTIONS BEFORE the loopback-Host check the file advertises as its DNS-rebind defense, so a rebind attacker (evil.com -> 127.0.0.1, fetch http://evil.com:7654/status) could read the node's torrent list, publisher pubkey, published keywords, and reputation. Fix: run the Host check for EVERY method; the Origin/Referer cross-origin CSRF check stays write-only. Matches the documented localhost model (writes already required a loopback Host). Test cases updated + a GET-rebind case added. #2 [MED] crawl-probe bound its query socket to 127.0.0.1, so the kernel refused to send to any non-loopback DHT node — the tool could not probe a single real node (its entire purpose). Fix: bind 0.0.0.0 like the crawl command. #3 [MED] watchCompletion added COMPANION bookkeeping-torrent infohashes to the known-good Bloom on completion, monotonically filling the fixed-size spam-signal filter (violating companion isolation, eroding Layer-D scoring). Fix: gate the Bloom auto-confirm on !h.companion (promotion stays unconditional per §6), mirroring autoIndex's guard. #12 [LOW] StartSync's registerSyncSession overwrote an incumbent session at the same txid WITHOUT StopPump (initiator/responder txid spaces collide), orphaning its pump goroutine. Fix: StopPump the incumbent before replacing, mirroring registerSyncSessionIfUnderCap. Both binaries rebuilt; changed packages race-clean; web smoke 21/21; gofmt/vet clean. Round-4: 4 of 12 fixed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.