Overnight/test harness 2026 04 20#10
Merged
Merged
Conversation
CLAUDE.md states the global --no-index flag "prevents Bleve from opening at all (and cascades to disable Layer D publishing)" but only the first half held: the daemon skipped indexer.Open while engine.startPublisher still launched the BEP-44 keyword publisher under the user's identity. A user running `swartznet add --no-index` was still announcing keyword pointers to the DHT — a privacy regression that contradicted the documented global opt-out. Mirror the daemon-level NoIndex flag into Config.NoIndex, gate startPublisher (and the sn_search Publisher capability bit) on it alongside the existing DisableDHTPublish, and lock the cascade in with a regression test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three small follow-ups from the whole-codebase review:
- dhtindex/dht.go — add nextSeq() helper to clamp BEP-44
sequence numbers at math.MaxInt64. The seqToPut closure
has no error return, so an int64 wrap to negative would
silently violate BEP-44 monotonicity. Defensive only:
unreachable in practice (≈146 trillion years at one
put/sec).
- CLAUDE.md / docs/05 — remove the empty internal/search
reference and the "shared SearchResult struct" claim.
Each layer carries its own response type
(indexer.SearchResponse, swarmsearch.QueryResponse,
dhtindex.LookupResponse) reconciled at the httpapi
boundary. Doc was aspirational; code never had it.
- cmd_create.go — document why the Create subcommand spins
up engine.New directly instead of going through
daemon.New (one-shot tool, no need for indexer /
publisher / API subsystems).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related GUI-test cleanups from the whole-codebase review:
(1) skip_helpers_test.go introduces skipMissing(t, d, name)
which logs the daemon's full wiring state (Eng / Index /
CompPub / CompSub / API / Bootstrap) before t.Skipf. The
bare `t.Skip("daemon did not wire up X")` pattern gave no
signal about whether the failure was X-specific (real
regression) or environmental (DHT unavailable, so
everything is nil); the diagnostic form makes the
difference obvious in `go test -v` output. Applied to
every gui-test silent-skip site.
(2) Five "no-panic only" tests now assert observable side
effects:
- run_search_empty_test.go: searchBtn stays enabled,
statusLbl stays empty, resultBox preserves preexisting
content. Catches a regression that fell through the
empty-query early-return.
- selected_actions_test.go: each action recovers from
any nil-dl.d panic, asserts dl.selected and dl.snaps
stay unchanged.
- pollloops_sync_test.go (status): asserts
torrentsLabels[0] flips off the "-" placeholder after
the initial refresh runs.
- pollloops_sync_test.go (companion): asserts pollLoop
exits within 500ms of ctx cancel — catches a dropped
ctx.Done() arm that would otherwise hang to the next
4s tick.
- app_loops_tick_test.go + downloads_pollloop_tick_test.go:
same 500ms-exit-budget assertion plus dl.snaps
non-empty after the first tick (proves tick.C body
actually ran).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion-index pointers used to carry just the infohash:
{ih: <20-byte>}
A subscriber following a publisher that stops re-publishing
would silently keep chasing the last known infohash with no
signal that the publisher had gone offline. Add a publisher-
asserted Unix-seconds timestamp:
{ih: <20-byte>, ts: <int64>}
so subscribers can detect stale pointers and apply policy
(warn, deprioritize, drop the follow). Wire-compat is
preserved by bencode's extension model — pre-ts publishers
emit `{ih: ...}` and decoders treat TS=0 as "unknown freshness,
accept"; pre-ts subscribers ignore the extra dict entry. No
coordinated upgrade required.
GetInfohashPointerInfo is the new freshness-aware accessor;
the legacy GetInfohashPointer is now a thin wrapper, so every
existing caller (companion subscriber, all test fakes) keeps
working without changes. Added pointer_ts_test.go with round-
trip, omitempty-on-zero, legacy-decode tolerance, and
forward-compat assertions to lock in the wire contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two extractor robustness improvements:
- engine.New now passes a 100 MiB global per-file extract
cap to NewPipeline instead of 0 (= "each extractor's own
default"). Previously the archive extractor enforced 64
MiB but PDF / EPUB / DOCX / FB2 silently buffered whatever
the file claimed; a torrent of pathological PDFs could
blow up resident memory before any per-extractor limit
kicked in.
- safeExtract arms a soft 60s watchdog that emits a
`pipeline.extract_slow` warning when an extract exceeds
its budget. Soft because Go can't terminate goroutines
externally — the watchdog observes and reports, the
worker keeps going. Real protection still comes from the
size cap plus per-extractor io.LimitReader plus panic
recovery; the watchdog surfaces hangs in logs before the
pipeline channel backs up.
The Extractor interface still doesn't take a context.Context
— plumbing one through every extractor is a larger refactor
not justified by current threat modelling. Size cap +
watchdog addresses the practical concern (memory blowups,
silent hangs) without the churn.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A multi-package review produced 55 adversarially-verified findings across 13 packages; all are fixed here, each with a regression test (except pure-comment nits). No on-wire bytes changed and Layer L/S/D isolation is preserved, so the mainline-compat matrix stays green. Untrusted-input availability (remote DoS, all unauthenticated): - engine: reject zero infohash from untrusted BEP-46 pointer before AddTorrentInfoHash (panicif.Zero crash); recover() on the add path - indexer/extractors: subtitle honours maxBytes; MKV/readFull bound allocation from untrusted EBML VINT; ZIP/XML/PDF extractors cap decompressed output (zip-bomb); ID3 tagSize clamped - companion: B-tree reader requires strictly-downward/increasing/ in-range child indices + depth budget + visited-set + leaf cap (self/back-pointing/DAG pages no longer overflow the stack) - swarmsearch: responder sync sessions bounded per peer + staleness reaper emitting sync_end aborted (unbounded-memory DoS) - dhtindex/indexer: DHT decode + infohash->query paths size-bounded / structured (no QueryString interpolation) Fail-closed determinism: - dhtindex: publish requires >=1 confirmed node before MarkPublished; manifest size estimate reserves the timestamp width - cli: explicit --key/--identity are load-only (no implicit mint); defaultIdentityPath honours XDG; --dht-insecure/--regtest gated; config.Validate fails closed on those flags outside tests - daemon: companion publisher writes to CompanionDir (was DataDir); anchor-fetch goroutine cancel-tracked + joined on Close; HTTPS bootstrap rejects non-https URLs - engine: restored paused torrent no longer flips files to Normal Concurrency, correctness & identity: - indexer: extract watchdog runs Extract against a hard deadline; deleteByQueryLocked bounded; Stats no longer truncates past 64k content docs; AllTorrentDocs preserves SignedBy - swarmsearch: RIBLT symbol Index validated; ShareLocal==1 fails closed; StartFeeler idempotent; misbehavior scores charged; mergeResponses dedups per peer - security: seed-list pubkeys normalised; corrupt Bloom (m==0) rejected; Bloom double-hash forces odd stride; loaded key public half re-derived from seed and verified - gui: deterministic follow-row ordering + pubkey-keyed unfollow; file-priority Select no longer re-fires OnChanged on recycle; no false "Download complete" toasts on startup - httpapi: CSRF/Origin/Host defense + non-loopback-bind guard on mutating endpoints; search timeouts clamped; status pubkey populated 104 files changed; 32 new regression tests. Build + full non-GUI and GUI suites green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fixes) The full `go test -race ./...` sweep had three independent failure modes, all test-only — no production code is racy or buggy. internal/gui (data races): under the Fyne *test* driver, fyne.Do runs its callback inline on the spawning goroutine instead of serializing onto the single UI thread (as the real GLFW driver does). Async UI flows whose tests did not join the spawned goroutine therefore rendered concurrently with other rendering against Fyne's unsynchronized process-global font/SVG cache. Fixed by giving each test-exercised async flow a nil-in-production seam invoked after its fyne.Do returns, and having tests deterministically join on it (replacing time.Sleep drains). Seams: afterCreateTorrent (create.go, prior commit), afterRunSearch (search.go), afterRefreshPublisher (companion.go), afterSetAllPriorities (files_dialog.go), afterRemoveSelected / afterAddMagnet (downloads.go). Production logic, signatures, and off-thread behavior are unchanged. Verified across 40+ full-suite -race iterations with zero DATA RACE reports. internal/testlab (timing flakes under load): - TestLayerDPublisherRefreshKeepsItemFresh: widened the refresh-advance budget 12s->30s, and replaced the single-shot post-refresh DHT lookup with a polling retry (45s budget, 10s per attempt). A loopback DHT traversal can return zero responders under saturated -race load even when nothing is wrong; the assertion (post-refresh the keyword still resolves to our infohash) is unchanged. cmd/swartznet (timing flakes under load): widened four "did this near-instant event happen?" assertion timeouts — the two signalContext cancellation tests (1s->5s) and the two progressLoop exit tests (2s->5s). A genuinely hung loop still fails. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…aware metadata waits Fixes the engine-unit findings from the 2026-06-09 whole-codebase review: - FetchCompanionTorrent rejects companion torrents whose info dict declares more than maxCompanionBytes (32 MiB) BEFORE requesting any pieces — the declared length behind a BEP-46 pointer is fully attacker-controlled and the only previous bound was the subscriber's wall-clock FetchTimeout (disk-fill). Fail closed with an error so the subscriber records the failure. - FetchCompanionTorrent validates the untrusted info.Name as a plain file name (no separators / dot entries / empty) before joining it into DataDir; defence-in-depth on the hand-built returned path. - Inbound sn_search reply writes are now gated on a bounded writer semaphore (gatedReplyWriter); a handler streaming many replies can no longer spawn unbounded write goroutines. On overload the reply is dropped with an error so the protocol layer sees the send fail. - autoDownload / autoIndex / upgradeMagnetSession metadata waits now select on bgCtx.Done() so Close reclaims the goroutines promptly instead of leaking them for up to 5-10 minutes. - restoreEntry rejects session TorrentFile values that are not plain file names, so a corrupted manifest cannot read .torrent files outside the torrents dir. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… nits Review findings (CODE_REVIEW_2026-06-09, gui unit): - downloads: the primary selection is now keyed by infohash (selectedKey) instead of a row index. pollLoop re-sorts snaps every 2 s, so the stored index could silently point at a different torrent by the time Remove/Pause/Files/Toggle-Index fired — Remove being destructive. The key is resolved against the live snaps and cleared when its torrent disappears. - downloads+search: magnet links are built via magnetLink(), which URL-escapes the dn value (and omits it when empty). Names are remote/DHT-sourced, so an embedded '&' could inject magnet params — and search fed the result straight back into AddMagnetURI. - search: flagHit no longer falls back to demoting EVERY known indexer when a hit has no source attribution (attacker- weaponizable reputation wipe; confirmHit never fanned out). It now no-ops with a user-visible "no reputations changed" note. - search: the Limit entry is parsed strictly (strconv.Atoi, positive only) instead of lax Sscanf that let negatives and trailing garbage through. - app: pollNotifications prunes lastNotified entries for torrents no longer present, bounding the map across remove/re-add churn. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three confirmed review findings in internal/httpapi:
- /flag no longer demotes every known indexer when the infohash has
no recorded source attribution. The old fallback was attacker-
weaponizable (seed an unattributed spam hit, get it flagged, crater
the whole reputation table including trusted publishers). Now the
endpoint fails closed: 200 OK, but zero pubkeys demoted, with the
new additive FlagResponse.IndexersFlagged field reporting how many
indexers were actually penalized. TestHTTPFlagFallbackNoSources
updated to lock the fail-closed behavior.
- file-index path segment in POST /torrents/{ih}/files/{index}/priority
parsed with strconv.Atoi instead of fmt.Sscanf("%d"), which silently
accepted trailing garbage ("3abc" parsed as 3).
- /search query string capped at 1024 bytes (maxSearchQueryBytes);
oversized queries are rejected with 400 before reaching Bleve or
any swarm/DHT fan-out.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ile index Three confirmed findings from the 2026-06-09 review: - create --seed used to seed with DHT disabled and an OS-assigned port, so a trackerless seed printed "Seeding..." while being unreachable. Engine config now lives in createEngineConfig: hash-only runs keep the minimal no-DHT/no-upload shape; --seed keeps DHT and uploads on. ListenPort stays 0 in both modes — the DHT announce carries the real bound port, and an ephemeral port avoids clashing with a daemon already on the default port. - create --identity without --sign was silently ignored (unsigned torrent while the user believed their key was used); now exitUsage with a hint. - files set-priority interpolated the raw index argument into the URL path; it is now validated as a plain non-negative integer and sent in canonical decimal form. Regression tests: TestCreateEngineConfig, TestCmdCreateIdentityWithoutSign, TestCmdFilesSetPriorityBadIndex, TestCmdFilesSetPriorityIndexCanonicalized. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
readBloom validated only the upper bound of the declared bitsLen, so a truncated/corrupt/tampered bloom file declaring a large m with a small bitset loaded fine and then panicked out-of-bounds on the first Add or Test — turning a recoverable load error into a daemon-wide crash. Require an exact match bitsLen == (m+63)/64 (the writer always emits exactly that), and reject degenerate m==0 / k==0 headers that would divide-by-zero or match-everything later in indices(). Regression test hand-crafts undersized/oversized/zero-parameter files and asserts LoadOrCreateBloom errors at parse time instead of panicking later; a round-trip test pins that writer-produced files still load. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…it cap Three confirmed nits from the 2026-06-09 whole-codebase review: - daemon.New's channel-A goroutine snapshotted the anchor set once and was never spawned when the set was empty (the dev default), so anchors added later via FallbackToHTTPS were never fetched. Replace the one-shot goroutine with Bootstrap.runAnchorLoop: an initial pass over configured anchors plus a re-run on a coalesced signal that FallbackToHTTPS raises whenever it extends the anchor set. Exits on ctx cancellation; RunAnchors stays idempotent. - LoadFollowFile decoded the follow file with no size bound. Reads are now capped at 1 MiB via io.LimitReader and an oversized file is rejected wholesale (fail closed) instead of half-loading. - Bootstrap.admit counted anchors against MaxTrackedPublishers and refused admissions silently. Anchors (small, project-controlled set) are now exempt from the cap so they can neither be starved nor shrink the candidate budget, and cap refusals log a daemon.aggregate_bootstrap.admit_capped warning so a starved node no longer looks identical to a quiet network. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…asked-peer gate Three confirmed findings from the 2026-06-09 review: - sync_session: the per-session max_bytes budget (SPEC §2.3/§2.9) was declared and echoed on the wire but never incremented or compared, and ApplyRecords had no phase guard — a capability-gated peer could stream unlimited sync_records frames, each forcing up to 500 synchronous ed25519 verifications on the read-loop goroutine. ApplyRecords now requires PhaseNeeded/PhaseFulfilled, accounts every frame's record+missing bytes into bytesIn, tracks a cumulative record count (echoed as sync_end 'decoded'), and returns ErrSyncBytesBudgetExceeded once over budget. BuildRecordsFrame symmetrically accounts bytesOut. No new wire fields — this implements the already-documented limit_exceeded path. - handler: onSyncSymbols and onSyncRecords no longer log violations at Debug and limp along. Any ApplySymbols/ApplyRecords error now fails closed with the package's existing teardown pattern: terminal sync_end (limit_exceeded for budget overruns, aborted otherwise), releaseSyncSession, chargeMisbehavior. Both handlers now take the reply func threaded through handleSyncFrame. - query: routeResult/routeReject accepted frames for a live txid from ANY peer (txids are a guessable monotonic counter). Each pendingQuery now carries the immutable asked-peer set, populated before registration; frames from unasked peers are dropped and charged. Regression tests: sync_records_budget_test.go (phase guard, byte budget incl. missing-ID accounting, handler fail-closed teardown for records and symbols) and route_unasked_peer_test.go (spoofed Result/ Reject dropped + charged, asked peer still delivered). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A getput.Put traversal that reaches zero DHT nodes returns a nil error even though the BEP-44 item never landed. PutInfohashPointer and PutPPMI discarded the traversal stats and reported success, so the live companion publisher recorded a green refresh for a pointer no subscriber could resolve. Factor the zero-node assertion into a single shared helper (checkPutStats) and run it from all three put paths — keyword, BEP-46 pointer, and PPMI — so the guard cannot drift between them again. Also in this pass: - nextSeq: clamp BEP-44 sequence increments at MaxInt64 instead of raw seq+1 in all three seqToPut closures (PutPPMI was the reported offender). - decodePointerValue: bound the remote-supplied BEP-46 pointer value at MaxValueBytes before handing it to the bencode decoder (signature-bound, but cheap to fail closed on malformed input). No on-wire bytes change: rejecting oversized values and surfacing zero-node puts as errors are local policy on already-documented BEP-44 limits. Regression tests: TestPutPathsFailClosedOnZeroNodes drives all three puts against a dead bootstrap node (nil error + zero responses) and fails on the old fail-open behavior; TestDecodePointerValueBounds proves the size cap fires before the decode; TestCheckPutStatsZeroNodes and TestNextSeqClampsAtMaxInt64 pin the helpers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…il closed)
The B-tree reader's walkToLeaves guarded only against intra-page
cycles: each child had to point strictly downward, in range, and
strictly increasing within its page. That cannot see cross-page
fan-in — pages laid out i -> {i+1, i+2} satisfy every per-page
check, yet the number of root-to-leaf paths grows Fibonacci-ally
(reproduced 5M+ Piece() calls for a 40-piece file; Find("")
defeats all range pruning). Interior bytes are not covered by the
trailer signature, so this was an attacker-controlled DoS.
Fix:
- Share a visited-set across the entire walk and fail closed the
moment any page is reached twice. Total page fetches are now
bounded at NumPieces and recursion depth by the page count, so
the visited-set subsumes the previous depth budget. Honest pure
trees consume each child exactly once, so this costs honest
builders nothing.
- Correct the now-false "visited at most once" comment block.
- Defense-in-depth in Find: checkLeafIndices fails closed on
duplicate leaf indices or more leaves than pieces, so records
can never be re-verified (ed25519 + PoW SHA256) and re-appended
even if the walk invariant were ever broken.
This worktree branch predates the partial fix on main (strict
child-index guard + depth budget, commit 08b5df0), so that guard
and its tests (walk_to_leaves_cycle_test.go,
verify_fingerprint_earlybail_test.go + the VerifyFingerprint
early-bail) are ported here verbatim and the visited-set is layered
on top — merging into main reduces to taking this version.
Regression test: walk_to_leaves_fanin_test.go builds a signed
hostile file with the i -> {i+1, i+2} layout, asserts Find("")
fails closed with the re-visit error, and asserts Piece() calls
stay within 3x NumPieces (the old code blows a 3x budget
immediately).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gnedBy-only search
Three review fixes in internal/indexer:
- safeExtract now owns closing the file reader: the io.Closer
Close moved from handle()'s defer into the extracting child
goroutine's defer, strictly after Extract returns. Previously,
when the hard extract watchdog fired, handle() closed the
reader while the wedged extractor was still reading it — a
Read-after-Close data race on anacrolix torrent.Reader. This
also ports the 60s hard watchdog (extractWatchdog /
errExtractTimeout) that the race was found in; the worker
fails the file and moves on, and the leaked goroutine closes
the reader itself once Extract returns.
- AllTorrentDocs / ContentDocsForInfoHash hold the global mutex
across their deep-pagination walks; both now bound the walk
with a maxPages ceiling scaled off a cheap count-only query
(mirroring the Stats text-scan guard) and log if it ever
fires, so a pathological index that never returns a short
page cannot pin the mutex forever.
- Search accepts a SignedBy-only request (empty Query), as the
SearchRequest doc has always promised; a request with neither
Query nor SignedBy is still rejected.
Regression tests: pipeline_watchdog_close_test.go (timeout must
not close the reader under a live Extract; success/panic paths
still close it), pipeline_watchdog_hard_test.go (deadline fires),
docs_walk_backstop_test.go (full-page stub index terminates at the
ceiling with a truncation warning), TestSearchSignedByOnlyQuery.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mits Fixes the confirmed 2026-06-09 review findings for the extractors unit: - docx/odt/odp/pptx/epub: wrap every zip-entry reader in io.LimitReader(rc, maxDocTextBytes) before the XML/HTML parser. The input caps only bound the COMPRESSED bytes; a deflate bomb (~1032:1) could buffer its whole decompressed body inside one Token()/Next() call — an OOM that recover() cannot catch. PPTX and EPUB shrink the remaining budget per slide/chapter so total output is bounded too (EPUB's text budget now aligns with the 64 MiB output cap instead of reusing the 256 MiB input budget). - htmltext: SetMaxBuf on the x/net/html tokenizer (default is unlimited) plus an internal LimitReader; ErrBufferExceeded and budget-EOF are graceful truncation, so one oversized chapter no longer poisons the rest of a book. - archive: enforce the documented-but-dead 4 MiB cap on the concatenated member-name list (the tar.gz branch read names from the unbounded DECOMPRESSED stream), and bound the tar.gz walk at 1 GiB decompressed so a bomb can't burn unbounded CPU skipping one huge member. - exif: keep cnt/offset arithmetic in int64 so attacker-controlled counts can't wrap a 32-bit int past the bounds checks (verified under GOARCH=386). Regression tests: TestZipDocExtractorsBoundDecompressedEntryStream (well-formed 65 MiB bombs per format), graceful-truncation + budget-clamp tests for extractHTMLText, name-list cap tests for both archive branches, and overflow tables for parseIFD/readValueBytes — all verified to fail against the pre-fix code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the CHANGELOG entry for the 11-commit fix stack, notes the 32 MiB companion acceptance limit and visited-once B-tree walk in docs/05, rewords the stale cmd_flag comment (flag no longer fans out to every indexer), and checks in the review report the fixes were driven from. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Moves the prior whole-codebase review report + findings JSON (the campaign fixed in 08b5df0) from untracked repo-root files into docs/reviews/, dated. The follow-up 2026-06-09 review that regression-checked it lives at CODE_REVIEW_2026-06-09.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ding 0%") A freshly-created torrent could stick at downloading/0% instead of seeding/100% despite the bytes being on disk. Two paths were broken: - CLI `create --seed` used the plain AddTorrentMetaInfo (storage rooted at cfg.DataDir), so without `--data-dir <parent-of-root>` the post-add VerifyData rehashed an empty dir. It now seeds in place from <root> via AddTorrentMetaInfoSeedFrom, matching the GUI. - Renamed torrents (GUI's editable Name field, on-by-default seed checkbox; CLI --name) stayed at 0% even on the seed-from path: anacrolix's default file storage resolves files under <base>/<info.Name>/..., which no longer points at the real bytes once the display name differs from the on-disk basename. The per-torrent seed storage now keys file paths on the real on-disk basename via storage.NewFileOpts + a custom FilePathMaker, decoupled from info.Name, so a renamed torrent seeds from its real location while downloaders still see the chosen name. The basename is persisted as a new session field (content_name) so restart restores the right storage; legacy entries fall back to info.Name. Regression tests cover single-file and multi-file renames and the restart/restore path; the GUI create-seed test now asserts 0 missing bytes. Verified end-to-end with the real CLI: a renamed create --seed restores as status=seeding, 350000/350000, missing=0. Wire-compat untouched (local storage wiring only; infohash/.torrent bytes identical). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ook Phase 1) Produced by a 175-agent extraction pass: 16 parallel subsystem readers, docs-drift/TODO/surface-audit cross-checks, adversarial verification of all 148 broken/half-finished claims (135 confirmed, 13 refuted and dropped), and a completeness critic. Legacy tree preserved as the legacy-snapshot branch. Section 0 (original vision) is intentionally a stub — the Phase 2 human gate. Section 7 lists open questions only the author can answer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reconstructed the vision section from the project name, README, and design docs since the author said "continue" rather than editing it. Marked PROVISIONAL with a checklist of the load-bearing calls (mission framing, Aggregate endgame, scope exclusions, invariants, design-center user) for the author to confirm before Phase 4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fresh architecture designed from SPEC.md (not from legacy structure) via a judge panel: three architects proposed independent designs (evolutionary "keep the seams", hexagonal ports-and-adapters, capability-vertical); a 3-judge panel scored them; the top-ranked evolutionary backbone was synthesized with the hexagonal swappable-Layer-D port and the vertical frozen contracts/* golden-vector tier grafted in. An adversarial critic returned APPROVE WITH REQUIRED FIXES; all 16 fixes were folded in (transport-seam cycle, orphaned aggregate CLI, serve/status contradiction, regtest-DHT config surface, 5 preservation notes, 2 logged assumptions, L1 fail-closed watch-item). PLAN.md is a 14-slice vertical build order: walking skeleton -> identity -> add/download/seed -> create/sign -> Layer L -> trust/reputation -> capability -> Layer S + wire-compat gate -> RIBLT -> Layer D -> companion -> GUI -> Aggregate seam -> hardening. Stops here for author review before Phase 4 (implementation), per playbook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Save the full provenance behind SPEC/ARCHITECTURE/PLAN so it survives beyond the working session: - docs/rebuild/README.md — phase-state tracker + what the author must decide before Phase 4 - docs/rebuild/phase1-verification-ledger.md — how SPEC.md was extracted (175 agents) + per-claim verifier notes, incl. the REFUTED claims that must NOT be "fixed" in the rebuild - docs/rebuild/phase3-design-record.md — the 3 architecture proposals considered, judge-panel scoring, and the critic's 16 required fixes - docs/rebuild/raw/ — raw workflow JSON outputs + workflow scripts Docs only; no product code touched. Rebuild remains paused at the human gate before Phase 4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…deliverable) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e/plan Gate cleared 2026-07-17: all five §0 checklist items confirmed unamended; ARCHITECTURE.md + PLAN.md approved. Phase 4 (Slice 0) begins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…shot) Removes cmd/, internal/, go.sum and trims go.mod to the bare module. The complete legacy implementation remains untouched on the legacy-snapshot branch as the read-only behavioral reference (DECISIONS.md P1/P4). The tree is rebuilt from scratch slice by slice per PLAN.md, starting with Slice 0. dist/swartznet-legacy-v0.8.0 preserves the last legacy CLI binary locally (dist/ is untracked). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first vertical slice of the from-scratch rebuild (PLAN.md Slice 0): one daemon, one frontend, saying hello. - internal/config: XDG share-root resolution (./swartznet-state homeless fallback), Validate() with rejections-before-mkdir (§7-Q48 resolved, create-set unchanged), and the single SWARTZNET_UNSAFE=1 + testing.Testing() unsafe gate — SWARTZNET_ALLOW_REGTEST is gone and a tree-grep test keeps it gone. - internal/httpapi: zero-subsystem-import server with the legacy CSRF/ DNS-rebind guard (exact 403 bodies), 1 MiB body cap (bodylimit ⊃ CSRF ⊃ mux), hardening knobs, frozen full /status DTO tree (honestly degraded at this slice), /healthz with instance-scoped version, embedded web stub, bind-and-warn non-loopback semantics, and a double-Start guard. - internal/daemon: single constructor, degraded-start warnings, teardown-aware goBG registration (no Add/Wait race), reverse-order idempotent Close with observable log vocabulary (daemon.close_begin → daemon.bg_joined → httpapi.stopped → daemon.close_done). - cmd/swartznet: dispatch with the documented exit-code contract (0/1/2/130), temporary 'serve' scaffold (folds into 'add' in Slice 2), thin-HTTP-client 'status' byte-compatible with legacy output incl. the per-keyword table, SWARTZNET_LOG documented in help with a warning on unrecognized values. - Tests: table-driven unit suites incl. a real-signal-path test (SIGINT/SIGTERM → exit 130) and golden /status bytes; go test -race clean; scripts/dod-slice0.sh reproduces the 37-check binary DoD run. - Docs: CHANGELOG slice entry; DECISIONS.md S0-1…S0-15; SPEC §2.1 share-root drift corrected; ARCHITECTURE.md middleware-order prose fixed. Adversarially reviewed by a 4-lens workflow before commit; all findings fixed and pinned by tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second vertical slice (PLAN.md Slice 1): the daemon loads-or-creates the ed25519 identity and the publisher pubkey reaches /status. - internal/identity: frozen raw-64-byte identity.key format; exact-0600 gate (0400 rejected) and all legacy validation error strings preserved byte-verbatim; seed→pubkey re-derivation on load; a present-but-invalid key is never overwritten or regenerated (invariant #2). Auto-create is gated on allowCreate inside Load's create branch — the single enforcement site, closing the legacy stat-then-create TOCTOU. Parent mkdir moved to the create arm (refused loads have zero side effects); post-create Chmod defeats hostile umasks (pinned under umask 0277). - daemon: identity loads before all subsystems; path-based auto-create rule (cleaned compare against config.Default().IdentityPath); §6 fix — httpapi Options.PublisherPubKey wired from identity.PublicKeyHex, and handleStatus renders it un-nested from any publisher collaborator. New log events daemon.identity_loaded / daemon.identity_load_err. - serve --identity: explicit path load failure is fatal (exit 1); default-path failure degrades per SPEC §2.8 (warn + publisher-less). - config.IdentityPath (empty = identity off); Validate never touches it. - Tests: full legacy inventory ported to Load (perm/size/corrupt/dir/ mismatch/symlink/umask), daemon persistence + degradation + defect- absence tests, hermetic XDG isolation everywhere; -race clean. - scripts/dod-slice1.sh: 19 binary DoD checks (create/reject/recover/ rotate/--identity); dod-slice0.sh now hermetic + pubkey-aware golden. Adversarially reviewed (4 lenses); all findings fixed and pinned. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… publish B9 [LOW] concurrent PATCH /capabilities could lose a field. handleSetCapabilities did a non-atomic read-merge-write across two separately-locked collaborator calls (Sharing() then SetSharing()), so two overlapping requests each read the same base and the last writer clobbered the other's toggle. Fix: serialize the read-merge- write under a new Server.capMu. B11 [LOW] a torrent whose cached name alone exceeds the BEP-44 1000-byte value cap made Manifest.AddHit evict the ONLY hit to an empty list and report success, so Publish put a useless empty KeywordValue and the torrent was undiscoverable under its own keyword. Fix: never evict the last hit — truncate its cached name to fit (fitHitToCap), keeping a non-empty, in-cap, still-discoverable value. Regressions: TestManifestAddHitKeepsSingleOverCapHit + the ordinary multi-hit eviction still works. Both binaries rebuilt; new tests race-clean. Remaining round-2: B4 (GUI follow persistence), B7 (sync >500-record chunking), B10 (late-metadata activation), B12 (sync-session release on peer close). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
B4 [MED] the native GUI's "Follow publisher" called CompSub.Follow directly on the raw subscriber worker, which mutates only the in-memory follow set — so a follow added in the GUI was silently lost on restart (LoadFollowFile read a file the GUI never wrote). The atomic follow-file persistence lived only in the companionAdapter used by the HTTP API. Fix: the persisting adapter is now built unconditionally and stored on the Daemon (d.compController); both the HTTP API and the GUI mutate follows through it via new Daemon.FollowPublisher / UnfollowPublisher methods. The GUI dialog routes through FollowPublisher (and now surfaces its error) instead of the raw worker, so a GUI follow survives a restart exactly like an API follow. The HTTP API reuses the same controller, so both frontends share one follow-file writer. Both binaries rebuilt; daemon + companion tests green. 9 of 12 round-2 bugs fixed; remaining: B7 (sync >500-record chunking), B10 (late-metadata activation), B12 (sync-session release on peer close). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…peer close B10 [LOW] a magnet/infohash whose metadata arrived after a 5-minute cap was left permanently un-downloadable. autoDownload (and autoIndex) selected on GotInfo with a `case <-time.After(5*time.Minute): return`, so when metadata resolved later the goroutine had already given up and nothing flipped file priorities off None (nor indexed/minted/published) — no automatic recovery. Fix: drop the wall-clock cap; the goroutine is already bounded by engine close (bgCtx) and torrent removal, so it now activates/indexes whenever metadata arrives. B12 [LOW] OnPeerClosed leaked in-flight sync sessions. On disconnect it dropped per-peer state but never released the peer's RIBLT sync sessions, so a responder's symbol pump kept producing to a dead PeerToken until its budget exhausted and the SyncSession lingered in the map until the lazy reaper happened to run. Fix: a releaseAllSyncSessions(addr) that StopPumps + forgets every session for the peer, called from OnPeerClosed. Regression: TestOnPeerClosedReleasesSyncSessions. Both binaries rebuilt; swarmsearch race-clean; engine suite green. 11 of 12 round-2 bugs fixed; only B7 (sync >500-record chunking) remains. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…iff transfers
B7 [MED] a one-directional reconciliation difference larger than
MaxRecordsPerMessage (500) silently transferred ZERO records while still
reporting convergence — exactly the large-catch-up case reconciliation exists to
serve (a fresh node syncing 600 records a peer holds). onSyncNeed and
initiatorConverge each built a single BuildRecordsFrame(found) which returned
ErrSyncTooLarge for >500 records; the `if err != nil { return }` then sent
nothing (no records, no sync_end), and the initiator's session lingered until the
2-minute stale reaper while WaitSyncConverged had already reported success.
Fix: a sendRecordsChunked helper splits the records into ≤MaxRecordsPerMessage
frames and sends each (the receiver ingests every sync_records frame
independently; `missing` rides the final frame; at least one frame always ships).
Both the responder's sync_need reply and the initiator's proactive push now
chunk. Regression: TestSyncReconcileOneDirectionalOverCap (600-record one-way diff
fully reconciles).
Both binaries rebuilt; full swarmsearch suite race-clean. This completes all 12
confirmed round-2 bugs (2 HIGH, 6 MED, 4 LOW) — every one fixed + regression-tested.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dAll (fix e2e regression) The initial B1 fix (c8b4c60) refused a companion fetch whenever the infohash collided with a non-companion handle. That was too aggressive: the 2-engine companion e2e legitimately pre-adds the companion infohash as a plain handle (a DHT-off peer-wiring shortcut), which is indistinguishable at fetch time from a malicious collision — so dod-slice10's real-transfer scenario regressed with "collides with a live torrent; refusing". Refined defense, same guarantee without the false positive: - DropCompanionTorrent still refuses to drop a non-companion handle — this is what actually prevents the HIGH-severity destruction of a live torrent. - FetchCompanionTorrent no longer refuses outright; instead it gates h.T.DownloadAll() on h.companion, so a colliding REAL torrent's file priorities are never overridden. A colliding real torrent is thus never mutated (no drop, no priority override); the companion-index decode simply fails and the fetch returns an error. - A legitimate companion fetch (fresh companion handle) still DownloadAll's; the e2e's pre-added plain handle downloads via its own autoDownload path. Regression test refocused on the load-bearing protection (TestCompanionDropGuardProtectsLiveTorrent: a companion drop never removes a non-companion torrent). dod-slice10 back to 21/21; wirecompat/scenarios green; both binaries rebuilt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…esurrect Round-3 adversarial hunt found 14 new bugs (0 HIGH, 10 MED, 4 LOW — a sharp severity drop from round 2). This batch fixes 4 high-value ones. R9+R10 [MED] DHT/PPMI decode allocation amplification. DecodeValue and DecodePPMI capped the outer payload at 1000 bytes but then called bencode.Unmarshal, which uses anacrolix's ~128 MiB default MaxStrLen and allocates make([]byte, declaredLen) BEFORE reading. A 16–21-byte hostile value declaring a ~128 MiB inner string (e.g. `d2:ih134217727:e`) forced a transient ~128 MiB allocation on the Layer-D / Aggregate resolve paths — a remote memory-amplification DoS (Query fans out one goroutine per indexer, so N gossiped keys → N spikes). Fix: a shared decodeBounded helper decodes with Decoder.MaxStrLen = MaxValueBytes, so an oversize string is rejected without the allocation. Golden vectors unaffected. Regressions: TestDecodeValue/PPMIBoundsAllocation. R2 [MED] `trust add <pubkey> --file <path>` silently swallowed "--file <path>" into the label (Go flag stops at the first positional), wrote to the DEFAULT trust store, and exited 0 — a silent data-integrity/security defect (the default store gates auto-confirm). Fix: apply parseFlagsAllowingLeadingPositionals (the same helper used for create/files/confirm/flag), so --file is honored before or after the positionals. R3 [MED] a removed torrent could resurrect in session.json. persistState (pause/resume/set-indexing) used the unconditional session.update upsert; racing a concurrent RemoveTorrent (which releases e.mu between deleting the handle and the session row), it re-inserted the deleted entry, so the torrent silently rejoined its swarm on the next restart. Fix: use updateExisting (the guarded method built for exactly this). Regression: TestSessionUpdateExistingDoesNotResurrect. Both binaries rebuilt; changed packages race-clean; gofmt/vet clean. Remaining round-3: R1 search --json schema, R4 content-doc hijack, R5-R8, R11-R14. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… primary miss
R7 [MED] composite.Lookup early-returned (nil, err) on ANY primary error and
never consulted the secondary, contradicting its documented dual-read merge. But
a routine keyword miss surfaces as an error from every Getter ("not found"), and
an indexer running LayerDMode=aggregatePPMI has NO legacy BEP-44 item at all — so
a composite-mode searcher querying such an indexer got ZERO hits for a keyword it
actually publishes in its aggregate tree, defeating the migration fill-in the
composite exists for.
Fix: query both backends and merge; only fail when BOTH error. A primary "not
found" now falls through to the secondary's hits. Regression:
TestCompositePrimaryMissDoesNotDropSecondary (existing dual-write + broken-
secondary tests still pass).
Changed package race-clean; gofmt/vet clean. Remaining round-3: R1 (search --json
schema), R4 (content-doc hijack), R5/R6, R8 (dup sync_begin), R11-R14.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…p leak R4 [MED] a followed publisher could overwrite the node's OWN locally-extracted content docs and relabel torrent metadata. IndexContent had no provenance guard (content docs carry no signer), so a companion snapshot listing an infohash the victim already extracted overwrote c:<ih>:<file>:<chunk> with the publisher's text; and B6's PreserveExistingSigner protected only SignedBy, so Name/FilePaths/ Size were still relabeled. Fix: - ContentDoc.PreserveExisting (set by the subscriber): IndexContent skips a write whose doc already exists, so the node's own extraction is never clobbered; a genuinely new content doc is still imported. - IndexTorrent's PreserveExistingSigner now SKIPS the entire write (new sentinel indexer.ErrForeignTorrent) when the torrent is already attributed to a DIFFERENT publisher — protecting Name/FilePaths, not just SignedBy. - The subscriber treats ErrForeignTorrent as a benign skip: it skips that torrent's metadata AND content and does not count/fail it. Regressions: TestIndexContentPreserveExistingBlocksOverwrite + the updated hijack test; dod-slice10 legit-import e2e still 21/21. R8 [MED] a duplicate sync_begin with the same txid orphaned the first symbol pump. registerSyncSessionIfUnderCap overwrote the incumbent session WITHOUT StopPump, so its runSyncPump goroutine became unreachable (no StopPump call site, not released by OnPeerClosed) and streamed to a dead token to budget exhaustion — a per-peer-cap bypass + pump leak surviving disconnect (which B12's fix could not reach). Fix: StopPump the incumbent before replacing it. Regression: TestReBeginStopsIncumbentPump. Both binaries rebuilt; changed packages race-clean; gofmt/vet clean. Round-3: 7 of 14 fixed; remaining R1, R5, R6, R11-R14. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…R14 dedup prune R11 [LOW] `companion follow <pubkeyA> <pubkeyB> --api-addr X` silently discarded the second positional AND the post-positional --api-addr (dialing the default :7654), following only pubkeyA with no error. Fix: use parseFlagsAllowingLeadingPositionals and require exactly one positional, so an extra arg is a usage error and flags after the pubkey are honored. R13 [LOW] engine.New leaked the swarm's announceWorker goroutine on the one fatal path where loadSession returns a nil session (torrents-dir mkdir failure): that branch did bgCancel()+cl.Close() but not swarm.Close(), and only swarm.Close closes p.done (bgCancel does not stop announceWorker). Fix: swarm.Close() on that path too, matching the NewClient error branch. R14 [LOW] the subscriber's dedup maps (imported/lastIH, keyed by pubkey) were never pruned — SubscriberWorker.Unfollow deleted only the worker's follow/lastSync maps — so they grew one permanent entry per distinct publisher ever synced. Fix: Subscriber.Forget(pubkey) prunes both maps; Unfollow calls it (a refollow starts fresh). Both binaries rebuilt; changed packages race-clean; gofmt/vet clean. Round-3: 10 of 14 fixed; remaining R1 (search --json schema), R5 (refetch efficiency), R6 (follow-file size cap), R12 (--help exit code). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
R6 [MED] persistFollows wrote json.MarshalIndent (pretty-printed, ~2x compact) with no size cap, while LoadFollowFile fails closed on any file > 1 MiB. With labels, ~8300 follows exceeded 1 MiB — below the constant's own documented ~10k — so a valid large follow set wrote fine but was rejected WHOLESALE on the next restart (only a log warning), silently wiping the user's entire follow list. Fix: write COMPACT JSON (halves the on-disk size, so compact ~90 B entries put 10k follows at ~0.9 MiB) and raise the read cap to 4 MiB for comfortable headroom even with long labels. An over-cap file still fails closed wholesale (not a silent suffix drop). daemon tests race-clean; both binaries rebuilt. Round-3: 11 of 14 fixed; remaining R1 (search --json schema), R5 (refetch efficiency), R12 (per-command --help exit code). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
R1 [MED] `search --json` (no --swarm/--dht/--signed-by) emitted two incompatible
JSON schemas for the identical invocation: with no daemon, searchDirect dumped
indexer.SearchResponse (flat, untagged `{"Total":..,"Hits":..}`); with a daemon
holding the index lock, the T3-2 lock-fallback routed through searchViaAPI and
dumped httpapi.SearchResponse (nested `{"local":{...}}`). A scripted `... | jq
'.Total'` returned the count with no daemon and null with one. Fix: searchDirect
now converts its result to the httpapi `{"local":{...}}` envelope
(toAPILocalBlock), so both paths share one stable schema. Regression:
TestSearchDirectJSONSchemaMatchesAPI.
R12 [LOW] every subcommand's `--help`/`-h` exited 2 (usage error) while the
top-level `swartznet --help` exits 0 — `cmd --help && echo ok` never printed ok.
Go's flag package returns flag.ErrHelp (after printing usage), which was mapped
to exitUsage. Fix: a shared parseErrExit(err) maps flag.ErrHelp → exitOK and any
other parse error → exitUsage, routed through all 18 subcommand parse sites; the
companion subcommand dispatcher also handles help. A genuine parse error still
exits 2. Regression: TestSubcommandHelpExitsZero.
Both binaries rebuilt; cmd/swartznet race-clean; gofmt/vet clean. Round-3: 13 of
14 fixed; only R5 (refetch efficiency) remains.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…anged corpus R5 [MED] BuildFromIndex stamped GeneratedAt=now on every rebuild and the publisher republished unconditionally each interval, so an UNCHANGED corpus still produced a new companion infohash every interval (GeneratedAt is inside the hashed payload). Every follower therefore re-downloaded the whole companion .torrent and re-ran the full IndexTorrent/IndexContent ingest for the entire corpus every interval, forever — cost scaling with corpus size × follower count, and SyncResult.Deduped essentially never firing in production. Fix: the publisher fingerprints the corpus (torrents + content, XOR of per-torrent hashes — order-independent, excludes the timestamp). When a rebuild's content is unchanged, it reuses the prior GeneratedAt so the payload — and thus the infohash — is byte-identical; the BEP-44 pointer still re-publishes (TTL refresh) but at the SAME infohash, so followers' pointer dedup fires and they do not re-fetch. Content changes still mint a fresh timestamp/infohash. Regression: TestPublisherReusesInfohashWhenCorpusUnchanged (drop-on-change test still passes). Both binaries rebuilt; companion race-clean; dod-slice10 21/21; gofmt/vet clean. Round-3 COMPLETE: all 14 confirmed bugs fixed + regression-tested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The B7/R7 records-chunking regression test passed in isolation (~1.6s under -race) but flaked in the full multi-package -race run: CPU starvation from all packages running at once slowed the heavy 600-record reconciliation past its 15s deadline (it reached 325/600 — still converging, records were NOT dropped, so this is a timing flake, not a chunking bug). Reduce the diff to 520 records (still > the 500-record MaxRecordsPerMessage cap, so 2-chunk delivery is exercised) and raise the deadline to 40s. Verified passing with the 5 heaviest -race packages running concurrently. Product code unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tion, sync pump R4-hunt found 12 more bugs (0 HIGH, 8 MED, 4 LOW — severity plateaued at MED since round 2). This batch fixes 4 clear, self-contained ones. #1 [MED] DNS-rebind read leak. withCSRFGuard exempted GET/HEAD/OPTIONS BEFORE the loopback-Host check the file advertises as its DNS-rebind defense, so a rebind attacker (evil.com -> 127.0.0.1, fetch http://evil.com:7654/status) could read the node's torrent list, publisher pubkey, published keywords, and reputation. Fix: run the Host check for EVERY method; the Origin/Referer cross-origin CSRF check stays write-only. Matches the documented localhost model (writes already required a loopback Host). Test cases updated + a GET-rebind case added. #2 [MED] crawl-probe bound its query socket to 127.0.0.1, so the kernel refused to send to any non-loopback DHT node — the tool could not probe a single real node (its entire purpose). Fix: bind 0.0.0.0 like the crawl command. #3 [MED] watchCompletion added COMPANION bookkeeping-torrent infohashes to the known-good Bloom on completion, monotonically filling the fixed-size spam-signal filter (violating companion isolation, eroding Layer-D scoring). Fix: gate the Bloom auto-confirm on !h.companion (promotion stays unconditional per §6), mirroring autoIndex's guard. #12 [LOW] StartSync's registerSyncSession overwrote an incumbent session at the same txid WITHOUT StopPump (initiator/responder txid spaces collide), orphaning its pump goroutine. Fix: StopPump the incumbent before replacing, mirroring registerSyncSessionIfUnderCap. Both binaries rebuilt; changed packages race-clean; web smoke 21/21; gofmt/vet clean. Round-4: 4 of 12 fixed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…torrents #6 [MED] completes the R4 authorship-hijack class. The ErrForeignTorrent guard only fired when a DIFFERENT NON-EMPTY signer was already stored, so a followed publisher could still overwrite the Name/FilePaths/Size + attribution of a torrent the node downloaded from a plain magnet (SignedBy "") — the common case: the node's own "Ubuntu ISO" gets relabeled "FREE MONEY" attributed to the attacker in its Layer-L index. Fix: IndexTorrent (PreserveExistingSigner) now checks doc EXISTENCE (bleve.Document) — if the node already holds the torrent under ANY different signer, including unsigned (stored ""), it returns ErrForeignTorrent and skips the whole write; the subscriber then skips its content too. A new torrent (no doc) or one already attributed to the same publisher is stamped normally. Regression: TestIndexTorrentPreserveExistingBlocksUnsignedRelabel; existing hijack/content tests + dod-slice10 legit-import e2e (21/21) still pass. Both binaries rebuilt; indexer/companion race-clean; gofmt/vet clean. Round-4: 5 of 12 fixed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the round-4 adversarial backlog; each fix has a regression test that fails when the fix is neutralized. All touched packages race-clean. - #10 companion: put the new BEP-46 pointer BEFORE dropping the old seed, so a failed put keeps the still-advertised infohash fetchable. - #7 dhtindex: TOCTOU between off-lock aggregate Refresh and a concurrent Retract — added a gen counter + post-distribute re-check so a retracted tree isn't left published for a full refresh interval. - #9 indexer/httpapi/daemon: a malformed query string is now a typed ErrBadQuery → HTTP 400 (client fault), not 500; classified at the daemon seam via a LocalBadRequest flag (httpapi imports no indexer types). - #5 companion: publisher GeneratedAt is now strictly monotonic across content changes, so a backward wall-clock step no longer trips the follower's replay guard and drops legitimate new content. - #11 companion: an Unfollow racing an in-flight Sync no longer resurrects the dedup maps — per-publisher Forget epoch, re-checked under lock in commitDedup. - #4 engine: persistAdd runs after Add releases e.mu; a racing RemoveTorrent could be resurrected — updateGuarded(abort) + Handle.isRemoved() make the create happens-before-correct (RemoveTorrent closes h.removed before its lock-guarded remove); persistSeedAdd DataPath refinement → updateExisting. - #8 swarmsearch: initiator/responder txid collision deferred with a fix sketch (needs a wire direction bit + the unwired StartSync path) — see DECISIONS T3-8. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convergence hunt (7 surfaces, adversarially verified). The dedicated regression finder cleared the round-4 fixes (no new defects); six NEW distinct bugs fixed, each with a neutralize-to-fail test. - HIGH (contracts/ltepwire): every sn_search decoder used unbounded bencode.Unmarshal (~128 MiB default MaxStrLen); a ~32-byte frame declaring a huge inner string allocated ~120 MiB before failing, on every inbound frame across up to 256 workers — remote OOM DoS. Added decodeBounded (MaxStrLen = len(payload)) across all 11 decoders, the ltepwire counterpart of dhtschema.decodeBounded. - MED (engine/queue): promoteQueued unconditionally counted an activated handle; a queued COMPLETE seed consumed a slot and starved a real download. Count only when isActive(). - MED (companion/publisher): a pointer-put failure after a content change left the freshly-seeded companion torrent seeded forever (referenced by no live pointer). Drop it on failure unless still advertised. - LOW (engine/index): re-enabling SetTorrentIndexing never re-wrote the torrent-level doc. Extracted writeTorrentDoc, called from both paths. - LOW (engine/companion): concurrent FetchCompanionTorrent of one infohash shared a handle; per-fetcher deferred drop tore it down mid-download. Reference-count so the drop fires only on the last fetcher. - LOW (contracts/snagg): DecodeLeaf/DecodeInterior pre-allocated from an untrusted uint16 count (~10 MB / ~2 MB). Cap via capHint(n, remaining). All touched + downstream packages race-clean; both binaries rebuilt. Convergence: r2=2 HIGH, r3-4=0 HIGH, r5=1 HIGH — a real distinct DoS, so NOT converged; a round 6 is warranted. See DECISIONS log 2026-07-19. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Decode-audit-led hunt (the class that produced the r5 HIGH). Each fix has a neutralize-to-fail regression test; all touched + downstream race-clean. - MED (contracts/snagg): DecodeRecord used unbounded bencode.Unmarshal — a ~15-byte hostile Aggregate-tree record forced a ~128 MiB alloc via an unauthenticated leaf page (the r5 snagg fix only capped the outer make-hint, not this inner record decode). Bounded to MaxStrLen = len(b). - MED (internal/dhtindex): the Layer-D indexer set (NotePublisherSeen) was uncapped and fed from untrusted synced-record pubkeys — a flood grows it and per-query DHT fanout without bound. Cap the auto/gossip set at 128 with FIFO eviction; explicit operator indexers exempt. - MED (internal/engine): residual TOCTOU in the r5 companion-fetch refcount — releaseCompanionFetch dropped the shared torrent outside companionFetchMu, so a concurrent fetcher could attach in the gap and hold a torn-down handle. retainAndAttachCompanion makes retain+attach and release+drop each atomic under the lock (order companionFetchMu -> e.mu). Added a -race stress test. - MED (internal/swarmsearch): the responder enforced FileHits/ContentHits only against explicit scope, never the response payload — scope="n" leaked per-file paths + content matches. hitsToWire now enforces the node's own caps. Convergence: r2=2 HIGH, r3-4=0 HIGH, r5=1 HIGH, r6=0 HIGH. The decode audit closed the last alloc-amplification sites (dhtschema, ltepwire, snagg x2). The r5-regression finder caught the engine self-regression above. See DECISIONS 2026-07-19. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Targeted the un-hunted document-extractor surface (parsers of downloaded, attacker-controlled files) + persistence robustness. The web/httpapi, r6-regression, and completeness-critic finders returned zero (r6 fixes clean). Each fix has a neutralize-to-fail test. - HIGH (indexer/extractors): the PDF extractor's decompressed-text cap was on the wrong side of the allocation — reader.GetPlainText() materializes the whole document's text before the io.LimitReader runs, so a small PDF whose FlateDecode content stream decompresses (up to ~1032:1) to gigabytes of text OOM-crashes the daemon (recover() can't catch a runtime OOM). Measured 2.67 GB from a 256 MiB bomb. Replaced with a bounded page-by-page loop that skips any page whose decompressed content exceeds the budget (detected via LimitReader->Discard, never materialized) and truncates the total at the budget. - MED (reputation): readBloom did make([]uint64, bitsLen) from an untrusted file header with no bound — a crafted/corrupt known-good.bloom with an absurd but internally-consistent m reaches a multi-TB alloc / makeslice panic, bypassing loadSpamResistance's fail-safe (which only downgrades error returns) and crash-loops the daemon. Cap m at 2^31 bits -> fail-safe error. - LOW (engine): session.remove discarded the saveLocked error — a removal that fails to persist silently resurrects the torrent on restart. remove now returns the error; RemoveTorrent logs it. Convergence: r2=2 HIGH, r3-4=0 HIGH, r5=1 HIGH, r6=0 HIGH, r7=1 HIGH. Each HIGH after r2 came from opening a NEW surface (r5 wire-decode, r7 extractors) — the hunt is surface-limited, not depth-limited. See DECISIONS 2026-07-19. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swept the two most-under-covered surfaces (the never-hunted GUI + the sibling ZIP extractors) + testbed. Each fix has a neutralize-to-fail test. - HIGH (gui): the Downloads tab resolved pause/resume/remove by a ROW INDEX into a snapshot list rebuilt every 1s from map-iteration (random) order — a poll between select and action retargeted the index, so Remove dropped a torrent the user never selected. Fixed two ways: capture the selected INFOHASH at click time (selectRow/selectedIH), and sort TorrentSnapshots deterministically so the order is stable across polls. - MED (indexer/extractors, EPUB): the per-chapter budget was output-text-based, so a chapter that decompresses gigabytes but emits no text (a <script> body) never shrank it -> ~1000x total decompression (CPU DoS + goroutine leak). Added a shared total-decompression budget charged by bytes consumed (countingReader). - MED (indexer/extractors, ZIM): the cluster LRU evicted only by count (32) while each cluster is <=64 MiB -> ~2 GiB from a ~2 MB .zim. Added an aggregate byte budget (128 MiB) to the eviction. - MED (testbed): s12's step-3 publish-wait was a `while true` with no timeout; the intended timeout+fail was orphaned in a `while false` block, so a Layer-D publisher regression hung the whole testbed run forever. Moved the budget check into the live loop. Convergence: r2=2 HIGH, r3-4=0, r5=1, r6=0, r7=1, r8=1 — again a NEW surface (GUI). The extractor class keeps yielding (PDF r7, EPUB+ZIM r8) because each extractor has its own decompression-bound gap. See DECISIONS 2026-07-19. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First 0-HIGH round since the GUI surface opened. The r8-regression and html-xml-walker finders returned zero. Three of four fixes are neutralize-to- fail pinned; the LOW #4 fix is structurally verified (see below). - MED (gui): the Status tab ran indexer.Stats() (full-corpus scan under the index lock) synchronously on the Fyne UI thread every 2s, freezing the whole GUI on a real corpus. Compute render() off the UI thread; only SetText on it. - MED (swarmsearch): handlePeerAnnounce (async worker) unconditionally created a p.peers entry, so a frame processed after the synchronous OnPeerClosed resurrected a zombie -> unbounded, remote-triggerable memory leak (no reaper). Update-only now: NotePeerAdded already creates the entry synchronously before any message, so a missing entry means the peer is gone -> skip. - LOW (gui): the Search tab gated rendering on the LIVE checkbox, so toggling Swarm/DHT off mid-search dropped already-fetched hits. Gate on res.Swarm/res.DHT (what was queried; nil when not). - LOW (engine): SetFilePriority on a queued torrent could download outside the max-active-downloads cap (queued torrents were gated only by None priorities). Add the anacrolix-level gate pause uses: DisallowDataDownload on queue, AllowDataDownload on promotion. Structurally verified (the flag is unexported; the full queue/promote/pause suite passes with no regression). Convergence: r2=2 HIGH, r3-4=0, r5=1, r6=0, r7=1, r8=1, r9=0. Strongest signal yet — GUI + follow-ups swept, shared-walker + r8-regression dry. See DECISIONS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…converged The final confirmation round (2-verifier bar, 5 cross-cutting audits) surfaced only ONE bug, found independently by two finders: decodePointerValue (internal/dhtindex/dht.go) decoded an untrusted BEP-46 pointer with unbounded bencode.Unmarshal, so a 16-byte signed value declaring a huge inner string forces a ~128 MiB allocation (measured 134,225,264 bytes) before failing — reached via the companion subscriber's hourly pointer resolution from a followed (untrustable) publisher. Same class as dhtschema/ltepwire/snagg/reputation, in a sibling site the systematic decode audit missed. Fixed with the same MaxStrLen = len(raw) bound + a neutralize-to-fail alloc test. A tree-wide grep of every bencode.Unmarshal/NewDecoder site then confirmed closure: the rest either re-decode our own encoded output (trusted) or decode into map[string]Bytes (lazy raw-slice capture, no amplification — a speculative DecodeDict bound was written, found unnecessary, and reverted to keep the frozen codec minimal). Only decodes into typed string/[]byte fields amplify, and the pointer decoder was the last such site on untrusted remote input. CONVERGENCE: r2=2 HIGH, r3=0, r4=0, r5=1, r6=0, r7=1, r8=1, r9=0, r10=0. Nine adversarial rounds fixed ~28 confirmed bugs; the whole tree (incl. wirecompat/scenarios) is race-clean; the alloc-amplification class is closed at 5 sites; the highest-bar final round found no new confirmed defect. See DECISIONS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Render docs/whitepaper.md as a polished, Bitcoin-whitepaper-styled PDF:
- docs/whitepaper.tex — self-contained LaTeX source (newtx serif, microtype,
bold numbered section headers via titlesec, a narrow run-in abstract, clean
hyperref bookmarks). Compiles with a stock TeX Live; 5 pages, 0 overfull /
0 underfull boxes, no warnings.
- docs/whitepaper.pdf — the compiled deliverable.
- docs/build-whitepaper.sh — reproducible md -> tex -> pdf pipeline (splits the
source, converts the prose with pandoc for faithful escaping, wraps it in the
preamble, and makes the long reference file-paths breakable via \path/xurl).
- docs/Makefile — `make whitepaper` (compile), `make regen` (rebuild .tex from
the .md), `make clean`.
- .gitignore — ignore LaTeX aux artifacts (*.aux, *.fls, *.fdb_latexmk, ...).
A 3-agent adversarial fidelity review (two independent md-vs-PDF diffs + a LaTeX
correctness pass) found the PDF faithfully reproduces the source; its only
finding was a pre-existing grammar typo in the prose ("a index" -> "an index"),
fixed here in whitepaper.md and regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The native GUI could add torrents but not CREATE them — that was CLI-only (`swartznet create`). New "Create torrent" button opens a dialog: - File… / Folder… pickers (or a typeable source path), - an auto-derived, editable output .torrent path (re-derives on source change unless the user edited it), - optional trackers / comment / private (BEP-27), - opt-in "Sign with my identity" (disabled when no identity is loaded), - "Seed the content after creating" — adds the result to the RUNNING engine (AddTorrentBytesSeedFrom) so it appears in the list immediately. Hashing runs off the Fyne UI thread behind a modal progress bar; all widget mutation is inside fyne.Do. Wraps the same engine.CreateTorrentFile / CreateTorrentOptions the CLI uses. An adversarial review of the dialog confirmed the concurrency, signer-pointer, seed-root, and error paths are correct, and caught one real defect that is fixed here: the earlier "Save as…" file picker used dialog.ShowFileSave, which os.Create()s (truncates to 0 bytes) the chosen file the moment it is picked — so selecting an existing file and then cancelling would destroy it. Removed it (Fyne has no non-destructive save-path picker); the output path is auto-derived next to the source and editable. Tests: splitList, deriveTorrentPath, and a nil-daemon no-op guard. GUI package race-clean; both binaries rebuilt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ive parity) Bring the "Create torrent" feature to the web UI (it was native-GUI/CLI only) and make the node's publisher id copyable in BOTH frontends. Create torrent (web): - New POST /torrents/create (httpapi): CreateTorrent* DTOs + Options collaborator, wired in the daemon to engine.CreateTorrentFile with optional identity signing and seed-in-place through the running engine. Because a browser can't pick server-side paths, root/output are typed daemon-side paths; the handler extends its write deadline (http.ResponseController) so a long hash isn't cut off by the 30s API WriteTimeout. - Downloads-tab form: source/output (auto-derived, editable), trackers, comment, private (BEP-27), sign (auto-disabled when no identity), seed. Full parity with the native dialog's options. Copyable publisher id (both UIs): - Web: copyText/copyButton helpers; Copy buttons on Status and Settings→About. - Native: a Copy button in the About dialog's Identity row (Fyne clipboard). An adversarial review (backend/security + web-frontend + parity) found the frontend clean and parity holding; its one MEDIUM is fixed here: a create that SUCCEEDED but whose seed leg failed was mapped to 400 and its infohash discarded. Now the collaborator returns the valid result with a seed_error warning (200), and the web UI surfaces it as a warning. Also added a cheap output-dir pre-flight so a typo'd output path fails fast instead of after hashing gigabytes. Verified: live e2e (create+sign+seed via the API → torrent seeding in /torrents), new unit tests (endpoint 503/400/200/partial-seed; adapter create + fail-closed sign + output pre-check), all JS node --check, httpapi/daemon/gui race-clean, both binaries rebuilt, docs/web-client-design.md + CHANGELOG updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- README: Go 1.22 → 1.24 (badge + build note), add whitepaper badge and a docs link to whitepaper.pdf/.md, fix the build-gui example arg to `dev`. - CHANGELOG: add the v0.9.0 (preview) release section summarizing the rebuild headlines above the Phase-4 slice log. - Bump in-source dev version to v0.9.1-dev (tagged binaries are stamped via -ldflags at release build time). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Polish for the public repo: - CONTRIBUTING.md — build/test workflow, the mainline-compat + UI-parity + MPL ground rules, commit/PR conventions. - SECURITY.md — private vulnerability reporting via GitHub advisories, supported versions, and the not-anonymity non-goal made explicit. - docs/README.md — list the whitepaper (md + PDF) as the "start here" overview under the architecture section. - README: link CONTRIBUTING.md and SECURITY.md from the Contributing note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The two RIBLT-sync reconciliation tests failed under CI's full `-race` sweep with "peers never advertised reconciliation to each other" — not a regression (they pass locally and the sync deadlines below them are a generous 10–40s) but a too-tight pre-condition wait: waitRecon only budgeted 200×5ms = 1s for the peer_announce round-trip, and on a runner whose cores are pinned by the indexer/extractor packages that round-trip can take several seconds. Switch waitRecon to a deadline-based loop (30s ceiling, 10ms poll) matching the other waits in this file. It returns the instant both sides see the capability, so the higher ceiling only costs wall-clock when the machine is overloaded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ot cause) The two RIBLT-sync tests kept failing on CI even after the waitRecon deadline was widened — and they failed AFTER the full 30s wait, which ruled out mere starvation: a 30s localhost round-trip means the announce was *dropped*, not delayed. Root cause is in the test harness, not the product. handlePeerAnnounce drops any announce for a peer it has no entry for (the anti-zombie-leak guard added earlier — must not change), relying on the production invariant that NotePeerAdded creates that entry synchronously before any announce can arrive. connect() skipped NotePeerAdded and let OnRemoteHandshake create the entry, so A's async announceWorker could deliver A's announce to B in the window between the two OnRemoteHandshake calls — before B.peers[A] existed — and B silently dropped it, never learning A's reconciliation cap. The window needs true parallelism + scheduler pressure, so it only bit the loaded -race CI runner (the large-record tests add GC pressure right before connect, widening it) and never reproduced locally. A scratch test with a Gosched between the handshakes reproduced it deterministically (b.recon(A)=false); adding NotePeerAdded before the yield makes it converge every time. Fix: connect() now calls NotePeerAdded on both sides before OnRemoteHandshake, mirroring production (PeerConnAdded → NotePeerAdded precedes the LTEP handshake that triggers any announce). waitRecon's deadline is kept (10s) purely as tolerance for announceWorker scheduling latency, with its comment corrected — it was never a lost-announce workaround. Verified: full package 5× and the two tests 15× green under -race. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.