Add archive coverage and wiretap progress reporting

[joshka]

Parent: #100

Related Gitcrawl context: openclaw/gitcrawl#81

Observed Workflow

I am using Discrawl and Gitcrawl as local archives to widen maintainer visibility across OSS work, with Codex querying the archives as needed.

During initial Discrawl use, local-only Discord Desktop cache import was useful, especially where bot access was unavailable. Running discrawl wiretap --watch-every while manually browsing/scrolling Discord Desktop could harvest more messages and channel metadata over time.

The hard part was answering: "what is actually covered?"

Current Workaround

I had to use ad hoc SQLite queries and a separate stats loop to inspect coverage and harvesting progress.

Useful observed stats included:

• total messages
• message-capable channels
• total channel records
• named channels
• synthetic channels
• top channels by message count
• earliest/latest cached messages
• skipped_messages and skipped_channels
• growth between import passes

This worked, but it is not something a user or Codex agent should have to reconstruct manually.

Request

Add first-class archive coverage and wiretap progress reporting.

Example shape:

discrawl coverage --json
discrawl coverage --guild 968932220549103686 --json
discrawl wiretap --watch-every 10s --stats --json

Useful Output

• guild id and name
• message count
• channel count
• named vs synthetic channel count
• earliest and latest message timestamps
• per-channel message counts
• per-channel earliest/latest timestamps
• history-complete marker when known
• wiretap skipped messages/channels
• last bot sync time
• last wiretap import time
• deltas since the previous sample, when running in watch mode

Why This Matters

Coverage is the first decision point before relying on a local archive. It tells a maintainer or Codex agent whether the archive is ready to answer a question locally, whether a channel needs more sync work, or whether local Discord Desktop cache data is only partial.

Acceptance Criteria

• A JSON mode suitable for agent/tool use.
• A human-readable table mode.
• Works across all guilds in the DB by default.
• Can filter to a specific guild.
• Can surface named vs synthetic channel counts without custom SQL.
• Can surface wiretap skipped-message counts without reading raw import logs.

---

Prepared with Codex, confirmed as accurate by human.

[clawsweeper]

Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve. Reviewed June 20, 2026, 10:43 PM ET / 02:43 UTC.

Summary
Keep open: current main and v0.11.1 do not implement first-class archive coverage reporting or wiretap --stats progress/delta output. Existing status, analytics, and wiretap stats are useful primitives, but this adds new CLI/reporting surface that needs maintainer scope confirmation.

Reproducibility: not applicable. this is a feature request, not a bug. Current-main inspection shows no coverage command, and status/wiretap only expose narrower aggregate or per-import stats.

Ways to help us reproduce this

• Add a screenshot or short recording showing the behavior.
• Add expected vs actual behavior.

Next step
The implementation is bounded, but the new CLI/reporting contract needs maintainer scope confirmation before a repair lane should build it.

Review details

Best possible solution:

Define a stable read-only coverage report contract, then implement it against existing store/import primitives without changing Discord auth, user-token, or publish boundaries.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is a feature request, not a bug. Current-main inspection shows no coverage command, and status/wiretap only expose narrower aggregate or per-import stats.

Is this the best way to solve the issue?

Unclear until maintainers confirm the CLI/output contract. A read-only coverage report is the likely narrow implementation path, with watch-mode deltas added carefully to preserve existing wiretap stats output.

AGENTS.md: not found in the target repository.

Remaining risk / open question:

• The permanent JSON schema, table shape, and which wiretap progress counters should be persisted or recomputed are still product/API decisions.

Codex review notes: model internal, reasoning high; reviewed against a82cacd73b26.

Label changes

Label changes:

• add P3: This is a useful but low-risk archive observability and ergonomics feature, not an urgent regression or runtime failure.
• add impact:other: The issue affects maintainer archive readiness visibility, which is meaningful but outside the specific owned impact categories.
• add issue-rating: 🌊 off-meta tidepool: Current issue advisory state selects this label.
• add clawsweeper:no-new-fix-pr: Current issue advisory state selects this label.
• add clawsweeper:needs-maintainer-review: Current issue advisory state selects this label.
• add clawsweeper:needs-product-decision: Current issue advisory state selects this label.

Label justifications:

• P3: This is a useful but low-risk archive observability and ergonomics feature, not an urgent regression or runtime failure.
• impact:other: The issue affects maintainer archive readiness visibility, which is meaningful but outside the specific owned impact categories.

Evidence reviewed

Acceptance criteria:

• go test ./...

What I checked:

• No coverage command on current main: The CLI dispatch table lists the current commands, including wiretap, analytics, channels, and status, but no coverage command or equivalent top-level report surface. (internal/cli/cli.go:204, a82cacd73b26)
• Wiretap watch lacks requested stats flag and deltas: runWiretap supports --watch-every by rerunning import stats on a ticker, but it has no --stats option and no previous-sample delta reporting. (internal/cli/admin_commands.go:349, a82cacd73b26)
• Status model is aggregate-only: store.Status exposes global counts and last sync/tail timestamps, but not named-vs-synthetic channels, per-channel message counts, per-channel bounds, or wiretap skipped counters. (internal/store/store.go:31, a82cacd73b26)
• Wiretap skipped counts exist only as import-run stats: discorddesktop.Stats includes skipped_messages and skipped_channels, which is a useful implementation primitive, but source review did not find a first-class coverage report over persisted archive state. (internal/discorddesktop/import.go:48, a82cacd73b26)
• Latest release still lacks the requested command: The v0.11.1 CLI dispatch contains wiretap and status but no coverage command, so the latest release does not already satisfy the request either. (internal/cli/cli.go:220, cc1247bfc6b1)
• Related tracker remains the parent, not a replacement: GitHub search and the parent issue body show this issue is the narrower coverage/progress follow-up under Track initial local Discord maintainer-archive use friction #100, with no merged PR or safer canonical replacement found.

Likely related people:

• steipete: Recent GitHub commit history and local blame tie this handle to wiretap import, CLI admin/status surfaces, and cache checkpointing. (role: recent area contributor; confidence: high; commits: 055095aadfc2, 40317aa53834, cc1247bfc6b1; files: internal/cli/admin_commands.go, internal/discorddesktop/import.go, internal/cli/analytics.go)
• mvanhorn: History shows this handle introduced analytics quiet, the closest existing report-style surface for channel activity observability. (role: adjacent reporting contributor; confidence: medium; commits: 71b602c5f6c8; files: internal/cli/analytics.go, internal/report/quiet.go)
• vincentkoc: GitHub history connects this handle to adjacent wiretap and control/remote command surfaces that may affect how a new report is exposed. (role: adjacent feature contributor; confidence: medium; commits: 7ba158b6b405, c4be70e52191, dcf0c9c8dbb3; files: internal/discorddesktop/import.go, internal/cli/admin_commands.go, internal/cli/control_commands.go)

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.