Track initial local Discord maintainer-archive use friction

[joshka]

Context

I am using Discrawl, alongside Gitcrawl, as a local archive layer to widen my scope of vision as an OSS maintainer. The intended workflow is that Codex can query these local archives as needed, then use live services only when freshness, permissions, or write actions require it.

The parallel Gitcrawl tracking issue is openclaw/gitcrawl#81. This issue is the Discrawl side of the same maintainer-archive workflow, focused on real friction observed while using a local Discord archive through Codex.

This issue tracks observations from initial Discrawl use across bot-backed archive sync, local Discord Desktop cache import, private/public snapshot checks, and stop/status diagnostics. The goal is not to draw firm product conclusions from one setup, but to capture concrete friction that showed up while operating Discrawl as local maintainer context.

Shared crawler/cache coordination across multiple crawler tools is intentionally out of scope for this issue. That may be useful elsewhere, but this issue is focused on direct Discrawl maintainer-archive use.

What Worked

• A local SQLite archive was useful for asking Discord-history questions without depending on Discord search.
• discrawl status --json and discrawl doctor were the fastest basic health checks.
• discrawl sync --source wiretap provided a practical no-bot path for local Discord Desktop cache data.
• discrawl wiretap --watch-every could run while manually browsing/scrolling Discord Desktop to improve local coverage.
• discrawl sql was useful for precise read-only inspection when normal query commands were too coarse.
• The existing search, messages, dms, tui, status --json, doctor --json, sync --source, wiretap, analytics, and publish --public-only surfaces covered a lot of the basic workflow once the right command shapes were known.

Observations

• Some useful archive questions quickly became coverage questions: how many messages are present, which channels are named vs synthetic, what is earliest/latest per channel, whether history is complete or just locally cached, and whether wiretap skipped anything.
• During local-only wiretap harvesting, progress needed a separate ad hoc SQLite stats loop while wiretap --watch-every ran. The useful stats were total messages, message channels, channel records, named channels, synthetic channels, top channels, and import deltas.
• Manual scrolling in Discord Desktop could materially improve local coverage, but the tool did not provide a first-class way to monitor that harvesting progress.
• Confirming that Discrawl had actually stopped required shell process checks for discrawl, wiretap, and sqlite3, then a read-only pragma quick_check, then discrawl status --json. status confirmed archive state, but not whether anything was still writing.
• The default discrawl sync source mix can matter. In one workflow, combined bot plus wiretap sync left guild metadata insufficient for public/private publish classification, and a follow-up discrawl sync --source discord repaired the role/visibility metadata.
• Channel-name ambiguity showed up in real use. A helper using the numeric channel id was more reliable than a bare channel name like help.
• Sparse database errors made sync failures harder to interpret. The attachment insert path needed row-level context before deciding whether behavior should change.
• Some useful workflows were already supported but easy to miss from memory alone, especially wiretap --watch-every, read-only discrawl sql, source-specific sync, and the local-only DM boundary.

Related Follow-Up Issues

• Add archive coverage and wiretap harvesting progress reporting so users and agents can see local completeness without custom SQL: Add archive coverage and wiretap progress reporting #101
• Add process, SQLite lock, and integrity diagnostics for checking whether archive work is still running or whether the DB is safe to inspect: Add process and SQLite lock diagnostics #102
• Add source/provenance and publish-scope preflight checks for bot metadata vs wiretap cache data: Add source provenance and publish-scope preflight checks #103
• Improve channel resolution and ambiguity handling for query commands: Improve channel resolution and ambiguity handling #104
• Persist or surface sync/import/write failures with enough row context for retries and issue reports: Surface sync and import failures with row-level context #105
• Document the initial local maintainer-archive workflows that proved useful during real Codex-assisted use: Document initial local maintainer-archive workflows #106

Non-Goals

• This is not proposing one large redesign.
• This is not about shared crawler/cache coordination across multiple crawler tools.
• This is not trying to make local Discord Desktop cache import equivalent to complete Discord history.
• This is not asking Discrawl to use user tokens or behave like a selfbot.
• This is not assuming every archive should publish public snapshots.
• This is not a replacement for narrower follow-up issues with concrete command/API proposals.

---

Prepared with Codex, confirmed as accurate by human.

[clawsweeper]

Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve. Reviewed June 20, 2026, 2:36 PM ET / 18:36 UTC.

Summary
Keep open: current main already has many primitives the report mentions, but the maintainer-archive friction is still split across six narrower open follow-ups and is not fully resolved as a workflow.

Reproducibility: not applicable. as a bug reproduction: this is a parent workflow-friction tracker backed by observed local use and narrower linked feature/docs requests. Source inspection confirms the requested umbrella workflow is not fully implemented on current main.

Root-cause cluster
Relationship: canonical
Canonical: #100
Summary: This issue is the canonical parent tracker for the initial maintainer-archive friction; the linked issues are focused slices of the same observed workflow rather than replacements for the parent context.

Members:

• partial_overlap: Add archive coverage and wiretap progress reporting #101 - Covers the archive coverage and wiretap progress slice of this parent workflow friction.
• partial_overlap: Add process and SQLite lock diagnostics #102 - Covers the process, SQLite lock, and database integrity diagnostic slice.
• partial_overlap: Add source provenance and publish-scope preflight checks #103 - Covers source provenance and publish-scope preflight diagnostics.
• partial_overlap: Improve channel resolution and ambiguity handling #104 - Covers channel resolution and ambiguity handling.
• partial_overlap: Surface sync and import failures with row-level context #105 - Covers durable row-level failure context for sync, import, and write errors.
• 1 more in the report.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Ways to help us reproduce this

• Add a screenshot or short recording showing the behavior.
• Add expected vs actual behavior.

Next step
This should stay as maintainer-triage context while the six narrower linked issues receive priority and scope decisions.

Review details

Best possible solution:

Keep this parent open until maintainers land, decline, or supersede the six linked follow-ups, then close it with links to the final decisions.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug reproduction: this is a parent workflow-friction tracker backed by observed local use and narrower linked feature/docs requests. Source inspection confirms the requested umbrella workflow is not fully implemented on current main.

Is this the best way to solve the issue?

Yes as a tracking shape: keeping this issue as the parent while narrower follow-ups carry implementation is more maintainable than treating it as one broad redesign.

AGENTS.md: not found in the target repository.

Remaining risk / open question:

• This is an umbrella tracker rather than one narrow implementation task, so closing it now would lose the parent context before maintainers decide the six linked follow-ups are complete, declined, or superseded.

Codex review notes: model internal, reasoning high; reviewed against a82cacd73b26.

Label changes

Label justifications:

• P3: This is a low-risk maintainer-workflow friction tracker with concrete follow-ups, not an urgent regression or runtime failure.
• impact:other: The issue affects archive ergonomics and diagnostic confidence for maintainers, which is meaningful but outside the specific owned impact categories.

Evidence reviewed

What I checked:

• Target policy check: No AGENTS.md or VISION.md exists under the target repository root; the parent workspace AGENTS.md was not target-repository policy. (a82cacd73b26)
• Existing CLI primitives: The dispatch table includes sync, wiretap, search, messages, dms, sql, channels, status, publish, and doctor, matching many primitives the issue says worked, but it has no top-level coverage or failures command. (internal/cli/cli.go:204, a82cacd73b26)
• Wiretap watch exists without first-class coverage reporting: wiretap supports --watch-every and repeats imports on a ticker, but the command only prints each import's stats rather than the requested coverage/progress view. (internal/cli/admin_commands.go:349, a82cacd73b26)
• Status and doctor remain narrow: status accepts only --json and doctor accepts only --json, so current main does not expose process, SQLite lock, integrity, or publish-scope diagnostic flags. (internal/cli/admin_commands.go:404, a82cacd73b26)
• Status model lacks coverage/provenance fields: Store status reports aggregate counts and last sync/tail times, but not named versus synthetic channels, per-channel bounds, source provenance, lock ownership, or failure ledger state. (internal/store/store.go:31, a82cacd73b26)
• Channel resolver remains missing: channels currently supports list and show only; there is no resolve subcommand or ambiguity candidate output for repeated agent workflows. (internal/cli/query_commands.go:337, a82cacd73b26)

Likely related people:

• Peter Steinberger: Local blame and commit metadata tie the current CLI/status/doctor surface and v0.11.1 release snapshot to this contributor. (role: recent area contributor; confidence: high; commits: cc1247bfc6b1, a82cacd73b26; files: internal/cli/cli.go, internal/cli/admin_commands.go, internal/cli/control_commands.go)
• vincentkoc: Provided merged PR context and prior review notes connect this contributor to adjacent cloud/publish and wiretap workflow surfaces involved in the parent friction. (role: adjacent feature contributor; confidence: medium; commits: 0a247939e841, 8848c2e6121f, 7ba158b6b405; files: internal/discorddesktop/import.go, docs/guides/wiretap.md, internal/cli/share_commands.go)
• joshka: The issue body records the observed maintainer-archive workflow and confirms the narrower follow-up issues that carry the remaining product work. (role: workflow reporter; confidence: high)

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.