chore: document manual smoke testing procedure for Dewey binary

[yvonnedevlinrh]

Summary

Document a repeatable manual testing checklist for verifying Dewey binary functionality across releases. This covers core CLI commands, configuration, and edge cases — especially changes introduced since v3.2.0.

Motivation

There's no documented procedure for manually verifying the compiled binary works end-to-end. Unit tests cover package-level behavior, but binary-level regressions (flag parsing, config loading, init scaffolding, graceful degradation) can slip through.

Manual Testing Checklist

Prerequisites

go build -o /tmp/dewey .
mkdir -p /tmp/dewey-test-vault/notes

Create test fixtures:

cat > /tmp/dewey-test-vault/notes/architecture.md << 'EOF'
# Architecture
The system uses a layered approach with clean separation of concerns.
## Backend Interface
All tools program against `backend.Backend`, not concrete implementations.
## Storage Layer
SQLite for persistent indexes via `modernc.org/sqlite`.
#decision #architecture
EOF

cat > /tmp/dewey-test-vault/notes/authentication.md << 'EOF'
# Authentication
OAuth2 with PKCE flow for all user-facing clients.
## Token Refresh
Tokens refresh silently via background goroutine.
#pattern #security
EOF

cat > /tmp/dewey-test-vault/notes/daily-standup.md << 'EOF'
# Daily Standup 2026-08-20
- Finished OAuth2 integration
- Started migration to new embedding model
#meeting
EOF

Phase 1: Init & Configuration

• dewey init --vault /tmp/dewey-test-vault — creates .uf/dewey/config.yaml, sources.yaml, knowledge-stores.yaml, .gitignore entries
• dewey init --vault /tmp/dewey-test-vault (second run) — idempotent, no error, no overwrite
• With .opencode/ present: dewey init scaffolds .opencode/dcp.jsonc and .opencode/commands/ (new since v3.2.0)
• config.yaml defaults: provider: ollama, model: granite-embedding:30m, endpoint: http://localhost:11434

Phase 2: Indexing

Configure a disk source first:

# .uf/dewey/sources.yaml
sources:
  - id: disk-notes
    type: disk
    name: test-notes
    config:
      path: "./notes"
      recursive: true
    refresh_interval: hourly

• dewey index --vault /tmp/dewey-test-vault — indexes 3 markdown files, creates graph.db, generates embeddings
• dewey index --vault /tmp/dewey-test-vault (second run) — incremental, completes faster
• dewey index --no-embeddings --vault /tmp/dewey-test-vault — skips Ollama entirely
• dewey index --source disk-notes --vault /tmp/dewey-test-vault — processes only the named source

Phase 3: Status & Diagnostics

• dewey status --vault /tmp/dewey-test-vault — shows page/block/embedding counts and source status
• dewey status --json --vault /tmp/dewey-test-vault — valid JSON output
• dewey doctor --vault /tmp/dewey-test-vault — emoji-styled pass/warn/fail with DCP check (new since v3.2.0)
• DEWEY_EMBEDDING_ENDPOINT=http://localhost:99999 dewey doctor --vault /tmp/dewey-test-vault — embedding warning, no crash

Phase 4: Search

• dewey search --vault /tmp/dewey-test-vault "OAuth2" — returns result from authentication.md
• dewey search --vault /tmp/dewey-test-vault --limit 1 "architecture" — respects limit
• dewey search --vault /tmp/dewey-test-vault "xyzzy_nonexistent" — exit 0, no results

Phase 5: Reindex

• dewey reindex --vault /tmp/dewey-test-vault — removes and rebuilds graph.db
• dewey reindex --no-embeddings --vault /tmp/dewey-test-vault — rebuilds without embeddings

Phase 6: Embedding Configuration (new since v3.2.0)

• DEWEY_CHUNK_MAX_CHARS=4096 dewey index -v --vault /tmp/dewey-test-vault — chunk size override visible in logs
• DEWEY_CHUNK_MAX_CHARS=notanumber dewey index -v --vault /tmp/dewey-test-vault — warning logged, falls back to 12288
• OLLAMA_HOST=localhost:11434 dewey index --vault /tmp/dewey-test-vault — works as fallback
• OLLAMA_HOST=0.0.0.0:11434 dewey index -v --vault /tmp/dewey-test-vault — http:// prepended automatically
• DEWEY_EMBEDDING_MODEL=nonexistent:latest dewey index --vault /tmp/dewey-test-vault — warns, continues in keyword-only mode

Phase 7: MCP Server

• echo '{"jsonrpc":"2.0","method":"initialize","id":1,"params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' | dewey serve --vault /tmp/dewey-test-vault --no-embeddings — JSON-RPC response returned
• dewey serve --vault /tmp/dewey-test-vault --http :18080 --no-embeddings & — starts HTTP transport, clean shutdown on kill
• dewey serve --vault /tmp/dewey-test-vault --read-only --no-embeddings — starts in read-only mode

Phase 8: Manifest

• dewey manifest --vault /tmp/dewey-test-vault — creates .uf/dewey/manifest.md with auto-generated header

Phase 9: Knowledge Lifecycle

• dewey lint --vault /tmp/dewey-test-vault — runs without crash, reports quality issues
• dewey lint --fix --vault /tmp/dewey-test-vault — auto-fixes mechanical issues
• dewey compile --vault /tmp/dewey-test-vault — without synthesis model: returns prompts or "no learnings" (no crash)
• dewey curate --vault /tmp/dewey-test-vault — without synthesis model: returns actionable error (hard error, unlike compile)

Phase 10: Synthesis Endpoint (new since v3.2.0)

• DEWEY_SYNTHESIS_ENDPOINT=http://localhost:11434 dewey compile -v --vault /tmp/dewey-test-vault — env var respected in logs

Phase 11: Source Management

• dewey source add github --org testorg --repos testrepo --vault /tmp/dewey-test-vault — adds to sources.yaml
• dewey source add web --url https://example.com --name test-site --vault /tmp/dewey-test-vault — adds to sources.yaml
• dewey source add web --url https://example.com --name test-site --vault /tmp/dewey-test-vault (duplicate) — rejected with error

Phase 12: Edge Cases & Flags

• dewey status --vault /tmp/nonexistent — clear error, no panic
• Empty vault (no markdown): init + index + status all succeed with 0 counts
• dewey version — outputs version string, exit 0
• dewey status --vault /tmp/dewey-test-vault --verbose — additional debug output
• dewey status --vault /tmp/dewey-test-vault --log-file /tmp/dewey-test.log — log file created with entries

Cleanup

rm -rf /tmp/dewey-test-vault /tmp/empty-vault /tmp/dewey-test.log

Changes Since v3.2.0 Coverage Map

Change	Test Phase
Batch embedding / IndexDocuments	2
Configurable chunk limit (DEWEY_CHUNK_MAX_CHARS)	6
DCP scaffold in dewey init + doctor check	1, 3
DEWEY_SYNTHESIS_ENDPOINT env var	10
OLLAMA_HOST fallback + graceful degradation	6
Array form for single-text Ollama embed	2 (implicit)
Compile truncation fix	9
Init slash command path fix	1

Notes

• All tests work on both Linux and macOS (only exception: timeout utility in Phase 7 needs coreutils on macOS)
• Phases 6 and 10 require Ollama running locally
• Vertex AI-specific tests (global region endpoint, curate reliability) require GCP credentials and are out of scope for this checklist