chore: add Go-based smoke tests for Dewey binary CLI

[yvonnedevlinrh]

Context

Dewey has 6 end-to-end tests in integration_test.go that exercise internal Go APIs (calling newIndexCmd(), vault.New(), store.New() directly). What's missing is binary-level smoke tests that build the actual dewey binary via go build and exercise it through os/exec.Command — the same way a user would interact with it.

This matters because:

• Internal API tests can pass while CLI flag parsing, output formatting, or exit codes are broken
• Changes since v3.2.0 introduced new env vars (DEWEY_CHUNK_MAX_CHARS, DEWEY_SYNTHESIS_ENDPOINT), new dewey init scaffolding (.opencode/dcp.jsonc), and embedding graceful degradation — none of which are tested at the binary level
• The sibling repo unbound-force/gaze uses this exact pattern (TestRunSelfCheck builds gaze and runs it against its own source)

Approach: Go test-based smoke tests

Write a smoke_test.go at the repo root with TestSmoke_* subtests. Each subtest builds the binary once (via sync.Once or TestMain), then exercises CLI commands via exec.Command against t.TempDir() vaults.

Why Go tests over shell scripts or workflow YAML:

Criteria	Go tests	Shell/YAML
Runs locally with same command as CI	✅	❌
Proper assertions with clear failure messages	✅	fragile
Race detection via -race	✅	❌
Auto-cleanup via t.TempDir()	✅	manual
Subtests, parallel execution, skip logic	✅	❌
Self-documenting (test names = specification)	✅	❌

Test Structure

// smoke_test.go — binary-level CLI smoke tests
// Build the dewey binary once, then test each command as a subtest.

var smokeOnce sync.Once
var smokeBinary string

func buildSmokeBinary(t *testing.T) string {
    t.Helper()
    smokeOnce.Do(func() {
        smokeBinary = filepath.Join(t.TempDir(), "dewey")
        // Use a shared temp dir so the binary outlives individual subtests
        cmd := exec.Command("go", "build", "-o", smokeBinary, ".")
        cmd.Stderr = os.Stderr
        if err := cmd.Run(); err != nil {
            // Can't t.Fatal inside sync.Once — panic is acceptable
            panic(fmt.Sprintf("go build failed: %v", err))
        }
    })
    return smokeBinary
}

Subtests to Implement

Phase 1: Core CLI (no Ollama required)

Test	What it exercises	Key assertions
TestSmoke_Version	dewey version	Exit code 0, output contains version string
TestSmoke_Init	dewey init --vault <tmp>	Creates .uf/dewey/config.yaml, sources.yaml, knowledge-stores.yaml; idempotent on re-run
TestSmoke_InitScaffoldsDCP	dewey init --vault <tmp> (with .opencode/ present)	Creates .opencode/dcp.jsonc — new since v3.2.0
TestSmoke_IndexNoEmbeddings	dewey index --no-embeddings	Exit code 0, graph.db created, pages indexed
TestSmoke_Status	dewey status	Text output contains "Dewey Index Status", "Pages:"
TestSmoke_StatusJSON	dewey status --json	Valid JSON, contains pages field
TestSmoke_Search	dewey search "test query"	Exit code 0, output contains matching content
TestSmoke_Reindex	dewey reindex --no-embeddings	Deletes and recreates graph.db, exit code 0
TestSmoke_Doctor	dewey doctor --vault <tmp>	Exit code 0, output contains diagnostic sections
TestSmoke_SourceAdd	dewey source add web --url https://example.com --name test	Source appears in sources.yaml
TestSmoke_Manifest	dewey manifest --vault <tmp>	Creates .uf/dewey/manifest.md
TestSmoke_LintClean	dewey lint --vault <tmp>	Exit code 0 on clean vault

Phase 2: Env var and config validation

Test	What it exercises	Key assertions
TestSmoke_ChunkMaxCharsEnv	DEWEY_CHUNK_MAX_CHARS=500 dewey index --no-embeddings	No crash, respects env var (verify via verbose output)
TestSmoke_OllamaHostFallback	OLLAMA_HOST=0.0.0.0:9999 dewey index --no-embeddings	Graceful degradation, no crash
TestSmoke_VaultEnvVar	OBSIDIAN_VAULT_PATH=<tmp> dewey status	Uses env var instead of --vault flag

Phase 3: Edge cases and error paths

Test	What it exercises	Key assertions
TestSmoke_IndexNoInit	dewey index (no .uf/dewey/)	Non-zero exit code, actionable error message
TestSmoke_ReindexWhileLocked	Hold dewey.lock, then dewey reindex	Non-zero exit code, error mentions stopping serve
TestSmoke_SearchEmptyIndex	dewey search "query" (empty graph.db)	Exit code 0, empty or "no results" output

CI Integration

Add a single step to the existing ci.yml — no new workflow file:

- name: Smoke tests
  run: go test -run TestSmoke -race -count=1 -timeout 5m ./...

This runs alongside the existing go test -race -count=1 ./... step (which already runs all tests). The separate step provides explicit visibility in the CI output. Alternatively, the smoke tests can simply run as part of the existing test step with no changes to CI at all.

Design Decisions

1. Separate file (smoke_test.go), not merged into integration_test.go — existing integration tests use internal APIs (newIndexCmd(), store.New()); smoke tests use exec.Command against the compiled binary. Different test strategies deserve separate files.
2. sync.Once for binary build — build once, share across all subtests. Avoids rebuilding for each test.
3. t.TempDir() per subtest — each test gets an isolated vault directory with automatic cleanup.
4. No Ollama dependency — all tests use --no-embeddings. Semantic search smoke tests (if added later) should use build tags to gate on Ollama availability.
5. Linux-only CI — matches the existing ci.yml matrix. Cross-platform is handled by Go's test portability.
6. Parallel safety — smoke tests can use t.Parallel() since each subtest has its own t.TempDir() and its own dewey.lock. No os.Chdir (unlike integration_test.go which explicitly warns against parallel execution).

Scope

In scope:

• All CLI subcommands that can run without Ollama
• Env var handling (DEWEY_CHUNK_MAX_CHARS, OBSIDIAN_VAULT_PATH, OLLAMA_HOST)
• Exit code verification
• Output format validation (text and JSON)
• Error path behavior (missing init, locked DB)

Out of scope (requires Ollama or external services):

• dewey compile (requires synthesis model)
• dewey curate (requires synthesis model)
• dewey promote (requires learnings in store — already covered by TestEndToEnd_StoreCompileSearch)
• Semantic search verification
• GitHub/web source fetching (requires network)
• MCP server stdio protocol testing

Acceptance Criteria

• smoke_test.go exists at repo root with TestSmoke_* subtests
• go test -run TestSmoke -race -count=1 ./... passes locally without Ollama
• Each subtest is independent (can run individually via -run TestSmoke_Version)
• Tests use exec.Command against the built binary, not internal APIs
• Binary is built once per test run, not per subtest
• All existing tests continue to pass (go test -race -count=1 ./...)
• CI step added (or confirmed that existing test step covers it)

References

• Existing integration tests: integration_test.go (6 tests using internal APIs)
• Gaze precedent: TestRunSelfCheck in unbound-force/gaze
• AGENTS.md testing conventions: standard library testing package, -race -count=1, t.TempDir()