Skip to content

Add evidence-bound Investigation Mode - #245

Merged
HomenShum merged 4 commits into
mainfrom
codex/investigation-mode-2026-07-28
Jul 29, 2026
Merged

Add evidence-bound Investigation Mode#245
HomenShum merged 4 commits into
mainfrom
codex/investigation-mode-2026-07-28

Conversation

@HomenShum

@HomenShum HomenShum commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add an evidence-bound Investigation Mode: versioned dataset, validated task DAG, bounded research pack, teaching case, runtime truth states, and durable launch/result contracts.
  • Close the research honesty loop so needs_review is never presented, counted, routed, or colored as complete.
  • Bind managed writes to exact source receipts and persisted terminal outcomes; final agent summaries re-read committed status cells before claiming sealed evidence.
  • Apply a Mobbin-derived, citation-backed design DNA to the investigation run details, selected-task detail, evidence grid, and source-detail surfaces without copying reference pixels.

Root causes closed

  1. The client investigation contract stopped before durable admission and terminal persistence.
  2. Managed research writes could downgrade payload status while leaving a status-cell scalar or downstream summary looking complete.
  3. Message receipts selected only completed rows, so a review-required run could bind to a stale company.
  4. Batch routing treated any non-pending row as finished and could claim an all-review sheet was complete.
  5. Decision progress used success-green styling even when review remained.

The fix moves truth into admission, mutation validation, persisted result reads, shared routing helpers, and visible outcome receipts.

Trust and reliability invariants

  • Launch receipts bind actor consent, plan digest, dataset/version/content hash, artifact/version, and idempotency identity.
  • Sparse scope fails closed above 8,192 writes; evidence, response bodies, URL lengths, registry size/TTL, redirects, chunks, time, and review concurrency are bounded.
  • Source fetches are HTTPS-only, block credentials/private/reserved/metadata addresses, validate every DNS answer and redirect hop, pin the validated connection, cap streamed bytes, and return honest non-2xx failures.
  • Only exact-byte network fetches can mint source receipts; synthetic fixtures, changed snippets, stale receipts, mixed fabricated evidence, and unsealed writes remain needs_review.
  • Convex enforces sealed_cell_payload_v1; the UI and planner consume persisted outcomes rather than requested write arguments.
  • Review-required rows remain inspectable and explicitly rerunnable; only a sheet whose every persisted row is complete receives the all-complete summary.
  • No alternate canonical graph or dual-write path is introduced.

Mobbin reference provenance

Derived atomic layout/interaction facts are recorded in docs/design-dna/; no screenshots or copied pixels are stored.

Reference validator: 6 observations, 58 atomic facts, 7 rules, 1 score receipt, 14 resolved citations, 0 findings.

Verification — exact head 4be78b6ed8591ee8327483045a18cccb5283221c

  • npm audit --omit=dev --audit-level=moderate: 0 vulnerabilities
  • Pre-build and dist security gates
  • Root and Convex TypeScript checks
  • Design audit, UI layer audit, UI contract, QA matrix, content fluency, proof freshness, fresh-room proofs, and SLO gate
  • Full Vitest suite: 380 files, 2,666 scenarios
  • Complete production-memory Playwright gate
  • Production build: 7,518 modules
  • Vercel deploy-input dry-run: provenance verifier included; unrelated scripts excluded
  • CSP source/dist integrity: executable inline scripts require an exact SHA-256 allowlist; structured-data blocks are excluded
  • CSP/deploy packaging scenarios: 23/23
  • Build provenance: index.html and ai-elements-check.html both carry the exact head SHA
  • Responsive Investigation matrix: 1920, 1280, 1024, 768, and 390 px
  • Adversarial trust, UI-truth, Mobbin-provenance, and final receipt-routing reviews: APPROVED
  • git diff --check

Evidence

Operational boundary

  • This PR does not run an out-of-band Convex deploy. Project policy requires coordinated reviewed deployment of shared schema/functions.
  • The browser can run the deterministic memory Investigation surface; durable Convex launch code is merged only after CI/preview proof and remains subject to the repository's normal deployment pipeline.
  • External human-study, signed-review, and production-backend attestations are not fabricated or marked complete here.

@vercel

vercel Bot commented Jul 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
noderoom Ready Ready Preview, Comment Jul 29, 2026 10:39am

Request Review

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

Scaffold Handoff — For Your Coding Agent

Your coding agent (Codex, Claude Code, etc.) should apply the accepted
scaffold proposals below. Do NOT touch any immutable files.

Immutability Check

Mode: advisory

✅ No immutable files were modified in this branch.

Changed Files

  • .vercelignore
  • convex/agentJobRunner.ts
  • convex/agentJobs.ts
  • convex/artifacts.ts
  • convex/convexRoomTools.ts
  • convex/schema.ts
  • design-dna/observations/obs-airtable-data-grid.yaml
  • design-dna/observations/obs-coinbase-asset-detail.yaml
  • design-dna/observations/obs-stackai-run-details.yaml
  • design-dna/observations/obs-stackai-selected-node.yaml
  • design-dna/rules.yaml
  • design-dna/score-receipts/score-investigation-mode-2026-07-28.json
  • docs/INVESTIGATION_MODE.md
  • docs/eval/OFFICIAL_BENCHMARK_READINESS.md
  • docs/eval/OFFICIAL_BENCHMARK_TASK_COVERAGE.md
  • docs/eval/OPENROUTER_CONVEX_BENCHMARK.md
  • docs/eval/agent-improvement-loop.md
  • docs/eval/agent-improvement-loop.svg
  • docs/eval/agent-improvement-loop/20260729T103829Z.json
  • docs/eval/agent-improvement-loop/latest.json
  • docs/eval/agent-workspace-sandbox-smoke.json
  • docs/eval/algorithm-artifact-smoke.json
  • docs/eval/bankertoolbench-official-contract.json
  • docs/eval/docker-sandbox-probe.json
  • docs/eval/eval-runs.jsonl
  • docs/eval/halo-convex-context-telemetry.json
  • docs/eval/halo-self-improvement-smoke.json
  • docs/eval/halo-variant-selection.json
  • docs/eval/official-benchmark-readiness.json
  • docs/eval/official-benchmark-task-coverage.json
  • docs/eval/openrouter-convex-benchmark.json
  • docs/eval/professional-catalog-proofs.json
  • docs/eval/professional-proof-ledger.json
  • docs/eval/spreadsheetbench-chart-visual-probe.json
  • docs/eval/traces/credit/20260729T103838316Z-56535f6a_dirty.33d483e88de037f4/cascade-healthy.json
  • docs/eval/traces/credit/20260729T103838316Z-56535f6a_dirty.33d483e88de037f4/delta-incomplete.json
  • docs/eval/traces/credit/20260729T103838316Z-56535f6a_dirty.33d483e88de037f4/mapping-correct.json
  • docs/eval/traces/credit/20260729T103838316Z-56535f6a_dirty.33d483e88de037f4/mapping-misbind.json
  • docs/eval/traces/credit/20260729T103838316Z-56535f6a_dirty.33d483e88de037f4/summit-stressed.json
  • docs/eval/traces/ladder/20260729T103837868Z-56535f6a_dirty.b2eb9351eb558e24/ladder_L1_read_scripted.json
  • docs/eval/traces/ladder/20260729T103837868Z-56535f6a_dirty.b2eb9351eb558e24/ladder_L2_edit_scripted.json
  • docs/eval/traces/ladder/20260729T103837868Z-56535f6a_dirty.b2eb9351eb558e24/ladder_L3_conflict_scripted.json
  • docs/eval/traces/ladder/20260729T103837868Z-56535f6a_dirty.b2eb9351eb558e24/ladder_L4_blocked_scripted.json
  • docs/eval/traces/ladder/20260729T103837868Z-56535f6a_dirty.b2eb9351eb558e24/ladder_L5_large_range_scripted.json
  • docs/eval/traces/ladder/20260729T103837868Z-56535f6a_dirty.b2eb9351eb558e24/ladder_L6_long_horizon_scripted.json
  • docs/eval/traces/ladder/20260729T103837868Z-56535f6a_dirty.b2eb9351eb558e24/ladder_L7_resume_scripted.json
  • e2e/decision-assistant.spec.ts
  • e2e/fixtures.ts
  • e2e/full-modern-ux-bar.spec.ts
  • e2e/investigation-responsive.spec.ts
  • evidence/investigation-mode-release-blockers/after.txt
  • evidence/investigation-mode-release-blockers/before.txt
  • evidence/investigation-mode-vertical-slice/after-build.txt
  • evidence/investigation-mode-vertical-slice/after-contract-search.txt
  • evidence/investigation-mode-vertical-slice/after-design-audit.txt
  • evidence/investigation-mode-vertical-slice/after-floor.txt
  • evidence/investigation-mode-vertical-slice/after-focused-tests.txt
  • evidence/investigation-mode-vertical-slice/after-nodeagent-frame-smoke.txt
  • evidence/investigation-mode-vertical-slice/after-omnigent-nodeagent-smoke.txt
  • evidence/investigation-mode-vertical-slice/after-research-pack.json
  • evidence/investigation-mode-vertical-slice/after-room-complete.png
  • evidence/investigation-mode-vertical-slice/after-room.png
  • evidence/investigation-mode-vertical-slice/after-teaching-case.png
  • evidence/investigation-mode-vertical-slice/after-ui.txt
  • evidence/investigation-mode-vertical-slice/after-vite-build-final.txt
  • evidence/investigation-mode-vertical-slice/before-contract-search.txt
  • evidence/investigation-mode-vertical-slice/before-floor.txt
  • evidence/investigation-mode-vertical-slice/before-nodeagent-frame-smoke.txt
  • evidence/investigation-mode-vertical-slice/before-omnigent-nodeagent-smoke.txt
  • evidence/investigation-mode-vertical-slice/before-room.png
  • evidence/investigation-mode-vertical-slice/before-ui.txt
  • evidence/investigation-mode-vertical-slice/before-vite-build.txt
  • evidence/investigation-mode-vertical-slice/comparison.md
  • evidence/investigation-mode-vertical-slice/mobbin-reference-trace.md
  • evidence/investigation-mode-vertical-slice/observable.txt
  • evidence/investigation-mode-vertical-slice/reference-render-receipt.json
  • package.json
  • scripts/design-system.ts
  • scripts/lib/cspIntegrity.ts
  • scripts/reference-provenance.ts
  • scripts/security-gate.ts
  • scripts/verify-build-provenance.ts
  • src/app/researchRouting.ts
  • src/app/store.tsx
  • src/app/styles.css
  • src/engine/types.ts
  • src/nodeagent/core/evidenceReceipt.ts
  • src/nodeagent/core/plans.ts
  • src/nodeagent/core/reasoningFrames.ts
  • src/nodeagent/core/types.ts
  • src/nodeagent/core/worldModel.ts
  • src/nodeagent/index.ts
  • src/nodeagent/investigation/contracts.ts
  • src/nodeagent/investigation/index.ts
  • src/nodeagent/investigation/workspace.ts
  • src/nodeagent/skills/integration/noderoomAdapter.ts
  • src/nodeagent/skills/search/captureSourceFirecrawlTool.ts
  • src/nodeagent/skills/search/fetchSource.ts
  • src/nodeagent/skills/spreadsheet/cellMutator.ts
  • src/ui/Chat.tsx
  • src/ui/investigation/InvestigationReport.tsx
  • src/ui/investigation/investigation.css
  • src/ui/panels/Artifact.tsx
  • tests/agentJobsRuntime.test.ts
  • tests/artifactsPseudoTabPersistence.test.tsx
  • tests/buildProvenance.test.ts
  • tests/convexFetchSourcePolicy.test.ts
  • tests/cspIntegrity.test.ts
  • tests/fetchSourceHardeningScenarios.test.ts
  • tests/fetchSourceNetworkGuard.test.ts
  • tests/fetchSourceSsrf.test.ts
  • tests/investigationMode.test.ts
  • tests/investigationSurface.test.tsx
  • tests/managedLockTools.test.ts
  • tests/referenceProvenance.test.ts
  • tests/researchHarness.test.ts
  • tests/researchRouting.test.ts
  • tests/trustedSourceReceiptRegistry.test.ts
  • tests/workflowEvals.test.ts
  • vercel.json
  • vite.config.ts

Needs Adversarial Review — Do NOT Apply Yet

These proposals passed the reject check but have not been approved by
an adversarial reviewer. A human or frozen LLM judge must approve them first.

  • scaf-001 (AGENTS.md): Add explicit instruction for step spreadsheetbench-runner-fixture: Step spreadsheetbench-runner-fixture failed — scaffold may need explicit instruction or evidence assertion.
  • scaf-002 (AGENTS.md): Add explicit instruction for step convex-boundaries: Step convex-boundaries failed — scaffold may need explicit instruction or evidence assertion.

Safety Boundary

Agent may improve the scaffold.
Agent may NOT weaken the proof gate.

Immutable files (never modify):

  • scripts/proofloop.mjs
  • scripts/agent-improvement-loop.ts
  • tests/harnessChangeEval.test.ts
  • .github/workflows/
  • src/eval/evalTrustPolicy.ts
  • src/eval/architectureBudget.ts
  • evals/evalStore.ts

Scaffold files (safe to modify):

  • AGENTS.md
  • CLAUDE.md
  • proofloop/scenarios/*.yaml
  • proofloop/rubrics/*.yaml
  • proofloop/subagents/*.md
  • proofloop/adapters/*.js
  • .proofloop/memory.jsonl
  • src/nodeagent/models/prompts/systemPrompt.ts

@HomenShum
HomenShum force-pushed the codex/investigation-mode-2026-07-28 branch from 432e94a to 4be78b6 Compare July 29, 2026 10:37
@HomenShum
HomenShum merged commit 9c99889 into main Jul 29, 2026
13 checks passed
@HomenShum
HomenShum deleted the codex/investigation-mode-2026-07-28 branch July 29, 2026 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant