From 96eae43ddd4c2bd59f52b3fa76de6c9cc020b20d Mon Sep 17 00:00:00 2001 From: stark Date: Mon, 6 Jul 2026 06:39:12 +0900 Subject: [PATCH] docs: split Codex Claude integration evidence --- ...E_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md | 34 +++++++++++++++++++ docs/NEXT_BACKLOG.md | 6 ++++ .../2026-07-05-promptlane-95-quality-plan.md | 8 +++++ package.json | 1 + scripts/quality-95-evidence.mjs | 22 ++++++++++++ src/cli/commands/quality-evidence.test.ts | 8 +++++ src/packaging/plugin-files.test.ts | 11 ++++++ src/packaging/quality-evidence-script.test.ts | 1 + tasks/todo.md | 26 ++++++++++++++ 9 files changed, 117 insertions(+) create mode 100644 docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md diff --git a/docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md b/docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md new file mode 100644 index 00000000..d547a95d --- /dev/null +++ b/docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md @@ -0,0 +1,34 @@ +# PromptLane Codex And Claude Local Integration Evidence 2026-07-06 + +This document records current non-operator Codex and Claude Code integration +evidence for the PromptLane 9.5 quality bar. + +## Repeatable Local Evidence + +| Evidence | Result | Details | +| --- | --- | --- | +| `corepack pnpm smoke:agent-setup` | PASS | Exercises setup dry-run, setup with MCP registration, Claude Code doctor, and Codex doctor with isolated HOME/data directories and fake provider binaries. | +| `corepack pnpm smoke:hooks` | PASS | Verifies `prompt-coach` hook status, Claude Code fail-open behavior, Codex fail-open behavior, and `loopdeck` compatibility hook status. | +| `corepack pnpm smoke:mcp-coach-loop` | PASS | Exercises score, coach, improve, clarification apply/record, `score_prompt` effectiveness evidence, and `coach_prompt` effectiveness guidance through the real stdio MCP server. | +| `corepack pnpm smoke:mcp-elicitation` | PASS | Verifies MCP `elicitation/create` answer collection, final prompt composition, and local-only privacy boundaries. | +| `corepack pnpm smoke:mcp-native-dialog` | PASS | Verifies the no-dialog fallback path with `allow_native_dialog: false`, `interaction_status: "unsupported"`, and no OS dialog. | +| `corepack pnpm dogfood:first-coach-loop` | PASS | Captures a Codex prompt through the hook, then runs coach, loop collect, and loop brief against an isolated local archive. | +| `corepack pnpm dogfood:loop-memory-approval` | PASS | Captures a Codex prompt, records a loop outcome through MCP, proposes and records approved memory, and proposes an instruction patch without writing instruction files. | + +## Boundary + +This evidence does not complete the native OS dialog dogfood blocker. + +`PROMPT_COACH_NATIVE_DIALOG_APPROVED=1 corepack pnpm +dogfood:mcp-native-dialog-approved` still requires an operator who is present +and expects a native dialog. The approved command refuses to run without the +approval environment variable and must not be added to automated CI or scheduled +checks. + +## Interpretation + +The Codex and Claude Code integration axis has current local setup, hook, MCP, +elicitation, no-dialog fallback, first-loop, and loop-memory dogfood evidence. +`quality-evidence` records this proven non-operator portion as +`codex_claude_local_integration_evidence` and keeps +`native_dialog_approved_dogfood` in remaining evidence. diff --git a/docs/NEXT_BACKLOG.md b/docs/NEXT_BACKLOG.md index 398950f3..21739afc 100644 --- a/docs/NEXT_BACKLOG.md +++ b/docs/NEXT_BACKLOG.md @@ -221,6 +221,12 @@ Decision: `dogfood:web-user-flow` browser evidence. `quality-evidence` records this as `manual_ui_patrol_artifact_evidence`, but `scheduled_ui_patrol` remains pending until a real cron `schedule` event exists. + `docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md` records current + non-operator setup, hook, MCP, elicitation, no-dialog fallback, first-loop, + and loop-memory evidence. `quality-evidence` records this as + `codex_claude_local_integration_evidence`, while + `native_dialog_approved_dogfood` remains pending until explicit operator + approval exists. - PR #478 moved that quality evidence gate onto the installed product CLI. Main CI run `28753458359` passed Node 22 and Node 24 after merge, so `prompt-coach quality-evidence --require-complete` is now a current default-branch release diff --git a/docs/superpowers/plans/2026-07-05-promptlane-95-quality-plan.md b/docs/superpowers/plans/2026-07-05-promptlane-95-quality-plan.md index 33f39db4..98ef22f5 100644 --- a/docs/superpowers/plans/2026-07-05-promptlane-95-quality-plan.md +++ b/docs/superpowers/plans/2026-07-05-promptlane-95-quality-plan.md @@ -222,6 +222,14 @@ `corepack pnpm ui-patrol`, and `dogfood:web-user-flow`. `quality-evidence` records this as `manual_ui_patrol_artifact_evidence` while keeping `scheduled_ui_patrol` pending until a real cron `schedule` event exists. +- `docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md` records current + non-operator Codex and Claude Code evidence: setup/doctor smoke, hook smoke, + MCP coach-loop smoke, MCP elicitation smoke, no-dialog native fallback + preflight, first-loop dogfood, and loop-memory approval dogfood. + `quality-evidence` records this as + `codex_claude_local_integration_evidence` while keeping + `native_dialog_approved_dogfood` pending until explicit operator approval + exists. - PR #478 proved that installed CLI path on the default branch; main CI run `28753458359` passed Node 22 and Node 24 after merge, so future agents can use the product CLI itself to decide whether 9.5 is still blocked before claiming diff --git a/package.json b/package.json index 12eac1a8..5e4fba6e 100644 --- a/package.json +++ b/package.json @@ -67,6 +67,7 @@ "docs/LOCAL_95_EVIDENCE_2026-07-06.md", "docs/PRODUCT_POSITIONING_EVIDENCE_2026-07-06.md", "docs/UI_PATROL_EVIDENCE_2026-07-06.md", + "docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md", "docs/RELEASE_CHECKLIST.md", "plugins", "integrations", diff --git a/scripts/quality-95-evidence.mjs b/scripts/quality-95-evidence.mjs index 22b4a3c0..80c8df5b 100644 --- a/scripts/quality-95-evidence.mjs +++ b/scripts/quality-95-evidence.mjs @@ -172,6 +172,7 @@ function readCompletedEvidence() { let claudePlugin = ""; let claudeMarketplace = ""; let uiPatrolEvidence = ""; + let codexClaudeEvidence = ""; try { plan = readFileSync(planPath, "utf8"); localEvidence = readFileSync("docs/LOCAL_95_EVIDENCE_2026-07-06.md", "utf8"); @@ -192,10 +193,15 @@ function readCompletedEvidence() { "docs/UI_PATROL_EVIDENCE_2026-07-06.md", "utf8", ); + codexClaudeEvidence = readFileSync( + "docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md", + "utf8", + ); } catch { return { product_positioning_metadata_alignment: false, manual_ui_patrol_artifact_evidence: false, + codex_claude_local_integration_evidence: false, web_user_flow_current_main_evidence: false, privacy_raw_free_regression_sweep: false, codex_claude_setup_smoke_refresh: false, @@ -228,6 +234,16 @@ function readCompletedEvidence() { uiPatrolEvidence.includes("Local `corepack pnpm ui-patrol`") && uiPatrolEvidence.includes("pending_no_schedule_run") && uiPatrolEvidence.includes("manual_ui_patrol_artifact_evidence"), + codex_claude_local_integration_evidence: + codexClaudeEvidence.includes("corepack pnpm smoke:agent-setup") && + codexClaudeEvidence.includes("corepack pnpm smoke:hooks") && + codexClaudeEvidence.includes("corepack pnpm smoke:mcp-coach-loop") && + codexClaudeEvidence.includes("corepack pnpm smoke:mcp-elicitation") && + codexClaudeEvidence.includes("corepack pnpm smoke:mcp-native-dialog") && + codexClaudeEvidence.includes("corepack pnpm dogfood:first-coach-loop") && + codexClaudeEvidence.includes("corepack pnpm dogfood:loop-memory-approval") && + codexClaudeEvidence.includes("native_dialog_approved_dogfood") && + codexClaudeEvidence.includes("codex_claude_local_integration_evidence"), web_user_flow_current_main_evidence: plan.includes("web_user_flow_current_main_evidence") && plan.includes("corepack pnpm dogfood:web-user-flow") && @@ -298,6 +314,12 @@ function axisEvidenceCoverage({ ) { satisfied.push("codex_claude_setup_smoke_refresh"); } + if ( + axis.id === "codex_and_claude_code_integration" && + completedEvidence.codex_claude_local_integration_evidence + ) { + satisfied.push("codex_claude_local_integration_evidence"); + } if ( [ "local_first_privacy_boundary", diff --git a/src/cli/commands/quality-evidence.test.ts b/src/cli/commands/quality-evidence.test.ts index 831e8855..d01fd3d9 100644 --- a/src/cli/commands/quality-evidence.test.ts +++ b/src/cli/commands/quality-evidence.test.ts @@ -95,6 +95,11 @@ describe("quality-evidence CLI command", () => { expect.objectContaining({ id: "codex_and_claude_code_integration", status: "blocked_external", + satisfied_evidence: expect.arrayContaining([ + "codex_claude_setup_smoke_refresh", + "codex_claude_local_integration_evidence", + "local_95_evidence_sweep", + ]), remaining_evidence: expect.arrayContaining([ "native_dialog_approved_dogfood", ]), @@ -169,6 +174,9 @@ describe("quality-evidence CLI command", () => { expect(text).toContain( "codex_and_claude_code_integration: blocked_external", ); + expect(text).toContain( + "satisfied=codex_claude_setup_smoke_refresh,codex_claude_local_integration_evidence,local_95_evidence_sweep", + ); expect(text).toContain("remaining=native_dialog_approved_dogfood"); expect(text).toContain("Scorecard review candidates"); expect(text).toContain("- none"); diff --git a/src/packaging/plugin-files.test.ts b/src/packaging/plugin-files.test.ts index fdb07663..b226617b 100644 --- a/src/packaging/plugin-files.test.ts +++ b/src/packaging/plugin-files.test.ts @@ -948,6 +948,9 @@ describe("plugin packaging files", () => { expect(packageJson.files).toContain( "docs/UI_PATROL_EVIDENCE_2026-07-06.md", ); + expect(packageJson.files).toContain( + "docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md", + ); expect(packageJson.scripts["evidence:quality"]).toBe( "node scripts/quality-95-evidence.mjs", ); @@ -972,6 +975,9 @@ describe("plugin packaging files", () => { "docs/PRODUCT_POSITIONING_EVIDENCE_2026-07-06.md", ); expect(content).toContain("docs/UI_PATROL_EVIDENCE_2026-07-06.md"); + expect(content).toContain( + "docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md", + ); expect(content).toContain("PR #478"); expect(content).toContain("28753458359"); expect(content).toContain("corepack pnpm evidence:quality"); @@ -985,6 +991,7 @@ describe("plugin packaging files", () => { expect(content).toContain("blocked_by_external_event"); expect(content).toContain("product_positioning_metadata_alignment"); expect(content).toContain("manual_ui_patrol_artifact_evidence"); + expect(content).toContain("codex_claude_local_integration_evidence"); expect(content).toContain("local-first privacy boundary"); expect(content).toContain("setup/doctor/MCP smoke"); expect(content).toContain("loop memory"); @@ -1047,6 +1054,10 @@ describe("plugin packaging files", () => { expect(evidenceScript).toContain("PRODUCT_POSITIONING_EVIDENCE_2026-07-06"); expect(evidenceScript).toContain("manual_ui_patrol_artifact_evidence"); expect(evidenceScript).toContain("UI_PATROL_EVIDENCE_2026-07-06"); + expect(evidenceScript).toContain("codex_claude_local_integration_evidence"); + expect(evidenceScript).toContain( + "CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06", + ); expect(evidenceScript).toContain("web_user_flow_current_main_evidence"); expect(evidenceScript).toContain("browser e2e passed"); expect(evidenceScript).toContain("privacy_raw_free_regression_sweep"); diff --git a/src/packaging/quality-evidence-script.test.ts b/src/packaging/quality-evidence-script.test.ts index a01f930c..67d94975 100644 --- a/src/packaging/quality-evidence-script.test.ts +++ b/src/packaging/quality-evidence-script.test.ts @@ -198,6 +198,7 @@ describe("quality 9.5 evidence script", () => { status: "blocked_external", satisfied_evidence: expect.arrayContaining([ "codex_claude_setup_smoke_refresh", + "codex_claude_local_integration_evidence", "local_95_evidence_sweep", ]), remaining_evidence: expect.arrayContaining([ diff --git a/tasks/todo.md b/tasks/todo.md index 16f1bc6d..7aa29b9a 100644 --- a/tasks/todo.md +++ b/tasks/todo.md @@ -1,5 +1,31 @@ # 작업 계획 +## 2026-07-06 PromptLane Codex Claude Evidence Split + +- [x] CHECK: Codex/Claude integration still has an external native-dialog + approved dogfood blocker, but local setup, hooks, MCP, elicitation, first + loop, and loop-memory dogfood evidence are already repeatable. +- [x] RED: quality evidence CLI/script tests required + `codex_and_claude_code_integration` to include + `codex_claude_local_integration_evidence` while still keeping + `native_dialog_approved_dogfood` and `scorecard_level_below_9_5` as remaining + evidence; tests failed while only setup smoke and local sweep evidence were + satisfied. +- [x] GREEN: + `docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md` records the + non-operator integration evidence and `quality-evidence` now exposes it as a + separate satisfied evidence ref. +- [x] EFFECT: Codex/Claude evidence is more accurate without claiming the + operator-approved native dialog blocker is complete. + +### 판단 기준 + +- Do not run native OS dialog dogfood without explicit operator approval. +- Do not promote Codex/Claude integration to 9.5 while native approved dogfood + remains pending. +- Keep local setup/hook/MCP/elicitation evidence visible so agents do not rerun + stale local work unnecessarily. + ## 2026-07-06 PromptLane Web Operations Evidence Split - [x] CHECK: GitHub `ui-patrol.yml` still has only workflow_dispatch runs and