Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# PromptLane Codex And Claude Local Integration Evidence 2026-07-06

This document records current non-operator Codex and Claude Code integration
evidence for the PromptLane 9.5 quality bar.

## Repeatable Local Evidence

| Evidence | Result | Details |
| --- | --- | --- |
| `corepack pnpm smoke:agent-setup` | PASS | Exercises setup dry-run, setup with MCP registration, Claude Code doctor, and Codex doctor with isolated HOME/data directories and fake provider binaries. |
| `corepack pnpm smoke:hooks` | PASS | Verifies `prompt-coach` hook status, Claude Code fail-open behavior, Codex fail-open behavior, and `loopdeck` compatibility hook status. |
| `corepack pnpm smoke:mcp-coach-loop` | PASS | Exercises score, coach, improve, clarification apply/record, `score_prompt` effectiveness evidence, and `coach_prompt` effectiveness guidance through the real stdio MCP server. |
| `corepack pnpm smoke:mcp-elicitation` | PASS | Verifies MCP `elicitation/create` answer collection, final prompt composition, and local-only privacy boundaries. |
| `corepack pnpm smoke:mcp-native-dialog` | PASS | Verifies the no-dialog fallback path with `allow_native_dialog: false`, `interaction_status: "unsupported"`, and no OS dialog. |
| `corepack pnpm dogfood:first-coach-loop` | PASS | Captures a Codex prompt through the hook, then runs coach, loop collect, and loop brief against an isolated local archive. |
| `corepack pnpm dogfood:loop-memory-approval` | PASS | Captures a Codex prompt, records a loop outcome through MCP, proposes and records approved memory, and proposes an instruction patch without writing instruction files. |

## Boundary

This evidence does not complete the native OS dialog dogfood blocker.

`PROMPT_COACH_NATIVE_DIALOG_APPROVED=1 corepack pnpm
dogfood:mcp-native-dialog-approved` still requires an operator who is present
and expects a native dialog. The approved command refuses to run without the
approval environment variable and must not be added to automated CI or scheduled
checks.

## Interpretation

The Codex and Claude Code integration axis has current local setup, hook, MCP,
elicitation, no-dialog fallback, first-loop, and loop-memory dogfood evidence.
`quality-evidence` records this proven non-operator portion as
`codex_claude_local_integration_evidence` and keeps
`native_dialog_approved_dogfood` in remaining evidence.
6 changes: 6 additions & 0 deletions docs/NEXT_BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,12 @@ Decision:
`dogfood:web-user-flow` browser evidence. `quality-evidence` records this as
`manual_ui_patrol_artifact_evidence`, but `scheduled_ui_patrol` remains
pending until a real cron `schedule` event exists.
`docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md` records current
non-operator setup, hook, MCP, elicitation, no-dialog fallback, first-loop,
and loop-memory evidence. `quality-evidence` records this as
`codex_claude_local_integration_evidence`, while
`native_dialog_approved_dogfood` remains pending until explicit operator
approval exists.
- PR #478 moved that quality evidence gate onto the installed product CLI. Main
CI run `28753458359` passed Node 22 and Node 24 after merge, so `prompt-coach
quality-evidence --require-complete` is now a current default-branch release
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,14 @@
`corepack pnpm ui-patrol`, and `dogfood:web-user-flow`. `quality-evidence`
records this as `manual_ui_patrol_artifact_evidence` while keeping
`scheduled_ui_patrol` pending until a real cron `schedule` event exists.
- `docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md` records current
non-operator Codex and Claude Code evidence: setup/doctor smoke, hook smoke,
MCP coach-loop smoke, MCP elicitation smoke, no-dialog native fallback
preflight, first-loop dogfood, and loop-memory approval dogfood.
`quality-evidence` records this as
`codex_claude_local_integration_evidence` while keeping
`native_dialog_approved_dogfood` pending until explicit operator approval
exists.
- PR #478 proved that installed CLI path on the default branch; main CI run
`28753458359` passed Node 22 and Node 24 after merge, so future agents can use
the product CLI itself to decide whether 9.5 is still blocked before claiming
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@
"docs/LOCAL_95_EVIDENCE_2026-07-06.md",
"docs/PRODUCT_POSITIONING_EVIDENCE_2026-07-06.md",
"docs/UI_PATROL_EVIDENCE_2026-07-06.md",
"docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md",
"docs/RELEASE_CHECKLIST.md",
"plugins",
"integrations",
Expand Down
22 changes: 22 additions & 0 deletions scripts/quality-95-evidence.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,7 @@ function readCompletedEvidence() {
let claudePlugin = "";
let claudeMarketplace = "";
let uiPatrolEvidence = "";
let codexClaudeEvidence = "";
try {
plan = readFileSync(planPath, "utf8");
localEvidence = readFileSync("docs/LOCAL_95_EVIDENCE_2026-07-06.md", "utf8");
Expand All @@ -192,10 +193,15 @@ function readCompletedEvidence() {
"docs/UI_PATROL_EVIDENCE_2026-07-06.md",
"utf8",
);
codexClaudeEvidence = readFileSync(
"docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md",
"utf8",
);
} catch {
return {
product_positioning_metadata_alignment: false,
manual_ui_patrol_artifact_evidence: false,
codex_claude_local_integration_evidence: false,
web_user_flow_current_main_evidence: false,
privacy_raw_free_regression_sweep: false,
codex_claude_setup_smoke_refresh: false,
Expand Down Expand Up @@ -228,6 +234,16 @@ function readCompletedEvidence() {
uiPatrolEvidence.includes("Local `corepack pnpm ui-patrol`") &&
uiPatrolEvidence.includes("pending_no_schedule_run") &&
uiPatrolEvidence.includes("manual_ui_patrol_artifact_evidence"),
codex_claude_local_integration_evidence:
codexClaudeEvidence.includes("corepack pnpm smoke:agent-setup") &&
codexClaudeEvidence.includes("corepack pnpm smoke:hooks") &&
codexClaudeEvidence.includes("corepack pnpm smoke:mcp-coach-loop") &&
codexClaudeEvidence.includes("corepack pnpm smoke:mcp-elicitation") &&
codexClaudeEvidence.includes("corepack pnpm smoke:mcp-native-dialog") &&
codexClaudeEvidence.includes("corepack pnpm dogfood:first-coach-loop") &&
codexClaudeEvidence.includes("corepack pnpm dogfood:loop-memory-approval") &&
codexClaudeEvidence.includes("native_dialog_approved_dogfood") &&
codexClaudeEvidence.includes("codex_claude_local_integration_evidence"),
web_user_flow_current_main_evidence:
plan.includes("web_user_flow_current_main_evidence") &&
plan.includes("corepack pnpm dogfood:web-user-flow") &&
Expand Down Expand Up @@ -298,6 +314,12 @@ function axisEvidenceCoverage({
) {
satisfied.push("codex_claude_setup_smoke_refresh");
}
if (
axis.id === "codex_and_claude_code_integration" &&
completedEvidence.codex_claude_local_integration_evidence
) {
satisfied.push("codex_claude_local_integration_evidence");
}
if (
[
"local_first_privacy_boundary",
Expand Down
8 changes: 8 additions & 0 deletions src/cli/commands/quality-evidence.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,11 @@ describe("quality-evidence CLI command", () => {
expect.objectContaining({
id: "codex_and_claude_code_integration",
status: "blocked_external",
satisfied_evidence: expect.arrayContaining([
"codex_claude_setup_smoke_refresh",
"codex_claude_local_integration_evidence",
"local_95_evidence_sweep",
]),
remaining_evidence: expect.arrayContaining([
"native_dialog_approved_dogfood",
]),
Expand Down Expand Up @@ -169,6 +174,9 @@ describe("quality-evidence CLI command", () => {
expect(text).toContain(
"codex_and_claude_code_integration: blocked_external",
);
expect(text).toContain(
"satisfied=codex_claude_setup_smoke_refresh,codex_claude_local_integration_evidence,local_95_evidence_sweep",
);
expect(text).toContain("remaining=native_dialog_approved_dogfood");
expect(text).toContain("Scorecard review candidates");
expect(text).toContain("- none");
Expand Down
11 changes: 11 additions & 0 deletions src/packaging/plugin-files.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -948,6 +948,9 @@ describe("plugin packaging files", () => {
expect(packageJson.files).toContain(
"docs/UI_PATROL_EVIDENCE_2026-07-06.md",
);
expect(packageJson.files).toContain(
"docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md",
);
expect(packageJson.scripts["evidence:quality"]).toBe(
"node scripts/quality-95-evidence.mjs",
);
Expand All @@ -972,6 +975,9 @@ describe("plugin packaging files", () => {
"docs/PRODUCT_POSITIONING_EVIDENCE_2026-07-06.md",
);
expect(content).toContain("docs/UI_PATROL_EVIDENCE_2026-07-06.md");
expect(content).toContain(
"docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md",
);
expect(content).toContain("PR #478");
expect(content).toContain("28753458359");
expect(content).toContain("corepack pnpm evidence:quality");
Expand All @@ -985,6 +991,7 @@ describe("plugin packaging files", () => {
expect(content).toContain("blocked_by_external_event");
expect(content).toContain("product_positioning_metadata_alignment");
expect(content).toContain("manual_ui_patrol_artifact_evidence");
expect(content).toContain("codex_claude_local_integration_evidence");
expect(content).toContain("local-first privacy boundary");
expect(content).toContain("setup/doctor/MCP smoke");
expect(content).toContain("loop memory");
Expand Down Expand Up @@ -1047,6 +1054,10 @@ describe("plugin packaging files", () => {
expect(evidenceScript).toContain("PRODUCT_POSITIONING_EVIDENCE_2026-07-06");
expect(evidenceScript).toContain("manual_ui_patrol_artifact_evidence");
expect(evidenceScript).toContain("UI_PATROL_EVIDENCE_2026-07-06");
expect(evidenceScript).toContain("codex_claude_local_integration_evidence");
expect(evidenceScript).toContain(
"CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06",
);
expect(evidenceScript).toContain("web_user_flow_current_main_evidence");
expect(evidenceScript).toContain("browser e2e passed");
expect(evidenceScript).toContain("privacy_raw_free_regression_sweep");
Expand Down
1 change: 1 addition & 0 deletions src/packaging/quality-evidence-script.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,7 @@ describe("quality 9.5 evidence script", () => {
status: "blocked_external",
satisfied_evidence: expect.arrayContaining([
"codex_claude_setup_smoke_refresh",
"codex_claude_local_integration_evidence",
"local_95_evidence_sweep",
]),
remaining_evidence: expect.arrayContaining([
Expand Down
26 changes: 26 additions & 0 deletions tasks/todo.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# 작업 계획

## 2026-07-06 PromptLane Codex Claude Evidence Split

- [x] CHECK: Codex/Claude integration still has an external native-dialog
approved dogfood blocker, but local setup, hooks, MCP, elicitation, first
loop, and loop-memory dogfood evidence are already repeatable.
- [x] RED: quality evidence CLI/script tests required
`codex_and_claude_code_integration` to include
`codex_claude_local_integration_evidence` while still keeping
`native_dialog_approved_dogfood` and `scorecard_level_below_9_5` as remaining
evidence; tests failed while only setup smoke and local sweep evidence were
satisfied.
- [x] GREEN:
`docs/CODEX_CLAUDE_LOCAL_INTEGRATION_EVIDENCE_2026-07-06.md` records the
non-operator integration evidence and `quality-evidence` now exposes it as a
separate satisfied evidence ref.
- [x] EFFECT: Codex/Claude evidence is more accurate without claiming the
operator-approved native dialog blocker is complete.

### 판단 기준

- Do not run native OS dialog dogfood without explicit operator approval.
- Do not promote Codex/Claude integration to 9.5 while native approved dogfood
remains pending.
- Keep local setup/hook/MCP/elicitation evidence visible so agents do not rerun
stale local work unnecessarily.

## 2026-07-06 PromptLane Web Operations Evidence Split

- [x] CHECK: GitHub `ui-patrol.yml` still has only workflow_dispatch runs and
Expand Down