Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/NEXT_BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,11 @@ Decision:
`native_dialog_preflight` evidence: it proves MCP elicitation and no-dialog
fallback behavior while explicitly refusing to treat that as
`native_dialog_approved_dogfood`.
- `quality-evidence` now includes structured external action criteria on its
recommended next slices. The scheduled `ui-patrol` and native dialog operator
dogfood recommendations include preconditions, completion evidence, and
guardrails so agents can unblock the remaining 9.5 work without confusing
manual/readiness evidence with real external completion.
- PR #478 moved that quality evidence gate onto the installed product CLI. Main
CI run `28753458359` passed Node 22 and Node 24 after merge, so `prompt-coach
quality-evidence --require-complete` is now a current default-branch release
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,10 @@
- `docs/NATIVE_DIALOG_DOGFOOD_AUDIT_2026-07-05.md` is packaged as
`native_dialog_preflight` evidence for MCP elicitation and no-dialog fallback
behavior. It must not be treated as approved native OS dialog dogfood.
- `quality-evidence` recommended next slices now carry explicit preconditions,
completion evidence, and guardrails for the remaining external blockers. This
makes the next operator or scheduled-event pass executable without changing
the rule that 9.5 remains pending until real external evidence exists.
- PR #478 proved that installed CLI path on the default branch; main CI run
`28753458359` passed Node 22 and Node 24 after merge, so future agents can use
the product CLI itself to decide whether 9.5 is still blocked before claiming
Expand Down
20 changes: 20 additions & 0 deletions scripts/quality-95-evidence.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -493,6 +493,16 @@ function recommendedNextSlices({
priority: 90,
blocked_by_external_event: true,
command: "corepack pnpm evidence:ui-patrol",
preconditions: [
"A real GitHub Actions schedule event exists for ui-patrol.yml.",
],
completion_evidence: [
"scheduled_ui_patrol status is complete",
"ui-patrol-screenshots artifact contains 9 png files",
],
guardrails: [
"Do not treat workflow_dispatch evidence as scheduled evidence.",
],
expected_effect:
"Verify the first real scheduled screenshot artifact after GitHub cron runs.",
});
Expand All @@ -506,6 +516,16 @@ function recommendedNextSlices({
blocked_by_external_event: true,
command:
"PROMPT_COACH_NATIVE_DIALOG_APPROVED=1 corepack pnpm dogfood:mcp-native-dialog-approved",
preconditions: [
"The operator explicitly approves opening a native OS dialog.",
],
completion_evidence: [
'interaction_status: "answered"',
"approved native dialog dogfood passed",
],
guardrails: [
"Do not run this command in automated CI or scheduled checks.",
],
expected_effect:
"Prove the real native ask UI handoff only after explicit operator approval.",
});
Expand Down
35 changes: 35 additions & 0 deletions src/cli/commands/quality-evidence.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ describe("quality-evidence CLI command", () => {
priority: number;
blocked_by_external_event: boolean;
command: string;
preconditions?: string[];
completion_evidence?: string[];
guardrails?: string[];
}>;
};

Expand Down Expand Up @@ -133,7 +136,32 @@ describe("quality-evidence CLI command", () => {
priority: 90,
blocked_by_external_event: true,
command: "corepack pnpm evidence:ui-patrol",
preconditions: expect.arrayContaining([
"A real GitHub Actions schedule event exists for ui-patrol.yml.",
]),
completion_evidence: expect.arrayContaining([
"scheduled_ui_patrol status is complete",
]),
guardrails: expect.arrayContaining([
"Do not treat workflow_dispatch evidence as scheduled evidence.",
]),
});
expect(parsed.recommended_next_slices).toEqual(
expect.arrayContaining([
expect.objectContaining({
id: "native_dialog_operator_dogfood",
preconditions: expect.arrayContaining([
"The operator explicitly approves opening a native OS dialog.",
]),
completion_evidence: expect.arrayContaining([
'interaction_status: "answered"',
]),
guardrails: expect.arrayContaining([
"Do not run this command in automated CI or scheduled checks.",
]),
}),
]),
);
expect(parsed.recommended_next_slices).not.toEqual(
expect.arrayContaining([
expect.objectContaining({
Expand Down Expand Up @@ -188,6 +216,13 @@ describe("quality-evidence CLI command", () => {
expect(text).toContain("scheduled_ui_patrol_cron_review");
expect(text).toContain("corepack pnpm evidence:ui-patrol");
expect(text).toContain("external event: yes");
expect(text).toContain(
"preconditions=A real GitHub Actions schedule event exists for ui-patrol.yml.",
);
expect(text).toContain("completion=scheduled_ui_patrol status is complete");
expect(text).toContain(
"guardrails=Do not treat workflow_dispatch evidence as scheduled evidence.",
);
expect(text).toContain("Privacy: local-only");
expect(text).not.toContain(process.cwd());
});
Expand Down
20 changes: 16 additions & 4 deletions src/cli/commands/quality-evidence.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ type QualityEvidenceSummary = {
priority: number;
blocked_by_external_event: boolean;
command: string;
preconditions?: string[];
completion_evidence?: string[];
guardrails?: string[];
expected_effect: string;
}>;
next_action: string;
Expand Down Expand Up @@ -107,10 +110,19 @@ function formatSummary(summary: QualityEvidenceSummary): string {
: ["- none"];
const recommendedRows =
summary.recommended_next_slices && summary.recommended_next_slices.length > 0
? summary.recommended_next_slices.map(
(slice) =>
`- ${slice.priority}. ${slice.id} (external event: ${slice.blocked_by_external_event ? "yes" : "no"}) ${slice.command} - ${slice.expected_effect}`,
)
? summary.recommended_next_slices.flatMap((slice) => [
`- ${slice.priority}. ${slice.id} (external event: ${slice.blocked_by_external_event ? "yes" : "no"}) ${slice.command} - ${slice.expected_effect}`,
...(slice.preconditions && slice.preconditions.length > 0
? [` preconditions=${slice.preconditions.join("; ")}`]
: []),
...(slice.completion_evidence &&
slice.completion_evidence.length > 0
? [` completion=${slice.completion_evidence.join("; ")}`]
: []),
...(slice.guardrails && slice.guardrails.length > 0
? [` guardrails=${slice.guardrails.join("; ")}`]
: []),
])
: ["- none"];
const axisCoverageRows =
summary.axis_evidence_coverage && summary.axis_evidence_coverage.length > 0
Expand Down
23 changes: 23 additions & 0 deletions src/packaging/quality-evidence-script.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,9 @@ describe("quality 9.5 evidence script", () => {
blocked_by_external_event: boolean;
command: string;
expected_effect: string;
preconditions?: string[];
completion_evidence?: string[];
guardrails?: string[];
}>;
next_action: string;
};
Expand Down Expand Up @@ -259,10 +262,30 @@ describe("quality 9.5 evidence script", () => {
expect.objectContaining({
id: "scheduled_ui_patrol_cron_review",
blocked_by_external_event: true,
preconditions: expect.arrayContaining([
"A real GitHub Actions schedule event exists for ui-patrol.yml.",
]),
completion_evidence: expect.arrayContaining([
"scheduled_ui_patrol status is complete",
"ui-patrol-screenshots artifact contains 9 png files",
]),
guardrails: expect.arrayContaining([
"Do not treat workflow_dispatch evidence as scheduled evidence.",
]),
}),
expect.objectContaining({
id: "native_dialog_operator_dogfood",
blocked_by_external_event: true,
preconditions: expect.arrayContaining([
"The operator explicitly approves opening a native OS dialog.",
]),
completion_evidence: expect.arrayContaining([
'interaction_status: "answered"',
"approved native dialog dogfood passed",
]),
guardrails: expect.arrayContaining([
"Do not run this command in automated CI or scheduled checks.",
]),
}),
]),
);
Expand Down
19 changes: 19 additions & 0 deletions tasks/todo.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,24 @@
# 작업 계획

## 2026-07-06 PromptLane External Evidence Action Criteria

- [x] CHECK: The remaining 9.5 blockers are external, but the recommended next
slices only exposed command and expected effect.
- [x] RED: quality evidence CLI/script tests required external recommendations
to include preconditions, completion evidence, and guardrails; tests failed
while those fields were absent.
- [x] GREEN: `quality-evidence` now emits those fields for scheduled
`ui-patrol` review and native dialog operator dogfood, and the CLI text
renders them below each recommendation.
- [x] EFFECT: future operators and agents can identify what must be true before
running the action, what proves completion, and what must not be substituted.

### 판단 기준

- Do not remove existing blockers through action criteria alone.
- Do not treat manual/readiness evidence as scheduled or approved evidence.
- Keep the structured criteria raw-free and path-free in CLI output.

## 2026-07-06 PromptLane Native Dialog Preflight Evidence

- [x] CHECK: Approved native OS dialog dogfood still requires explicit operator
Expand Down