[I001] Add first-class Kata UAT skill

[gannonh]

Design

Problem

Kata CLI currently has an evidence-oriented verification workflow through kata-verify-work, but no first-class human UAT workflow.

After execution, the system correctly routes users toward evidence-backed verification, but there is no portable CLI skill that helps a human manually accept a completed slice, records acceptance evidence, or routes failed acceptance into follow-up milestone work.

Goals

• Add a dedicated kata-uat skill to the CLI skill bundle.
• Keep kata-verify-work as the default next action after execution.
• Make UAT human-discretionary, not automatic state-machine routing.
• Let kata-uat synthesize a manual UAT playbook from completed slice context.
• Persist UAT results as a durable uat artifact.
• If UAT fails, guide the user toward planning a gap slice through kata-plan-phase.

Non-goals

• Do not replace kata-verify-work.
• Do not require every slice to run UAT.
• Do not make UAT a task-level concept.
• Do not introduce complex automatic routing or new verification states.
• Do not automatically create gap slices from failed UAT without normal planning approval.

Proposed approach

Introduce kata-uat as a separate portable CLI skill.

The default post-execution recommendation remains:

Next up: run `kata-verify-work` to record verification evidence.

But kata-execute-phase should also mention that users may run kata-uat when they want human acceptance:

Optional: run `kata-uat` for a human UAT playbook and sign-off.

When invoked, kata-uat loads the active milestone/slice context, including the slice plan, tasks, summaries, requirements, and relevant verification artifacts. It then prepares a manual UAT playbook for the human to run through.

The human reports pass/fail results back to the agent. The agent records those results in a uat artifact.

If UAT passes, the artifact captures sign-off and evidence.

If UAT fails, the artifact captures blockers, reproduction notes, and recommended follow-up. The next recommended workflow should be kata-plan-phase for a gap slice in the active milestone.

Affected files or surfaces

• apps/cli/skills-src/manifest.json: add kata-uat.
• apps/cli/skills-src/workflows/uat.md: new UAT workflow.
• apps/cli/skills-src/templates/UAT.md: strengthen as the primary UAT artifact template.
• apps/cli/skills-src/workflows/execute-phase.md: keep kata-verify-work next, add optional kata-uat suggestion.
• apps/cli/skills-src/workflows/verify-work.md: clarify that UAT is separate and human-invoked.
• apps/cli/scripts/bundle-skills.mjs: update bundle expectations if needed.
• CLI skill-surface and bundle tests under apps/cli/src/tests/.

Risks and edge cases

• Risk: agents may treat UAT as required for every slice.
  • Mitigation: workflow text should say UAT is human-discretionary unless the user asks for it.
• Risk: kata-uat and kata-verify-work may overlap confusingly.
  • Mitigation: define the boundary clearly: verification records evidence; UAT records human acceptance.
• Risk: failed UAT could bypass planning discipline.
  • Mitigation: failed UAT should recommend kata-plan-phase for a gap slice, not create scope directly.
• Risk: generated installed skills may drift from source skills.
  • Mitigation: include bundle generation and tests in validation.

Verification

• Run CLI tests covering skill manifest and bundle generation.
• Confirm kata-uat appears in generated installed skills.
• Confirm kata-execute-phase still recommends kata-verify-work as the primary next action.
• Confirm kata-execute-phase also mentions optional kata-uat.
• Confirm kata-uat workflow writes uat artifacts and routes failed acceptance to kata-plan-phase.

Plan

Tasks

• Step 1: Update the portable CLI skill manifest.
  • Add a kata-uat entry to apps/cli/skills-src/manifest.json.
  • Give it a description focused on human acceptance testing, manual UAT playbooks, sign-off, and failed-UAT follow-up.
  • Include required operations needed to load context and write UAT artifacts, likely: project.getContext, project.getSnapshot, milestone.getActive, slice.list, task.list, artifact.list, artifact.read, and artifact.write.
• Step 2: Add the new UAT workflow source.
  • Create apps/cli/skills-src/workflows/uat.md.
  • Workflow stages should cover loading project/snapshot/slice context, reading slice plan/task summaries/requirements/verification artifacts, drafting a human UAT playbook, asking the human to run it and report pass/fail, writing a uat artifact, summarizing sign-off when passed, and recommending kata-plan-phase for a gap slice when failed.
• Step 3: Strengthen the UAT artifact template.
  • Update apps/cli/skills-src/templates/UAT.md to support scope, playbook/checklist, human observations, pass/fail result, issues found, sign-off, and recommended gap-slice inputs when failed.
• Step 4: Update execution workflow messaging.
  • Edit apps/cli/skills-src/workflows/execute-phase.md.
  • Keep the primary completion next step as kata-verify-work.
  • Add optional wording that users can run kata-uat for a human UAT playbook/sign-off.
• Step 5: Clarify verification workflow boundary.
  • Edit apps/cli/skills-src/workflows/verify-work.md.
  • State that kata-verify-work remains evidence-oriented.
  • State that human acceptance/sign-off should use kata-uat when the user asks for UAT.
  • Avoid making UAT automatic or required for every slice.
• Step 6: Update bundle generation expectations if needed.
  • Inspect apps/cli/scripts/bundle-skills.mjs.
  • Update any fixed list of expected skills to include kata-uat.
  • Run the skill bundle generation command used by this repo.
• Step 7: Add or update tests.
  • Update apps/cli/src/tests/phase-a-skill-surface.vitest.test.ts and related bundle/quality tests as needed.
  • Assertions should cover: manifest exposes kata-uat, generated skill bundle includes apps/cli/skills/kata-uat, kata-uat references templates/UAT.md, execute-phase.md keeps kata-verify-work as primary next step, failed UAT routes to kata-plan-phase, and UAT is human-discretionary rather than automatic task-level verification.
• Step 8: Generate and validate.
  • Run the CLI skill bundle script.
  • Run targeted CLI tests, for example pnpm --filter @kata-sh/cli test.
  • If needed, run broader validation with pnpm run validate:affected.

Acceptance criteria

• kata-uat is available as a first-class portable CLI skill.
• kata-uat produces a human UAT playbook from completed slice context.
• UAT results are persisted as uat artifacts.
• Failed UAT recommends kata-plan-phase for a gap slice.
• kata-execute-phase still recommends kata-verify-work as the primary next action.
• kata-execute-phase also mentions optional kata-uat.
• kata-verify-work remains evidence-oriented and does not absorb human UAT.
• UAT is not modeled at the task level.
• CLI skill bundle tests pass.

Execution notes

• Do not change backend status semantics unless tests reveal a necessary source update.
• Prefer workflow/template-only changes unless bundle generation requires script updates.
• Keep kata-uat human-invoked and human-discretionary.
• Do not automatically create gap slices from failed UAT; only route to kata-plan-phase.
• Generated skill files under apps/cli/skills/ may need updating after source changes.

[coderabbitai]

⚠️ Possible Duplicate Issue(s)

• Add human UAT workflow to Kata CLI skills #446

🔗 Related PRs

#84 - refactor: unify prompt templates across file and Linear backends [merged]
#100 - CLI v0.4.0 [merged]
#195 - [S02] auto.ts Decision Logic Extraction & Tests [merged]
#314 - CLI v0.15.4 [merged]
#332 - [M009/S02] GitHub planning artifact authoring in CLI [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!