Audit remaining TDD/HIL agent customization debt after #139

[zarfld]

Scope

This follow-up issue audits the remaining customization debt after the HIL/TDD baseline implemented in #138 and #139. The implemented HIL baseline is treated as complete and is not redesigned here.

Findings

• The TDDDriver agent still uses generic TypeScript/Jest/JWT/auth examples instead of PoKeysHal-specific C99, PoKeysLib, LinuxCNC HAL, and RT-safe TDD guidance.
• The TestingSpecialist agent still reflects a generic web-app test stack (Jest, Cypress, Playwright, databases, REST, browser flows) rather than the repository's real test layers.
• The repository prompts still contain off-repo examples that should be removed or replaced with PoKeysHal-specific patterns.
• There is still overlap between agent roles, prompts, instructions, and skills around TDD flow, issue traceability, and evidence-based verification.

Planned corrective phases

Phase 1 — Reframe agent responsibilities

• Rewrite TDDDriver for PoKeysHal's actual implementation workflow: C99, PoKeysLib, LinuxCNC HAL, RT-safe changes, and HIL-aware TDD.
• Rewrite TestingSpecialist for the repository's real test layers: compile smoke tests, userspace HAL smoke tests, RT tests, protocol/async tests, and HIL fixture validation.
• Keep the responsibilities distinct: TDDDriver owns implementation-side Red-Green-Refactor execution; TestingSpecialist owns test design, coverage strategy, and verification evidence.

Phase 2 — Remove generic examples and consolidate duplicated rules

• Remove TypeScript, JWT, REST, database, browser, Java, Jest, Cypress, and Playwright examples from the relevant agent and prompt content.
• Consolidate duplicated rules that currently appear in agents, prompts, instructions, and skills into a smaller, repo-specific guidance set.

Phase 3 — Align prompts with the repository reality

• Rewrite tdd-compile.prompt.md so it reflects PoKeysHal's actual TDD workflow and traceability expectations.
• Rewrite repository-audit.prompt.md so it audits the repository's issue traceability, HIL/TDD scaffolding, C/RT build/test layers, and documentation consistency rather than generic web-app architecture.
• Validate the result with a short repository-specific checklist before closing the issue.

Acceptance criteria

• TDDDriver and TestingSpecialist no longer contain off-repo examples.
• The two agents still have distinct responsibilities and do not duplicate each other.
• The prompts and audit guidance reflect the PoKeysHal/HIL baseline implemented in feat: HIL-TDD infrastructure skeleton — fixture definition, instruction, skill, test directories #138 and feat: HIL-TDD infrastructure skeleton (issue #138) #139.
• No redesign of the already implemented HIL baseline is introduced in this issue.

[zarfld]

Audit output for #140

This audit is scoped to the current repository state on main after PR #139 and limits itself to the remaining customization debt described in #140.

Baseline treated as approved and implemented

The following baseline items are not redesigned here:

• .github/instructions/hil-tdd.instructions.md
• .github/skills/hil-tdd/SKILL.md
• .github/skills/hil-tdd/references/result-schema.md
• tests/hil/setups/pokeys57e-loopback-v1.yaml
• tests/hil/setups/pokeys57e-loopback-v1.md
• tests/hil/basic/
• tests/hil/machine/linuxcnc/
• existing HIL routing hooks in the testing agents

Scope of this audit

• tdd-driver.agent.md
• testing-specialist.agent.md
• tdd-compile.prompt.md
• repository-audit.prompt.md
• duplicated TDD, traceability, test-layer, and evidence rules across active agents, prompts, instructions, and skills

Severity-ranked findings

P0 — invalid or misleading active customization

1. TDDDriver is not repository-specific: it still teaches generic TypeScript/Jest/JWT/auth examples, which are not aligned with PoKeysHal, C99, PoKeysLib, LinuxCNC HAL, or RT-safe implementation work.
2. TestingSpecialist is not repository-specific: it still centers on web-app stacks with Jest, Cypress, Playwright, browser flows, databases, and REST, which are not representative of this repository's actual test layers.
3. tdd-compile.prompt.md is mis-scoped: it still instructs a generic web/auth application TDD workflow and is not a good fit for PoKeysHal's C/RT/HAL/HIL reality.
4. repository-audit.prompt.md is over-generic: it audits a generic issue-driven web-app template rather than the repository's actual structure, test layers, and HIL customization debt.

P1 — excessive or contradictory ownership

5. The two agents overlap too much: both currently describe TDD cadence, issue traceability, test-first behavior, and generic coverage expectations, even though they should have distinct responsibilities.
6. The prompts duplicate instruction/skill content: the same TDD and traceability guidance is effectively repeated in prompts, agents, and repository-wide instructions instead of being centralized into short mandatory guidance and a focused skill.

P2 — missing infrastructure for the next phase

7. No compact customization architecture exists yet for the remaining debt: there is no clear split between agent role, prompt workflow, and shared instruction/skill vocabulary.
8. The audit output needs a tighter line budget and file disposition plan so future edits remain small, reviewable, and scoped to the approved HIL baseline.

Deferred

9. Actual HIL implementation remains out of scope for this audit, including hardware execution, fixture validation, and new HIL tests.

Target customization architecture

Keep the approved HIL baseline intact and introduce a small repo-specific customization layer:

• Instructions: short mandatory invariants only (RT safety, fixture gate, evidence, traceability).
• Skills: detailed repeatable procedures only for HIL/TDD workflow and evidence capture.
• Agents: one implementation-focused agent and one test/verification-focused agent; each with a short, distinct purpose.
• Prompts: one narrow TDD prompt for PoKeysHal implementation work and one repository-audit prompt for repo-specific audits.
• Canonical docs: one authoritative HIL/TDD synthesis and one result-schema reference.

File-by-file disposition

Current file	Keep	Rewrite	Split	Archive	Delete	Target size	Reason
.github/agents/tdd-driver.agent.md		✓				120-180 lines	Rewrite for PoKeysHal C99/PoKeysLib/LinuxCNC HAL/RT-safe TDD.
.github/agents/testing-specialist.agent.md		✓				120-180 lines	Rewrite for actual PoKeysHal test layers and evidence-based verification.
.github/prompts/tdd-compile.prompt.md		✓				80-140 lines	Replace generic web/app examples with PoKeysHal-focused TDD workflow.
.github/prompts/repository-audit.prompt.md		✓				100-160 lines	Rewrite for repo-specific audit scope instead of generic web-project assumptions.
.github/instructions/hil-tdd.instructions.md	✓					40-80 lines	Approved baseline; keep as mandatory contract.
.github/skills/hil-tdd/SKILL.md	✓					180-260 lines	Approved baseline; keep detailed procedure.
.github/skills/hil-tdd/references/result-schema.md	✓					60-120 lines	Approved baseline; keep shared evidence vocabulary.

Preferred and maximum line budgets

• TDDDriver: preferred 120-180 lines; maximum 220 lines.
• TestingSpecialist: preferred 120-180 lines; maximum 220 lines.
• TDD compile prompt: preferred 80-140 lines; maximum 180 lines.
• Repository audit prompt: preferred 100-160 lines; maximum 220 lines.
• HIL routing additions: preferred 20-40 lines; maximum 60 lines.
• Future HIL instruction: preferred 40-80 lines; maximum 120 lines.
• Future HIL skill: preferred 180-260 lines; maximum 320 lines.

Small phased PR plan

Phase 1 — Rewrite agents

Goal: make the two agents distinct and repo-specific.

Files changed:

• .github/agents/tdd-driver.agent.md
• .github/agents/testing-specialist.agent.md

Acceptance criteria:

• both agents contain PoKeysHal-specific examples and terminology;
• TDDDriver focuses on implementation-side Red-Green-Refactor for C99/PoKeysLib/HAL/RT-safe changes;
• TestingSpecialist focuses on test design, verification evidence, and repository test layers;
• no TypeScript/Jest/JWT/REST/database/browser/Java/Cypress/Playwright examples remain in either agent.

Verification commands:

• grep -RInE "TypeScript|JWT|REST|database|browser|Java|Jest|Cypress|Playwright|bcrypt|jsonwebtoken" .github/agents
• git diff --check

Out of scope:

• HIL tests, hardware, fixture changes, and new HAL code.

Human approval gate:

• approve that TDDDriver and TestingSpecialist have distinct responsibilities before Phase 2 begins.

Phase 2 — Rewrite prompts

Goal: replace generic prompt content with PoKeysHal-specific workflow guidance.

Files changed:

• .github/prompts/tdd-compile.prompt.md
• .github/prompts/repository-audit.prompt.md

Acceptance criteria:

• prompts no longer contain generic web/auth examples;
• prompts reference PoKeysHal build/test layers, issue traceability, and approved HIL baseline;
• prompts stay within the preferred line budgets.

Verification commands:

• grep -RInE "TypeScript|JWT|REST|database|browser|Java|Jest|Cypress|Playwright" .github/prompts
• python3 - <<PY ... (basic line-budget check)
• git diff --check

Out of scope:

• implementation of new tests or HIL behavior.

Human approval gate:

• approve the new prompt scope and terminology before consolidation begins.

Phase 3 — Consolidate duplicated customization rules

Goal: remove duplicated TDD, traceability, and evidence guidance across active customization layers.

Files changed:

• one small shared guidance update, if needed, in .github/instructions/engineering-discipline.instructions.md or a dedicated repo-specific guidance file;
• optional narrow routing text in agent metadata or prompt front matter only.

Acceptance criteria:

• duplicated rules are reduced to one short mandatory source and one detailed skill;
• prompts and agents reference the same evidence and traceability vocabulary;
• no broad rewrite of the approved HIL baseline is introduced.

Verification commands:

• grep -RInE "Red-Green-Refactor|Verifies: #N|Implements: #N|result schema|fixture gate" .github
• git diff --check

Out of scope:

• any new hardware-facing test or fixture design.

Human approval gate:

• approve the final rule split between instructions, skills, prompts, and agents.

Explicit exclusions

• No file edits in this audit.
• No hardware operation.
• No HIL test implementation.
• No redesign of the approved baseline from feat: HIL-TDD infrastructure skeleton — fixture definition, instruction, skill, test directories #138/feat: HIL-TDD infrastructure skeleton (issue #138) #139.
• No broad one-PR rewrite across the entire customization stack.

Recommended child issues

1. Rewrite TDDDriver and TestingSpecialist
2. Rewrite TDD compile and repository-audit prompts
3. Consolidate duplicated customization rules

Recommendation

Implement the above as three small child issues and keep each PR focused on one phase. This is safer than a single large PR and keeps the approved HIL baseline unchanged.

[zarfld]

Phase completion: all phases merged ✓

This issue is complete. All corrective phases have been implemented and merged.

Approved HIL/TDD baseline (#139)

PR #139 merge: da6a82157fc5d485a8711e4f0bad4c230e464928

Implemented: HIL instruction, skill, result schema, fixture YAML skeleton,
and test directories. The physical fixture is fixture_status: draft and
runnable: false — hardware qualification is a separate workstream
explicitly deferred by #138.

Phase 1 — Agent rewrite (#141 / PR #142)

Merge: 7adb24d8fc0f7ece758f2d411605c5a7d842f303

Rewrote TDDDriver for PoKeysHal C99/PoKeysLib/LinuxCNC HAL/RT-safe work and
TestingSpecialist for the repository's real test layers. Removed all
TypeScript/JWT/REST/database/browser/Jest/Cypress/Playwright references from
the two target agents. Established distinct primary responsibilities:
TDDDriver owns Red-Green-Refactor; TestingSpecialist owns verification-layer
selection, oracle review, and evidence interpretation.

Phase 2 — Prompt rewrite (#143 / PR #145)

Merge: b3b08234f38e7380d9abc7bc984ed9e877d026a6

Rewrote tdd-compile.prompt.md (narrow orchestration entry point, 65 lines)
and repository-audit.prompt.md (read-only PoKeysHal audit workflow, 150 lines).
Both reference HIL status vocabulary from result-schema.md and contain no
generic web-application content. VS Code prompt-picker invocation was not
verified after amendment; content and syntax were reviewed.

Phase 3 — Rule consolidation (#146 / PR #147 + PR #148)

PR #147 merge: 7485fadcc40246f81e6cdf1c706307819e2dc340

• Replaced legacy Hardware-verified with HIL-observed, HIL-test-executed,
  and HIL-verified in AGENTS.md and engineering-discipline.instructions.md.
• Removed duplicated PASS/FAIL/ERROR/SKIPPED preflight outcome table from
  SKILL.md; replaced with reference to result-schema.md.
• Added explicit migration guidance for legacy HIL terminology.

PR #148 merge: cd1f0dc8a0f64f92dc6bdce21bddcb36d33a14cd

• Removed the remaining operative use of Hardware-verified from
  update-submodule/SKILL.md.
• Build/test evidence (Compiled/Tested) and HIL status are now distinct
  requirements rather than alternatives.

Acceptance criteria

Criterion	Result
No off-repo examples in TDDDriver and TestingSpecialist	PASS — zero matches for web-stack patterns in the four target files
Agents have distinct primary responsibilities, no longer duplicate the complete workflow	PASS — TDDDriver owns implementation; TestingSpecialist owns verification strategy
Prompts reflect PoKeysHal/HIL baseline	PASS — both prompts are PoKeysHal-specific; HIL definitions delegated to result-schema
#138/#139 HIL baseline not materially redesigned	PASS — only deliberate Phase 3 consolidation to SKILL.md; fixture, safety rules, result schema, and tests unchanged

Final verification (run on main at cd1f0dc)

# Generic stack content — four target files
git grep -E 'TypeScript|JWT|REST|database|browser|Jest|Cypress|Playwright' \
  .github/agents/tdd-driver.agent.md \
  .github/agents/testing-specialist.agent.md \
  .github/prompts/tdd-compile.prompt.md \
  .github/prompts/repository-audit.prompt.md
# → no matches

# Repository-wide legacy HIL terminology
git grep -E 'Hardware-verified|hardware-verified|HIL-tested' -- \
  AGENTS.md .github/agents .github/instructions .github/prompts .github/skills
# → apply-tdd-infrastructure-plan.prompt.md:94 (explicit migration warning — permitted)
# → hil-tdd-guidance-synthesize.prompt.md:147 (read-only analysis prompt — permitted)
# → no operative guidance uses legacy terms

# HIL baseline diff from PR #139
git diff da6a82157fc5d485a8711e4f0bad4c230e464928..HEAD -- \
  .github/instructions/hil-tdd.instructions.md \
  .github/skills/hil-tdd/references/result-schema.md \
  tests/hil
# → no diff

Explicit statement

No HIL fixture YAML, wiring description, safety rules, result schema,
production C/HAL behavior, or hardware test implementation was redesigned
or changed by issues #141, #143, or #146.

The physical fixture remains draft and non-runnable. Hardware qualification
is a separate workstream explicitly deferred by #138.

Closing as complete.