Skip to content

[codex] add Judge QA delivery gates - #66

Draft
greenthree wants to merge 1 commit into
codex/judge-qa-executionfrom
codex/judge-qa-cli
Draft

[codex] add Judge QA delivery gates#66
greenthree wants to merge 1 commit into
codex/judge-qa-executionfrom
codex/judge-qa-cli

Conversation

@greenthree

Copy link
Copy Markdown
Owner

Summary

  • add the judge-qa CLI with content-addressed compile caching while rerunning every fixture and probe
  • require current passed Judge QA evidence in seal and formal build checkpoints
  • expose bounded evidence state in lint, status, and report
  • document the Schema v1 workflow and Agent delivery requirements
  • strip Validator, Checker, and Interactor diagnostic bodies from evidence and retain only bounded byte summaries

Why

P0 defined the fixture schema and P1 added isolated execution, but authors still had no supported CLI or formal delivery gate. This change makes active Checker/Interactor testing part of the Schema v1 sealing workflow without changing WebUI or Legacy behavior.

User impact

New or modified custom/interactive problems can run probhub judge-qa <ID> --no-cache. Existing workspaces without judge.qa remain compatible. Failed, cancelled, stale, or invalid QA runs do not replace the last successful evidence or formal PDF/ZIP/Manifest artifacts.

Validation

  • npm run check: 663 passed, 30 skipped
  • Judge QA/build focused regression: 80 passed
  • npm run pack:check: 218 main-package files, 5 compatibility-package files
  • Skill validation and git diff --check
  • local-only phtest nine-problem lint
  • L02 no-cache Judge: 33 inputs, 3 solutions, 99 case executions, all_expectations_met

Stacked on Draft PR #65 (codex/judge-qa-execution).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant