You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add field testing to the QEP process: once a QEP is in Draft, run one or more real decisions against the draft text and record the outcome in the discussion PR, before the decision deadline. This would land as an in-place amendment to QEP-1's "How a QEP is decided" section if adopted.
Where the idea comes from
QEP-3 (#7) got two rounds of field testing during its comment window, and both were more informative than any amount of reading the text:
A validation. A live placement decision (project-reporting vs workspace-reporting) was run against the draft's goal-vs-fleet boundary rules, which decided it without interpretation. That is direct evidence the rules work in practice — exactly what a reviewer wants to know before accepting a standard, and something a read-through cannot establish.
A defect found while the draft was still cheap to change. Pulling on one thread the test exposed (reports-activity doesn't parse under the registry) revealed that the workflow-* type names a mechanism rather than a role, and produced a concrete restructure proposal (reporter-*/task-*) before acceptance rather than as a post-acceptance amendment.
An earlier field report on the same PR (the translation program's bench placement) surfaced two text gaps that were fixed in-flight (8150620).
The pattern generalises: a standard-type QEP exists to settle recurring decisions, so the natural test of a draft is to settle one real decision with it and see whether the text was sufficient. Every outcome is useful — a clean decision becomes validation evidence, a strained one becomes an amendment while amending is free.
Proposal sketch
Amend QEP-1's decision process with an optional step between Draft and Decide:
(Optional, encouraged for standards) Field test. Apply the draft to at least one real decision it claims to govern, and record the outcome as a field report comment on the discussion PR: what was decided, whether the text sufficed, and any amendments it motivated. Field reports are review evidence at the deadline.
Deliberately lightweight: no template, no gate, no new status — just a named, expected activity so authors and reviewers know it counts as review.
Points to decide
Optional vs expected. Fully optional, or expected for type: standard QEPs (which govern recurring decisions and therefore always have a testable case) while remaining optional for process/informational QEPs?
Interaction with the deadline. Leave the one-to-two-week window unchanged, or note that a substantive amendment arising from a field test should reset a short fresh window (as is happening on QEP-3: Repository Naming and Types #7)?
Who runs it. The author testing their own draft is still valuable (both QEP-3: Repository Naming and Types #7 rounds were author-run), but a field test by a second maintainer is stronger evidence; worth encouraging without requiring.
Idea
Add field testing to the QEP process: once a QEP is in Draft, run one or more real decisions against the draft text and record the outcome in the discussion PR, before the decision deadline. This would land as an in-place amendment to QEP-1's "How a QEP is decided" section if adopted.
Where the idea comes from
QEP-3 (#7) got two rounds of field testing during its comment window, and both were more informative than any amount of reading the text:
project-reportingvsworkspace-reporting) was run against the draft's goal-vs-fleet boundary rules, which decided it without interpretation. That is direct evidence the rules work in practice — exactly what a reviewer wants to know before accepting a standard, and something a read-through cannot establish.reports-activitydoesn't parse under the registry) revealed that theworkflow-*type names a mechanism rather than a role, and produced a concrete restructure proposal (reporter-*/task-*) before acceptance rather than as a post-acceptance amendment.The pattern generalises: a standard-type QEP exists to settle recurring decisions, so the natural test of a draft is to settle one real decision with it and see whether the text was sufficient. Every outcome is useful — a clean decision becomes validation evidence, a strained one becomes an amendment while amending is free.
Proposal sketch
Amend QEP-1's decision process with an optional step between Draft and Decide:
Deliberately lightweight: no template, no gate, no new status — just a named, expected activity so authors and reviewers know it counts as review.
Points to decide
type: standardQEPs (which govern recurring decisions and therefore always have a testable case) while remaining optional for process/informational QEPs?If there's support, I'll draft the QEP-1 amendment (version bump per QEP-1's own rules) as a PR.