docs: prepare M01 learner-validation evidence gate - #3
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 261f260d21
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Prototype transfer rate: | ||
|
|
||
| ```text | ||
| credible or policy-defined completed field applications |
There was a problem hiding this comment.
When a participant submits the field application before observing an outcome, the session template explicitly permits Evidence actually observed: not yet, but this formula can still count it as a “policy-defined completed” application. That conflicts with the L2 definition in docs/product/METRICS.md, which requires a real-project application with evidence, and can inflate the transfer rate used for the Phase 1 decision. Restrict the numerator to applications with credible observed evidence.
Useful? React with 👍 / 👎.
|
|
||
| Use only when: | ||
|
|
||
| - learner evidence indicates meaningful reasoning improvement or useful discrimination; |
There was a problem hiding this comment.
When cases distinguish stronger from weaker learners but the cohort shows flat or negative pre/post results, “useful discrimination” alone satisfies this learning criterion and can authorize Phase 2. The roadmap's Phase 1 exit gate requires the module to reveal and improve reasoning, while the existing protocol uses discrimination only to decide whether an individual interaction is useful. Keep discrimination as drill evidence, but require evidence of improvement for the module-level promotion decision.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8d79022276
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| **Keep the five dimensions stable for the first learner cohort.** | ||
|
|
||
| Do not tune weights or thresholds between participants unless a question is clearly invalid. Changing rubric semantics mid-cohort would make pre/post and participant comparisons harder to interpret. |
There was a problem hiding this comment.
When a question is found invalid after one or more participants have completed it, this exception permits changing weights or thresholds for later sessions while the cohort template still compares all sessions as one dataset. That violates the stable-rubric requirement in docs/product/METRICS.md and can turn the reported pre/post deltas into measurements with different semantics. Freeze the rubric for the cohort, or require restarting/re-scoring every affected session under a new version.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ae769d5832
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| For each completed learner session, retain only the evidence needed for the Phase 1 decision: | ||
|
|
||
| - anonymized participant ID and role/experience; | ||
| - baseline total, post total, delta, and improved dimensions; |
There was a problem hiding this comment.
When the facilitator resets the browser for the next participant, bindReset() deletes the only stored baseline/post answer IDs, but this packet retains only aggregate scores and improved dimensions. The required cohort rubric review asks reviewers to detect option-score mismatches, ambiguity, and false classifications; without the selected responses, those analyses and any later re-scoring are impossible. Retain the anonymized baseline/post answer IDs alongside the scores before resetting.
Useful? React with 👍 / 👎.
| @@ -0,0 +1,231 @@ | |||
| # M01 Cohort Review Template | |||
|
|
|||
| Use this document after the first learner cohort is complete. The first review should normally include at least **5 completed sessions**. | |||
There was a problem hiding this comment.
When a cohort has fewer than five completed sessions, “normally” still permits reviewers to complete this template and select A, whose authorization section then allows Phase 2. This contradicts the mandatory minimum in the validation protocol and docs/ROADMAP.md, so the document designated as the explicit exit-decision record can bypass the Phase 1 evidence gate. Make five completed sessions a hard prerequisite to recording a promotion decision.
Useful? React with 👍 / 👎.
GitHub Actions
Failure
GitHub Actions
Failure
GitHub Actions
Inactive
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f7a749e85c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| on: | ||
| push: | ||
| branches: [main] | ||
| branches: [main, docs/m01-validation-readiness-audit] |
There was a problem hiding this comment.
A push to docs/m01-validation-readiness-audit now runs actions/deploy-pages against the repository's configured GitHub Pages site, so it can replace the version deployed from main; naming the Actions environment m01-pages-preview does not create a separate Pages target. The shared pages concurrency group can also cancel an in-progress main deployment. Keep this workflow main-only or publish previews to a genuinely separate target.
Useful? React with 👍 / 👎.
| - **Experience level:** | ||
| - **Facilitator:** | ||
| - **Browser/device:** | ||
| - **Fresh validation state confirmed:** yes / no |
There was a problem hiding this comment.
For consecutive participants using the same browser, confirming only fresh validation state is insufficient: the documented reset removes pm01-validation-m01-v1, while isStudied() continues reading completed M01 lessons from pm01-state-v1. After the first participant, subsequent sessions therefore begin with both lessons already marked complete and can unlock the post-case after answering only the drills, contaminating the cohort comparison. Require a fresh profile or explicitly clear and verify the two legacy lesson-completion records here.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 00cce1c779
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| -H "Authorization: Bearer ${GH_TOKEN}" \ | ||
| -H 'X-GitHub-Api-Version: 2026-03-10' \ | ||
| https://api.github.com/repos/${GITHUB_REPOSITORY}/pages \ | ||
| -d '{"build_type":"legacy","source":{"branch":"gh-pages","path":"/"}}') |
There was a problem hiding this comment.
On a push to docs/m01-validation-readiness-audit, this PUT changes the repository-wide publishing source to the legacy gh-pages branch, while the inspected main workflow in .github/workflows/pages.yml still uploads an artifact with actions/deploy-pages; GitHub treats branch publishing and GitHub Actions as alternative publishing sources, so later main pushes are left mismatched with the configured source and may no longer update the site (publishing-source documentation). Fresh evidence relative to the earlier review is that the branch workflow now issues this global configuration PUT rather than invoking deploy-pages itself. Keep the source set to GitHub Actions, or explicitly restore it before relying on the main deployment workflow.
Useful? React with 👍 / 👎.
| - baseline total, post total, delta, and improved dimensions; | ||
| - baseline and post free-text diagnoses; | ||
| - whether each Decision Drill exposed or changed reasoning; | ||
| - field transfer: credible / partial / no; |
There was a problem hiding this comment.
When the cohort reviewer validates an application's credible classification, this packet preserves only the classification rather than the learner's anonymized mechanism, intervention, early signal, observed evidence, and next decision. The cohort template subsequently asks reviewers to identify common mechanisms/interventions/evidence and verify that evidence follows from diagnosis, which cannot be audited or reclassified from this label alone after browser-local responses are cleared. Retain an anonymized summary of each transfer-chain element in the external session record.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4c8a02fb23
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| assert.equal(baseApp.includes('route: "validation-m01"'), true, 'base router must recognize validation/m01'); | ||
| assert.equal(baseApp.includes('if (route === "validation-m01")'), true, 'base renderer must yield validation/m01 to the extension'); |
There was a problem hiding this comment.
Every node --test tests/*.test.js run now fails here because app.js still maps #/validation/m01 to not-found and contains neither asserted string; I reproduced the failure locally with 19 passing tests and this single failure. Update the base router/render path to yield this route to the validation extension before adding the contract assertion, otherwise the required CI check cannot pass.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a49d3e8ae8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| **Status: READY FOR LEARNER TESTING / NOT READY FOR PHASE 2.** | ||
|
|
||
| The M01 slice is coherent enough to run the first real validation cohort. Its core learning target is narrow enough to measure, its interactions test meaningful PM reasoning, and no current observation requires a new permanent content entity solely to support the baseline/post experiment. |
There was a problem hiding this comment.
When participants notice the repeated option pattern, the post-case score can improve without better reasoning: every baseline and post-case question in m01-validation-data.js orders its options by score as 0, 1, 2, 3, making the final option consistently strongest across all ten questions. Declaring the instrument ready risks spending the cohort and authorizing Phase 2 using position-driven gains; randomize or counterbalance the option order before learner sessions.
Useful? React with 👍 / 👎.
|
|
||
| function render() { | ||
| const { route, id } = parseRoute(); | ||
| if (route === "validation-m01") return; |
There was a problem hiding this comment.
When a learner enters #/validation/m01 from a scrolled page or while the mobile drawer is open, this early return skips the base router's scrollTo(0, 0) and mobile-menu reset. renderValidationRoute() only focuses #main with preventScroll: true and does not perform either cleanup, so the validation flow can open midway down the page or remain covered by the drawer; move the shared transition cleanup before this return or reproduce it in the extension.
Useful? React with 👍 / 👎.
1 participant
Goal
Close the non-human readiness work for PMO01 Phase 1 and make the real learner-evidence gate operational.
Added
Key decision
The baseline/post comparison is treated as validation harness, not automatically as permanent production content. It must not force an experimental
BaselineAssessmententity into the future v1 content model.M01 is mapped primarily to C1 System diagnosis and secondarily to C8 Intervention design. Current drill/case/field-application gaps are classified as canonical metadata enrichment, not new content-model exceptions.
Gate status
Ready for learner testing; Phase 2 remains blocked.
Before v1 domain contracts or framework selection begin:
Tracking issue: #4 —
Phase 1: run M01 learner validation cohort.Verification
feature/m01-learning-validation: documentation/readiness only; no runtime/application changes;82c4418de634ed2b824de5714bab34dec4e74ae9;Stack
Base:
feature/m01-learning-validation(PR #2). Review/merge after PR #2.