chg-34: GC-027 expects PRE_FLIGHT_ESCALATE — branch 6's trade-off, accepted and recorded - #86
Open
drdgreed wants to merge 3 commits into
Open
chg-34: GC-027 expects PRE_FLIGHT_ESCALATE — branch 6's trade-off, accepted and recorded#86drdgreed wants to merge 3 commits into
drdgreed wants to merge 3 commits into
Conversation
…recorded Third case (after GC-036 and GC-034) asserting an outcome a pre-flight boundary forbids. Unlike those two, this one is NOT a case-authoring error, and the distinction is the whole point of the change. On the merits GC-027 is a denial: ACC/AHA and Choosing Wisely both say non-invasive testing first, invasive cath not first-line. Aligned, unambiguous. Branch 6 fires anyway, because its heuristic infers conflict from approval AND rejection language co-occurring -- and well-written guidance is structurally "do X, don't do Y". Narrowing the check was investigated and rejected (David, option A, 2026-08-03). A genuine two-source conflict uses identical wording -- "NCCN: recommended ... CMS: not recommended" -- so dropping those markers would blind Branch 6 to real conflicts, trading the acceptable failure direction for the unacceptable one. The discriminator is what the polarities refer to: semantic, not lexical. No keyword edit can express it; structured per-source retrieval can. So the case now pins the CURRENT contract and says plainly that it documents an accepted false positive rather than a clinically correct escalation. The cost is recorded rather than buried: while this holds, guideline-based DENIED is unreachable whenever guidance carries both polarities, which sits in tension with PACCA making coverage determinations at all. must_include re-points at the machine-generated pre-flight rationale. judge_scoring_criteria previously said "Penalize ... for IN_REVIEW (the guidelines are unambiguous; no human review needed)" -- true clinically, and it would have scored the now-correct outcome down. must_not_include is UNCHANGED and was checked first: neither "approved" nor "high risk" appears in the pre-flight rationale, so there was nothing to resolve. chg-33 taught that lesson by nearly deleting a backfill for a conflict that did not exist.
Docstring only — no behaviour change, no logic touched. The existing note said the heuristic is coarse and that false positives are acceptable while false negatives are not. That posture is still right; what it did not anticipate is that the false positives are not randomly distributed. Good clinical guidance is structurally "do X, don't do Y", so the check fires hardest where guidance is CLEAREST, and the consequence is systemic rather than noisy: guideline-based DENIED becomes unreachable because the pre-flight escalates before the DecisionAgent is called. Records the rejected fix and WHY, so it is not re-proposed: the obvious marker split fails because a genuine two-source conflict is worded identically to an aligned one, and dropping "not recommended" would trade the acceptable failure direction for the unacceptable one. Names the real fix as structural -- per-source recommendation polarity from retrieval -- rather than leaving "a production system would use NLP" as the only forward pointer. Also records what "contraindicated" actually detects. In GC-085 the guideline reads "cetuximab + radiation is appropriate ... when cisplatin is contraindicated" -- a contraindication to the ALTERNATIVE, i.e. the reason the requested therapy qualifies. The marker matches the word, not the clinical fact; a real check would read the patient record. Kept because it contributes only when paired with a separate approval marker, which no case does, so it costs nothing today. This lives on the function rather than only in a manifest because the next person to look at Branch 6 will read the docstring, not iter-24.json.
The change that resolves branch 6 by NOT changing branch 6, so the reasoning has to carry the weight the code normally would. risk_cases leads with the uncomfortable part: this encodes a known-wrong outcome as expected. GC-027 is clinically a denial and the dataset now says escalate. Deliberate -- the alternative is a case that can never pass -- but the golden set no longer asserts the clinically correct answer for it, and a reader who skips the rationale would take the expectation at face value. It also states the accumulated cost plainly. With GC-027 and GC-034 both now expecting escalation, two of the three deterministic routes to a denial are closed, leaving benefit-cap exhaustion as effectively the only reachable denial class. That is in direct tension with the 2026-07-31 decision that PACCA makes coverage determinations, and it is the strongest argument yet for the structural fix (per-source recommendation polarity from retrieval) rather than another dataset correction. One limit recorded honestly: the marker split was rejected on a single decisive worked example. It genuinely is decisive -- the split provably breaks real conflict detection -- but no sweep quantified how many real conflicts are worded that way versus explicitly. The conclusion is sound; the magnitude is not measured.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Option A, per your call
This resolves the branch-6 question by not changing branch 6.
Why the marker split was rejected
I recommended splitting
GUIDELINE_CONFLICT_MARKERS— keepconflicting evidence/guideline discordant, dropnot recommended/not supported/avoid. That recommendation was wrong, and testing it against the existing genuine-conflict case before writing code is what caught it:A real two-source conflict is worded identically to an aligned guideline. Under the split it matches zero markers and fires nothing. That trades the acceptable failure direction for the one the docstring calls unacceptable.
The discriminator is what the polarities refer to — semantic, not lexical. No keyword edit expresses it.
What changed instead
PRE_FLIGHT_ESCALATE/BRANCH_6_CONFLICTING, pinning the current contract.must_includere-points at the pre-flight rationale;judge_scoring_criteriapreviously said "Penalize … for IN_REVIEW (the guidelines are unambiguous)" — clinically true, and it would have scored the now-correct outcome down._check_conflicting_guidelines's docstring records the trade-off, why the split fails, and names the real fix: per-source recommendation polarity from retrieval. Docstring only — no logic touched.must_not_includeis unchanged, and was checked first. chg-33 taught that lesson by nearly deleting a backfill to resolve a conflict that didn't exist.The uncomfortable part, stated plainly
This encodes a known-wrong outcome as expected. GC-027 is clinically a denial; the dataset now says escalate. Deliberate — the alternative is a case that can never pass — but the golden set no longer asserts the clinically correct answer here.
And the cost is accumulating: with GC-027 and GC-034 both now expecting escalation, two of the three deterministic routes to a denial are closed, leaving benefit-cap exhaustion as effectively the only reachable denial class. That's in direct tension with the 2026-07-31 decision that PACCA makes coverage determinations — and it's the strongest argument yet for the structural fix rather than another dataset correction.
On
contraindicatedYou asked to keep contraindication escalating. It stays — but it detects the word, not the fact. GC-085's guideline reads "cetuximab + radiation is appropriate … when cisplatin is contraindicated" — a contraindication to the alternative, i.e. the reason the requested therapy qualifies. It costs nothing today (it only contributes when paired with a separate approval marker, which no case does), so it's inert rather than correct. A real check would read the patient record.
Verification
make test-all→ 1043 passed, 5 skippedLimit recorded: the split was rejected on one decisive worked example. It genuinely is decisive, but no sweep quantified how many real conflicts are worded that way versus explicitly. Conclusion sound; magnitude unmeasured.
🤖 Generated with Claude Code
https://claude.ai/code/session_019UrGtq26sBHzoQZkm9M27A