Skip to content

fix(metrics): recognize approval_requested/replanned as honest-red outcome tokens - #44

Merged
SollanSystems merged 1 commit into
mainfrom
fix/issue-36-outcome-vocab
Jul 10, 2026
Merged

SollanSystems merged 1 commit into
mainfrom
fix/issue-36-outcome-vocab

Conversation

@SollanSystems

Copy link
Copy Markdown
Owner

Closes #36.

Adds approval_requested and replanned to _HONEST_RED_OUTCOME_TOKENS in scripts/metrics.py — they are known, non-success outcomes emitted by the sanctioned writer (loop/emit.py _ITERATION_OUTCOMES), so a RUNLOG written entirely through emit.append_iteration no longer surfaces them under provenance.unrecognized_outcomes. "replan" is retained for prose-token variants (it never matched "replanned" — different string).

Round-trip regression test added to scripts/test_metrics.py: iterates the live emit._ITERATION_OUTCOMES tuple (not a hardcoded copy, so future vocabulary drift re-triggers it), appends one iteration per token, asserts provenance.unrecognized_outcomes == []. The test unlinks the scaffolded RUNLOG first so it does not depend on the #40 fix landing.

Verification (independent adversarial pass):

  • Negative control: the new test overlaid on an unfixed main archive fails with AssertionError: ['approval_requested', 'replanned'] == [] — the test bites.
  • The honest-red set feeds only the provenance surface: FCR derives from the unchanged completion/progress token sets, RP from repair records — zero FCR/RP numbers change anywhere (docs/metrics-baseline.json untouched).
  • Gates: targeted pair 62 passed; full suite 394 passed / 14 skipped (engine-extra env guards run in CI); self_eval 13/13.

Copilot AI review requested due to automatic review settings July 10, 2026 03:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@SollanSystems
SollanSystems merged commit 40e039f into main Jul 10, 2026
7 checks passed
@SollanSystems
SollanSystems deleted the fix/issue-36-outcome-vocab branch July 10, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reconcile emit's iteration-outcome vocabulary with metrics' recognized tokens

2 participants