fix(suggest-validator): ECMAScript flavor, named groups, and error-state rules

[iteuscher]

Summary

• Align the suggest-validator skill with Paramify's ECMAScript regex engine (named groups as (?<name>…), verify matches in Node with gs flags).
• Require named capture groups and document why pinning schema_version is a silent-pass hazard.
• Add Phase 3b guidance for Error disposition rules anchored on exit_code, so failed collections are not mistaken for compliance Failures.

Test plan

• Skim .claude/skills/suggest-validator/SKILL.md Phase 3 / 3b / 4 for clarity
• Optionally run the skill against a real evidence file and confirm the suggested Node verify snippet works
• Confirm an error-state pattern captures non-zero exit_code on a failed run

Made with Cursor

[iteuscher]

And now a word from our sponsor, Claude....

Fix regex flavor, add named capture groups and error-state guidance to suggest-validator

Why

Three gaps surfaced while building error-state validators in Paramify against real envelope evidence.

1. The skill specifies the wrong regex flavor. It currently says to use "standard PCRE / Python-re syntax" and tells the user to confirm the flavor themselves. Paramify's validator engine is ECMAScript (JavaScript). Most constructs port cleanly, so this was harmless until named groups entered the picture — and there the two flavors are mutually exclusive:

Python  (?P<name>...)  → compiles in Python, hard error in JS: "Invalid group"
JS      (?<name>...)   → compiles in JS, hard error in Python: "unknown extension ?<"

A suggestion emitted in Python flavor won't compile in Paramify at all. Phase 4 also verified matches with Python re, which would reject the very syntax the skill should now be recommending.

2. No named capture group guidance. Paramify supports (?<name>...), and the validator library is being migrated to use it. Worth noting for authors: naming does not renumber anything — (?<x>…) is still group 1 — so Paramify's rule table, which references groups by number, is unaffected. That property is what makes the migration a regex-only change, and it's non-obvious enough to be worth stating.

3. No error-state guidance. Paramify validation rules carry a disposition of Pass, Fail, or Error, and rules are evaluated Error → Fail → Pass. Without an Error rule, a failed collection (expired token, network timeout) produces no payload, the regex finds nothing, and the artifact reports Fail — indistinguishable from a real compliance failure. The envelope already carries everything needed to tell those apart; the skill just never said so.

What changed

Phase 3 — flavor and naming

• Replaced the PCRE/Python guidance with ECMAScript, including that g and s are applied automatically while m is not (so ^/$ match the whole document and lines are spanned with [\s\S]*?).
• Added the named-group convention: derive from the captured key, snake_case, unique per pattern, and the "naming doesn't renumber" note.
• Updated the knowbe4 worked example and both variants to use named groups.

Phase 3b — error state (new section)

• Anchor on exit_code, not status. envelope.py sets "status": "success" if exit_code == 0 else "failed", so the two are perfectly redundant — and status additionally collides with payload-level status keys that use a different vocabulary (envelope says failed, a payload may say error). exit_code is numeric and appears exactly once per file.
• Pattern shape: error anchor first, existing compliance pattern in an optional group, so it matches exactly once whether or not the payload arrived.
• Rule table with the drift canary described below.
• Documented exit_code as the one sanctioned exception to the skill's existing envelope-key golden rule, since here the envelope is the subject of the check.

Phase 4 — verification

• Switched the verification snippet from Python re to Node, with gs flags to mirror Paramify.
• Made both-directions testing explicit: a regex that only ever matches proves nothing, so show it returning 0 against a payload that should fail, and against a failed run if an error rule was added.

Anti-patterns — added Python-flavored syntax, unnamed groups, pinning schema_version, and shipping a compliance regex with no error rule.

On the drift canary, and why not to pin schema_version

Pinning the envelope version looks like prudent version-safety but fails in the dangerous direction. A zero-match validator does not fail — a Match Group rule with nothing to read passes silently. Tested across four fixtures:

Scenario	Unpinned	Pinned to 1.0
v1 success / error	works	works
v2, exit_code unchanged	works	0 matches → silent pass
v2, exit_code renamed	0 matches → silent pass	0 matches → silent pass

• Flavor incompatibility confirmed in both directions.
• Phase 4's Node snippet was executed as written.

Notes

Docs only — no schema, framework, or fetcher changes. The skill remains read-only by design.

Related: the metadata.error field currently reports a trailing INFO log line rather than the failure reason on runs where a fetcher catches its own exception. That's tracked separately and does not affect this change — the error rule here keys off exit_code, which is correct today.