fix: strengthen Research Pack validator — add Stop condition check, strip code blocks, detect empty sections#79
Merged
Conversation
added 4 commits
May 11, 2026 14:26
…trip code blocks, detect empty sections (#66) - Added "## Stop condition" to REQUIRED_HEADINGS (was missing despite schema requiring it) - Strip fenced code blocks (```, ~~~) before heading detection, so headings inside code blocks no longer falsely satisfy the check - Detect empty required sections (heading with no non-whitespace content below it) and reject - Added CI regression tests for all bad-sample scenarios
…ection, fence parser, empty section check (#66) P1: Heading detection now uses exact H2 parsing (^## title$) instead of substring containment. Rejects H3, blockquote, leading space, partial title matches. Empty section bounds use any heading as boundary and exclude sub-heading lines from body content. P2: Fenced code block stripping uses line-by-line state machine. Handles up-to-3-space indent, proper closing fence (same char, >= length, trailing whitespace only). Inline backticks no longer terminate early. CI regression tests use isolated single-mutation samples (valid baseline + one mutation per test) instead of ad-hoc partial packs.
…op condition to schema example (#66) - CI YAML: replaced broken inline multi-line python3 -c with scripts/test_validator_regression.py (the inline code broke YAML block scalar parsing because Python code started at column 0) - schemas/research-pack.md: added missing ## Stop condition to minimal example shape so it matches the schema's required sections
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Strengthen
scripts/validate_research_pack.pyto catch structural issues that the previous simple string-contains check missed, reducing false confidence in Research Pack validation.Changes
P1: Proper heading detection
h in cleanedsubstring match — falsely accepted### Stop condition,> ## Stop condition,## Stop condition details^## title$regex — only real H2 headings with exact title text countP2: Robust fenced code block stripping
^[ ]{0,3}(+ ```+|~~~`+`)`, closing requires same char, >= length of opening, trailing whitespace onlyAdded
## Stop conditiontoREQUIRED_HEADINGS(was missing despite schema)scripts/test_validator_regression.py— dedicated 8-test regression suite covering all edge casesschemas/research-pack.mdminimal example now includes## Stop conditionFixed
python3 -c(broke YAML block scalar parsing when Python code was unindented) with dedicated test scriptOut of scope (by design)
checklists/source-traceability.md)Closes #66