Skip to content

docs(verify): clarify Format 2 k-field rule and common failure modes#20

Merged
bensonwong merged 2 commits intomainfrom
docs/verify-skill-format2-k-field
Apr 12, 2026
Merged

docs(verify): clarify Format 2 k-field rule and common failure modes#20
bensonwong merged 2 commits intomainfrom
docs/verify-skill-format2-k-field

Conversation

@bensonwong
Copy link
Copy Markdown
Contributor

@bensonwong bensonwong commented Apr 12, 2026

Summary

  • Adds a dedicated Format 2 trigger explanation to the citation formatting section, with three prose/source contrast examples showing when to choose Format 2 over Format 1
  • Expands the k field description in the CITATION_DATA table to explicitly state that for Format 2, k equals the tick-quoted sourceMatch — not the prose claimText
  • Adds a worked example block showing a full Format 2 body + CITATION_DATA pair, annotated to highlight the k assignment
  • Adds two new rows to the common failure modes table covering the Format 2 k confusion pattern
  • Updates the parallel-generation sub-agent prompt to include Format 2 instructions and a Format 2 CITATION_DATA example

Test plan

  • Read through the Format 2 trigger examples and verify they accurately contrast Domain A prose vs Domain B verbatim text
  • Confirm the annotated k-field example block (automatically convert) matches the body citation syntax
  • Check that the updated parallel-generation prompt correctly instructs sub-agents on both formats and the k rule
  • Verify the new failure-mode table rows use the correct before/after pattern

… modes

- Clarify that in Format 2, k = tick-quoted sourceMatch (Domain B), NOT prose claimText (Domain A)
- Add three concrete failure-mode examples to the table
- Update parallel-agent prompt to reference both Format 1 and Format 2 with examples
- Add Format 2 trigger: use it when prose phrasing would fail the CoT substring check

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@claude
Copy link
Copy Markdown

claude bot commented Apr 12, 2026

PR Review: docs(verify) — Format 2 k-field clarification

This is a documentation-only change to skills/verify/SKILL.md. Overall the PR is well-structured and addresses a real confusion pattern clearly. A few observations:


Strengths

  • Concrete, matched examples: The three Format 2 trigger examples (meals/50%, converts/automatically convert, age/age of the individual) directly mirror the Domain A vs Domain B framing used throughout the skill — easy to pattern-match against.
  • Failure mode table additions: The two new rows follow the existing table convention (trap → fix → explanation) and cover both the prose-summarizes-operational-state case and the common k=claimText mistake.
  • Consistency across sections: The change propagates through all the right places — the trigger explanation, the CITATION_DATA table, the worked example block, and the parallel-generation sub-agent prompt. No section is left stale.

Issues / Suggestions

1. "CLI drops it" is unexplained (minor, clarity)
In the new failure mode row:

**fully deductible** … "fully deductible" isn't in f, so CLI drops it

"CLI" hasn't been defined in this document and isn't obvious to a reader unfamiliar with the verification pipeline internals. Consider replacing with something self-contained, e.g. "so the citation validator rejects it" or "so the CoT substring check fails".

2. Annotated code block alignment is fragile (minor, maintainability)
The ^^^... underline in the worked example:

"k": "automatically convert", …
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^

The pointer is aligned for a fixed-width display. If the JSON key/value order or spacing ever changes, the annotation silently mis-points. A prose comment directly below the block (like you already added) is sufficient — the ^^^ line adds noise without adding safety. Could remove the ^^^ line and rely solely on the trailing prose annotation.

3. k-field table cell is getting long (minor, readability)
The k row now contains two full sentences differentiating Format 1 vs Format 2 inside a pipe-delimited table cell. This renders fine in GitHub but becomes hard to scan when the table is the primary quick-reference. Consider keeping the table cell to the key constraint (must be a substring of f) and moving the Format 1/Format 2 distinction to the dedicated section below the table (where the worked example already lives).

4. Parallel-generation prompt: example placement (minor)
The Format 2 example in the updated sub-agent prompt (the {"n": 2, …} JSON) now appears inline in an already-long instruction paragraph. The Format 1 example was similarly inline. Since this is a prompt passed verbatim to sub-agents, verbosity isn't a problem per se, but the "note k is the tick-quoted sourceMatch, not 'converts automatically'" trailing annotation duplicates the same wording from elsewhere. Consider whether it's load-bearing enough to keep (i.e., does the sub-agent need the reinforcement here specifically?) or whether a single clear rule earlier in the paragraph is sufficient.


No blocking issues

All changes are in documentation/prompt text — there is no executable code, no security surface, and no test coverage gap to address. The substantive guidance is correct and consistent. The issues above are all minor clarity/maintainability nits; none should block the merge.

…tring check

- Remove fragile ^^^ underline from Format 2 worked example; prose annotation below is sufficient
- Replace "CLI drops it" with "CoT substring check fails" (self-contained, no unexplained jargon)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@bensonwong bensonwong merged commit 433236f into main Apr 12, 2026
@bensonwong bensonwong deleted the docs/verify-skill-format2-k-field branch April 12, 2026 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant