Skip to content

feat(skill-creator, frontend-visual-qa): independent adversarial review as standing discipline + capture-instrument calibration#196

Merged
daymade merged 2 commits into
mainfrom
feat/skill-creator-independent-review-discipline
Jul 22, 2026
Merged

feat(skill-creator, frontend-visual-qa): independent adversarial review as standing discipline + capture-instrument calibration#196
daymade merged 2 commits into
mainfrom
feat/skill-creator-independent-review-discipline

Conversation

@daymade

@daymade daymade commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Opening a PR for two commits that had been sitting on a branch with no PR — the work is finished and valuable, it just had no path to main. Both commits are unmodified from the authoring session; this PR only gives them a merge path.

Summary

skill-creator — generalize discipline #5, add discipline #6

  • Discipline Add Trail of Bits Security Skills to Related Resources #5 was scoped to corpus-distilled skills and completeness only. Generalized: nothing ships on self-review alone, because the author's judgment runs on the model that produced the artifact and shares its blind spots. Grounded in measured results rather than folklore (the self-correction blind spot; Huang et al., ICLR 2024 on intrinsic self-correction; the information-theoretic point that correlated generator/evaluator error modes make iterative self-critique raise confidence without adding information).
  • Keeps the corpus case verbatim as a named instance; adds the fork-is-not-independent rule, the reader-spec requirement (state the target reader before the run, never after, or the reviewer measures against itself and buries real findings in noise), and re-review with a new agent after substantive edits.
  • New discipline docs(skill-creator): Add comprehensive YAML frontmatter reference with context: fork #6: design checks so they cannot self-certify green. Verification must cover every clause of its rule; each item must be a falsifiable observation rather than a self-assessment; and borrow settled distinctions from checklist practice (READ-DO vs DO-CONFIRM, pause points, killer items) and poka-yoke (prefer control over warning, since a prose reminder depends on vigilance).

frontend-visual-qa — calibrate the capture instrument before trusting it

  • Step 4 covered using DOM geometry to explain a visible problem, but not the opposite direction: a capture can manufacture a defect that isn't in the product. Before filing "content is clipped / overflowing", falsify it against DOM geometry — if scrollWidth == clientWidth and the suspect element's right edge sits inside the viewport, the clipping lives in the capture, not the CSS.
  • Pin --force-device-scale-factor=1, capture only after fonts and animation settle, and treat a false pass as the worse failure: headless follows standardized software rendering paths while headed uses host GPU and OS font hinting, so font substitution, subtle layout shift, z-index and animation defects are exactly the class headless hides.
  • One routing line added to Scope: this skill audits whether an artifact renders correctly, not whether it communicates.

Test plan

  • Content verified present and unmodified from the authoring session (git cherry confirms neither commit's patch is in main)
  • git diff main...HEAD limited to the two intended SKILL.md files (+50 / -2)
  • Author review — this is finished work from a parallel session; flagging for a look before merge rather than landing it unilaterally

⚠️ Overlap note: daymade-skill/skill-creator/SKILL.md is also touched by the in-flight PR #195. Whichever lands second will need a rebase — worth sequencing deliberately rather than letting one of them go stale.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kr1NvXALrkB86wYD9FmAFT

daymade and others added 2 commits July 22, 2026 02:30
…iscipline

Discipline #5 was scoped to corpus-distilled skills and completeness only.
Generalize it: nothing ships on self-review alone, because the author's
judgment runs on the model that produced the artifact and shares its blind
spots. Grounded in the measured result rather than folklore — the
self-correction blind spot, Huang et al. (ICLR 2024) on intrinsic
self-correction, and the information-theoretic point that correlated
generator/evaluator error modes make iterative self-critique raise confidence
without adding information. Keeps the corpus case verbatim as a named
instance, adds the fork-is-not-independent rule, the reader-spec requirement
(state who the target reader is before the run, never after, or the reviewer
measures against itself and buries real findings in noise), and re-review
with a new agent after substantive edits.

Add discipline #6: design the checks you write so they cannot self-certify
green. A check the executing context can pass while violating its own rule
manufactures confidence. Three rules — verification must cover every clause
of its rule; each item must be a falsifiable observation rather than a
self-assessment; and borrow the settled distinctions from checklist practice
(READ-DO vs DO-CONFIRM, pause points, killer items) and poka-yoke (prefer
control over warning, since a prose reminder depends on vigilance).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0145dhbApaujC3iLiJ2rgYbh
…sting it

Step 4 told you to use DOM geometry to *explain* a visible problem. It never
covered the opposite direction: a capture can manufacture a defect that is not
in the product. A frame taken at a device scale factor that differs from the
viewport it claims to show will crop or letterbox, and pixel evidence is
already noisy from anti-aliasing, sub-pixel positioning and font smoothing. So
before filing "content is clipped / overflowing", falsify it against DOM
geometry — if scrollWidth equals clientWidth and the suspect element's right
edge sits inside the viewport, the clipping lives in the capture. Report it the
other way and you send someone to fix CSS that was already correct.

Also pin --force-device-scale-factor=1, capture only after fonts and animation
settle, and treat a false pass as the worse failure: headless follows
standardized software rendering paths while headed uses host GPU and OS font
hinting, so font substitution, subtle layout shift, z-index and animation
defects are exactly the class headless hides. A green check from a thumbnailer
or preview renderer whose layout engine differs from the target application
manufactures confidence and is worse than running no check.

Add one routing line to Scope: this skill audits whether an artifact renders
correctly, not whether it communicates. A page can pass every check here while
its labels are opaque abbreviations and its numbers contradict each other
across sections — that question is answered by a reader, not by a rendering.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0145dhbApaujC3iLiJ2rgYbh
@daymade
daymade merged commit 132c885 into main Jul 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant