Skip to content

fix: Pre-existing rule-content drift exposed by P038 audit (SK/FM008/vendor cache) #40

@Jamie-BitFlight

Description

@Jamie-BitFlight

Story

As a developer relying on this plugin, I want to pre-existing rule-content drift exposed by p038 audit (sk/fm008/vendor cache) so that the tool works correctly and reliably.

Description

Problem

During P038 (#38) a rule-content audit + QG Phase 1 code review surfaced several pre-existing drift items in the rule metadata, sourcing, and registry that were carried forward from the monolithic plugin_validator.py into the per-series modules. Each item is a correctness or traceability concern, not a P038 regression.

Observations

SK004/SK005 orphaned architecture citations

  • packages/skilllint/rules/sk_series.py:318-321,340_MIN_DESCRIPTION_LENGTH = 20 comment cites "Architecture lines 349-350 — minimum 20 characters"
  • packages/skilllint/rules/sk_series.py:400-402,436_REQUIRED_TRIGGER_PHRASES comment cites "Architecture line 357 — trigger phrase requirements"
  • packages/skilllint/plugin_validator.py:2796, 2910 — same "Architecture" citations in the class docstrings

No ARCHITECTURE.md exists in docs/ or the repo root (only GSD template files). These constraints have no traceable source per CLAUDE.md "every threshold has an origin" policy. T14 agent kept _MIN_DESCRIPTION_LENGTH = 20 with a reasoned comment; the citation needs to either point at a real authoritative document or be removed/replaced with a sourced rationale.

FM008 / ErrorCode enum divergence

  • fm_series.py registers FM008 in RULE_REGISTRY via @skilllint_rule
  • plugin_validator.py ErrorCode enum does NOT contain FM008 (removed in commit 5a88d60 which also fixed FM008/AS008 false positives)
  • Result: RULE_REGISTRY has 58 rules, ErrorCode has 50 members → P038 acceptance criterion AC4 ("total rule count equals number of ErrorCode members excluding explicitly scoped-out series") is structurally unachievable
  • Also affects PA001, CU001, CU002, CX001, CX002 — none have corresponding ErrorCode entries either

Need a decision: re-add ErrorCode.FM008 (and add enum entries for PA/CU/CX), or remove FM008 registration from fm_series.py and document that RULE_REGISTRY is authoritative, not ErrorCode.

Vendor cache empty — authority URLs cannot be validated offline

  • .claude/vendor/ contains only CLAUDE.md (a stub)
  • No cached vendor sources exist — .claude/vendor/sources/ is not populated
  • Every @skilllint_rule(authority={"reference": URL}) across the 14 series points at URLs (anthropic.com/skills, cursor.com/rules-for-ai, github.com/openai/codex, github.com/jamie-bitflight/claude_skills/.../ERROR_CODES.md) that cannot be offline-verified
  • The schema-drift-auditor agent reported "no drift pending" because the .drift-pending.json file was absent — the SessionStart hook flag was stale relative to on-disk state

LK/PD/PL/HK/NR/SL/TC/PR authority URLs point at an internal project catalog

  • All 8 agentskills-platform series use authority.reference = https://github.com/jamie-bitflight/claude_skills/blob/main/plugins/plugin-creator/docs/ERROR_CODES.md
  • This is an internal project catalog, not a platform spec
  • For rules that encode Claude Code / hooks / plugin schema constraints (HK002's event-type list, PL006's marketplace root keys), a platform authority URL would be more authoritative

SK006/SK007 token threshold sourcing

  • packages/skilllint/rules/sk_series.py:500-501,535-537 — docstrings say "Source: token_counter.TOKEN_WARNING_THRESHOLD / TOKEN_ERROR_THRESHOLD — sourced from skilllint.token_counter module"
  • The constant location is cited but the upstream origin (Anthropic guidance, benchmark, or empirical measurement) is not
  • Constants are unsourced at their definition site in token_counter.py

User context

None — discovered during automated audits of P038 work. User confirmed during session that agentskills.io specification (https://agentskills.io/specification.md) is the authoritative source for name-length constants (64 chars), which led to the MAX_SKILL_NAME_LENGTH fix landing in commit 69e5e77. The items above are the remaining pre-existing drift.

Research questions

  • Does an authoritative "Architecture" document exist that SK004/SK005 docstrings should cite, or should the 20-char and trigger-phrase constraints be removed or resourced?
  • Should FM008 be re-added to ErrorCode or should the PR series / PA / CU / CX be treated as registry-native rules that do not require ErrorCode entries?
  • Can skilllint docs fetch be run for the 4 primary authority URLs to populate .claude/vendor/sources/ so a re-audit can verify URL validity offline?
  • For LK/PD/PL/HK/NR/SL/TC/PR, do any platform-level specs exist that would be more authoritative than the internal ERROR_CODES.md catalog?

Suggested location

Multiple: packages/skilllint/rules/{sk,fm,pa,cu,cx}_series.py, packages/skilllint/plugin_validator.py (ErrorCode enum), packages/skilllint/token_counter.py, .claude/vendor/sources/ (new cache), docs/ (potential new Architecture doc).

Scope boundary

This item catalogs the drift. Resolution likely needs its own grooming pass per series — each rule has a different research question. A single SAM plan may not be the right shape.

Acceptance Criteria

  • Work matches description
  • Plan or implementation complete

Context

  • Source: P038 rule-content audit (session 2026-04-13) — 4 MAJOR / 5 MINOR findings. Not P038 regressions, faithfully carried forward from the monolithic plugin_validator.py.
  • Priority: P2
  • Added: 2026-04-13
  • Research questions: None

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions