Skip to content

fix: skilllint rules command surfaces only 3 of 12 documented rule series #38

@Jamie-BitFlight

Description

@Jamie-BitFlight

Story

As a developer relying on this plugin, I want to skilllint rules command surfaces only 3 of 12 documented rule series so that the tool works correctly and reliably.

Description

Running uv run skilllint rules returns output for AS001–AS009, FM001–FM010, and PA001 only (22 rules across 3 series). The README "What gets validated" table documents SK, LK, PD, PL, HK, NR, SL, TC, PR, CU, CX series — none appeared in the command output.

Observed: uv run skilllint rules on 2026-04-11 returned a table containing only AS, FM, and PA series. The README documents 12 rule series total. SK001–SK009, LK001–LK002, PD001–PD003, PL001–PL006, HK001–HK005, NR001–NR002, SL001, TC001, PR001–PR005, CU001–CU002, CX001–CX002 were all absent from the command output.

Impact: Users running skilllint rules to discover what the tool validates see a fraction of the tool's actual validation coverage. The missing series are not discoverable through the CLI.

User-provided context: "bigger issue where there's unfinished work in the migration from a monolithic app in the plugin validator to the modular system with per rule parsing and help menus — that's a bigger set of tasks that needs some planning around first"

Acceptance Criteria

  • Work matches description
  • Plan or implementation complete

Context

  • Source: Session observation — uv run skilllint rules output reviewed 2026-04-11; user confirmation of incomplete migration
  • Priority: P1
  • Added: 2026-04-11
  • Research questions: None

RT-ICA

RT-ICA Final: fix: skilllint rules command surfaces only 3 of 12 documented rule series
Date: 2026-04-12
Goal: Fix skilllint rules CLI to surface all documented rule series, not just AS, FM, and PA.
Conditions:

  1. Root cause location: only as_series.py, fm_series.py, pa_series.py exist in rules/ | Snapshot: AVAILABLE → Final: AVAILABLE | Citation: Glob packages/skilllint/rules/*.py returned 3 files
  2. Missing series validators in plugin_validator.py | Snapshot: DERIVABLE → Final: AVAILABLE | Citation: fact-checker VERIFIED — all 14 series defined in plugin_validator.py; 11 unmigrated
  3. How CLI rules command imports rule modules: rules/init.py imports only fm_series; as/pa load via direct imports in plugin_validator.py | Snapshot: DERIVABLE → Final: AVAILABLE | Citation: impact-analyst read rules/init.py directly
  4. Exact rule IDs for each missing series | Snapshot: DERIVABLE → Final: AVAILABLE | Citation: fact-checker verified against plugin_validator.py — SK001-SK009, LK001-LK002, PD001-PD003, PL001-PL006, HK001-HK005, NR001-NR002, SL001, TC001, PR001-PR005, CU001-CU002, CX001-CX002
  5. Scope of migration: extract-from-monolith (validators exist in plugin_validator.py) | Snapshot: DERIVABLE → Final: AVAILABLE | Citation: fact-checker VERIFIED — validators exist in plugin_validator.py
  6. Test coverage gaps | Snapshot: DERIVABLE → Final: AVAILABLE | Citation: impact-analyst — test_cli.py asserts exit 0 only; test_provider_validation.py scoped to AS only
  7. CU/CX adapter registration mechanism | Snapshot: DERIVABLE → Final: AVAILABLE | Citation: impact-analyst — adapters emit raw dicts; cursor/adapter.py line 93, codex/adapter.py line 125
  8. Scope boundary: plan-first per user context | Snapshot: DERIVABLE → Final: AVAILABLE | Citation: User: "bigger set of tasks that needs some planning around first"
  9. Monolith call-site migration strategy | Snapshot: DERIVABLE → Final: DERIVABLE | Citation: resolvable from reading plugin_validator.py call sites; no human decision required
    Changes from snapshot:
  • Conditions 2–8: DERIVABLE → AVAILABLE (resolved by fact-checker + impact-analyst)
  • Condition 9: DERIVABLE (no change — resolvable from codebase without human input)
  • No conditions moved to MISSING
    AVAILABLE count: 8
    DERIVABLE count: 1
    MISSING count: 0
    Decision: APPROVED

Groomed (2026-04-12)

Impact

  • Users: Developers running skilllint rules to understand validation coverage see a fraction of actual rules. No way to discover SK, LK, PD, PL, HK, NR, SL, TC, PR, CU, or CX rules through the CLI.
  • Agents: Agent skills routing on rule series (e.g. plugins/agentskills-skilllint/skills/skilllint/SKILL.md) are operating from incomplete information.
  • Documentation drift: README documents 12 series; CLI surfaces 3. The gap grows as new rules are added to plugin_validator.py without migration.
  • Test coverage: tests/test_cli.py only asserts exit 0 for rules command — no series completeness assertion; regressions are invisible.
  • CU/CX architectural gap: Cursor and Codex adapter rules are emitted as raw dicts, bypassing @skilllint_rule. These series cannot be fixed by module creation alone.
  • CM001: Cursor Memory rule exists in plugin_validator.py but is unmigrated and absent from both CLI and README documentation.
  • Scope: 13 files/systems affected (see Systems Inventory in Impact Radius section).

Code — Producers (write the changed interface)

  • packages/skilllint/rules/__init__.py — currently only imports fm_series; must be updated to import all new series modules so RULE_REGISTRY is populated at package import time
  • packages/skilllint/plugin_validator.py — contains all unextracted rule logic, ErrorCode enum, alias block, rules_cmd, rule_cmd; source of truth for migrating 11 series
  • packages/skilllint/rules/sk_series.py — NEW: SK001–SK009 (does not exist)
  • packages/skilllint/rules/lk_series.py — NEW: LK001–LK002 (does not exist)
  • packages/skilllint/rules/pd_series.py — NEW: PD001–PD003 (does not exist)
  • packages/skilllint/rules/pl_series.py — NEW: PL001–PL006 (does not exist)
  • packages/skilllint/rules/hk_series.py — NEW: HK001–HK005 (does not exist)
  • packages/skilllint/rules/nr_series.py — NEW: NR001–NR002 (does not exist)
  • packages/skilllint/rules/sl_series.py — NEW: SL001 (does not exist)
  • packages/skilllint/rules/tc_series.py — NEW: TC001 (does not exist)
  • packages/skilllint/rules/pr_series.py — NEW: PR001–PR005 (does not exist)
  • packages/skilllint/rules/cu_series.py — NEW: CU001–CU002; NOTE: cursor adapter currently emits these as raw dicts (architectural gap — see below)
  • packages/skilllint/rules/cx_series.py — NEW: CX001–CX002; NOTE: codex adapter currently emits these as raw dicts (architectural gap — see below)

Code — Consumers (read the changed interface)

  • packages/skilllint/adapters/cursor/adapter.py — emits CU001/CU002 as raw Python dicts bypassing @skilllint_rule decorator; must be changed to use decorated functions or registry registration
  • packages/skilllint/adapters/codex/adapter.py — emits CX001/CX002 as raw dicts bypassing @skilllint_rule; same fix required as cursor adapter

Code — Other References

  • packages/skilllint/rule_registry.py — registry logic is correct; no change needed, verify only
  • packages/skilllint/rules/as_series.py — no change needed, verify only
  • packages/skilllint/rules/fm_series.py — no change needed, verify only
  • packages/skilllint/rules/pa_series.py — no change needed, verify only

Documentation (will become stale)

  • README.md — "What gets validated" table documents 12 series; CU and CX are absent from README (fact-checker REFUTED claim that README documents CU/CX); table may need updating depending on scope decision for CU/CX
  • plugins/agentskills-skilllint/README.md — series table missing PR, CU, CX rows
  • plugins/agentskills-skilllint/skills/skilllint/references/rule-catalog.md — Cursor section uses wrong IDs (cursor-mdc-frontmatter/cursor-mdc-glob instead of CU001/CU002); pre-existing documentation bug

Configuration / CI

  • .github/workflows/test.yml — runs pytest but no skilllint rules output assertion; a completeness check would prevent regression

Agent Instructions (instruct AI to use current interface)

  • plugins/agentskills-skilllint/skills/skilllint/SKILL.md — routing logic and trigger description omit several series; will become stale

Systems Inventory

  1. packages/skilllint/rules/ — rule module directory (3 files today, needs 11 more)
  2. packages/skilllint/rules/__init__.py — controls which rule modules are imported at package load
  3. packages/skilllint/plugin_validator.py — monolithic validator containing all unextracted rule logic (~5600 lines)
  4. packages/skilllint/rule_registry.py — RULE_REGISTRY dict + @skilllint_rule decorator
  5. packages/skilllint/adapters/cursor/adapter.py — CU001/CU002 emitted as raw dicts
  6. packages/skilllint/adapters/codex/adapter.py — CX001/CX002 emitted as raw dicts
  7. packages/skilllint/tests/test_cli.py — only checks exit 0 for rules command; no series completeness assertion
  8. packages/skilllint/tests/test_provider_validation.py — registry checks scoped to AS only
  9. README.md — documents 12 series (missing CU/CX)
  10. plugins/agentskills-skilllint/README.md — series table incomplete
  11. plugins/agentskills-skilllint/skills/skilllint/references/rule-catalog.md — wrong CU IDs
  12. plugins/agentskills-skilllint/skills/skilllint/SKILL.md — routing omits several series
  13. .github/workflows/test.yml — no rules completeness assertion

Architectural Gap (CU/CX)

CU001/CU002 and CX001/CX002 are emitted by platform adapters as raw dicts — no @skilllint_rule registration. Creating cu_series.py/cx_series.py alone is insufficient; the adapters must also be changed to use the registry.

Ecosystem Completeness Checklist

  • Every code producer updated or verified compatible
  • Every code consumer migrated to new interface
  • Every stale document updated
  • Every agent instruction updated
  • Old interface deprecated or removed (if replacing)
  • CI/config files updated and validated

Issue Classification

Type: Defect — traceable failure with identifiable, deterministic cause chain
Analysis method: Structural tracing from CLI output → registry → module imports → filesystem
Cause chain: skilllint rules queries RULE_REGISTRY → registry is populated only at import time → rules/__init__.py imports only fm_seriesas_series and pa_series load via direct imports in plugin_validator.py → 11 series module files do not exist → those series are absent from the registry → absent from CLI output
Scenario target: Monolith-to-modular migration in progress; user confirmed this is incomplete work, not a regression from a previously working state
Scope boundary: This item documents the defect and maps the full problem space. The migration itself is a larger set of tasks that requires a SAM plan before execution.

Root-Cause Analysis

5-whys chain:

  1. Why does skilllint rules show only 3 series?
    RULE_REGISTRY contains entries only for AS, FM, and PA series at runtime.

  2. Why only 3 registry entries?
    The registry is populated by the @skilllint_rule decorator at import time. Only 3 rule module files exist under packages/skilllint/rules/: as_series.py, fm_series.py, pa_series.py. No modules exist for SK, LK, PD, PL, HK, NR, SL, TC, PR, CU, or CX.

  3. Why only 3 module files?
    The migration from the monolithic plugin_validator.py to the per-series module system was started but not completed. Only 3 of 14 series have been extracted.

  4. Why is the migration incomplete?
    The monolithic plugin_validator.py is ~5600 lines and contains all validator logic, the ErrorCode enum, alias blocks, and CLI command handlers (rules_cmd, rule_cmd). Migrating each series requires extracting logic, wiring registry decoration, and updating call sites — non-trivial scope per series.

  5. Why was it not finished?
    User confirmed: "bigger issue where there's unfinished work in the migration from a monolithic app in the plugin validator to the modular system with per rule parsing and help menus — that's a bigger set of tasks that needs some planning around first."

Secondary finding: CU001/CU002 (cursor adapter) and CX001/CX002 (codex adapter) are emitted as raw Python dicts — these series bypass @skilllint_rule registration entirely; module file creation alone is insufficient for these two series.

Additional finding: CM001 (Cursor Memory) exists in plugin_validator.py but is unmigrated and absent from both CLI and README documentation.

Reproducibility

  1. Install the package: cd /home/user/skilllint && uv sync
  2. Run: uv run skilllint rules
  3. Observe the output table.

Expected: Rows for all documented rule series — FM, AS, SK, LK, PD, PL, HK, NR, SL, TC, PR, PA, CU, CX (and CM if in scope).

Observed (as of 2026-04-11): Table contains only 20 rules across 3 series — AS001–AS009, FM001–FM010, PA001. SK, LK, PD, PL, HK, NR, SL, TC, PR, CU, CX are entirely absent.

Correction to item description: The item stated "22 rules" — the verified count is 20 (9 AS + 10 FM + 1 PA).

Priority

P1 — skilllint rules is the primary discoverability surface for the tool's validation coverage. Users and agents relying on this command see less than 22% of the actual rule count. The remaining series (SK, LK, PD, PL, HK, NR, SL, TC, PR, CU, CX) are invisible through the CLI.

Effort: High — 11 new series module files, adapter-level changes for CU/CX, init.py rewiring, test coverage, CI assertions, documentation updates across 3 files. User confirmed this requires a SAM plan before execution.

Benefits

  • skilllint rules output matches the tool's actual validation coverage — users can trust the command as a complete reference.
  • Every rule series has structured help text accessible via skilllint rule <ID>.
  • Agents can route correctly on rule series without relying on stale documentation.
  • The migration to per-series modules unblocks structured per-rule metadata (severity, fix availability, platform scope) for all series.
  • CI can assert series completeness, preventing future regressions from partial migrations.
  • The CU/CX architectural gap resolution ensures platform-adapter rules follow the same registration path as all other rules.

Expected Behavior

When skilllint rules is run after this work is complete:

  • The output table includes rows for all rule series currently implemented in the codebase: FM, AS, SK, LK, PD, PL, HK, NR, SL, TC, PR, PA, CU, CX (and CM if in scope).
  • skilllint rule <ID> returns documentation for any rule ID from any of those series.
  • The count of rules surfaced by the CLI matches the count of ErrorCode members defined in plugin_validator.py.
  • No rule that has a validator implementation is absent from the CLI output.

Acceptance Criteria

  1. uv run skilllint rules output contains rows for SK, LK, PD, PL, HK, NR, SL, TC, PR series — verified by running the command and inspecting the table.
  2. uv run skilllint rules output contains rows for CU and CX series — verified by running the command.
  3. uv run skilllint rule SK001 exits 0 and prints non-empty documentation — repeat for at least one rule from each migrated series.
  4. The total rule count shown by uv run skilllint rules equals the number of ErrorCode members defined in plugin_validator.py (excluding any explicitly scoped out of migration).
  5. packages/skilllint/rules/__init__.py imports every new series module — verified by inspecting the file.
  6. A CI check (or test assertion) exists that fails if uv run skilllint rules returns fewer series than a defined minimum — verified by confirming the test passes in CI.
  7. README.md "What gets validated" table matches the series present in uv run skilllint rules output — no series in one and absent from the other.

Files

File Role
packages/skilllint/rules/__init__.py Controls which series modules are imported at package load; currently imports only fm_series
packages/skilllint/plugin_validator.py ~5600-line monolith; source of all unextracted rule logic, ErrorCode enum, alias block, rules_cmd, rule_cmd
packages/skilllint/rule_registry.py RULE_REGISTRY dict and @skilllint_rule decorator; no change needed, verify only
packages/skilllint/rules/as_series.py Migrated reference — canonical pattern to follow for new series modules
packages/skilllint/rules/fm_series.py Migrated reference — canonical pattern to follow
packages/skilllint/rules/pa_series.py Migrated reference — canonical pattern to follow
packages/skilllint/adapters/cursor/adapter.py Emits CU001/CU002 as raw dicts bypassing @skilllint_rule
packages/skilllint/adapters/codex/adapter.py Emits CX001/CX002 as raw dicts bypassing @skilllint_rule
packages/skilllint/tests/test_cli.py Asserts exit 0 only; no series completeness check
packages/skilllint/tests/test_provider_validation.py Registry checks scoped to AS only
README.md Documents 12 series; CU/CX absent; will drift further without sync
plugins/agentskills-skilllint/skills/skilllint/references/rule-catalog.md Uses wrong IDs for CU series (cursor-mdc-frontmatter/cursor-mdc-glob instead of CU001/CU002)
.github/workflows/test.yml Runs pytest; no skilllint rules completeness assertion

Resources

Type Item
Prior work — migrated series (pattern reference) packages/skilllint/rules/as_series.py
Prior work — migrated series (pattern reference) packages/skilllint/rules/fm_series.py
Prior work — migrated series (pattern reference) packages/skilllint/rules/pa_series.py
Source of truth for unextracted validator logic packages/skilllint/plugin_validator.py
Registry and decorator implementation packages/skilllint/rule_registry.py

Dependencies

  • Depends on: None — self-contained. The migration plan this item requires is a downstream planning artifact, not a prerequisite for grooming.
  • Blocks: Any item that depends on skilllint rules completeness as a precondition (e.g. agent skill accuracy improvements, documentation generation from CLI output, per-rule CI checks).

Effort

High — 11 new series module files required, each needing logic extracted from plugin_validator.py (~5600 lines). CU/CX require additional adapter-level changes beyond module creation. rules/__init__.py, test coverage, CI assertions, and documentation (README, rule-catalog, SKILL.md) all need updating. The user has explicitly noted this requires a SAM migration plan before execution, which itself is non-trivial scoping work.

Design Intent Alignment

Alignment assessment: DIVERGENT

Design intent (from item description): skilllint rules command surfaces all documented rule series and their rules, providing complete discoverability of the tool's validation coverage.

Current implementation: skilllint rules surfaces only 3 of 12+ documented series (AS, FM, PA). 11 series present in the codebase (SK, LK, PD, PL, HK, NR, SL, TC, PR, CU, CX) are absent from CLI output. The RULE_REGISTRY is not populated for these series because the module files for them do not exist in packages/skilllint/rules/.

Divergences:

Aspect Designed / documented intent Actual implementation
Series surfaced by skilllint rules All documented series (12+ per README) AS, FM, PA only (3 series, 20 rules)
Registry population All rule series registered via @skilllint_rule Only 3 series registered; 11 unmigrated
rules/__init__.py Imports all series modules Imports only fm_series
CU/CX registration Rules accessible via registry Emitted as raw dicts by adapters, bypassing registry
CLI discoverability All rule IDs inspectable via skilllint rule <ID> Only AS/FM/PA IDs resolvable

Summary: 1 DIVERGENT aspect (5 sub-dimensions). The monolith-to-modular migration is incomplete.

Fact-Check

Claim 1: "uv run skilllint rules returns output for AS001–AS009, FM001–FM010, and PA001 only (22 rules across 3 series)"
verdict: REFUTED
evidence: Command output shows 20 rules (9 AS + 10 FM + 1 PA) — not 22. Count in item description is incorrect.
source: uv run skilllint rules run 2026-04-12

Claim 2: "The README 'What gets validated' table documents SK, LK, PD, PL, HK, NR, SL, TC, PR, CU, CX series"
verdict: REFUTED (partial)
evidence: README.md lines 196–209 show 12 documented series (FM, AS, SK, LK, PD, PL, HK, NR, SL, TC, PR, PA). CU and CX are absent from README. Item description incorrectly claims README documents CU/CX.
source: README.md:196-209

Claim 3: "SK001–SK009, LK001–LK002, PD001–PD003, PL001–PL006, HK001–HK005, NR001–NR002, SL001, TC001, PR001–PR005, CU001–CU002, CX001–CX002 were all absent from the command output"
verdict: VERIFIED
evidence: Only as_series.py, fm_series.py, pa_series.py exist in packages/skilllint/rules/. No modules for any of the listed series. Confirmed by Glob on rules directory.
source: packages/skilllint/rules/ directory (Glob output)

Claim 4: "There is unfinished work in the migration from monolithic plugin_validator.py to modular per-rule system"
verdict: VERIFIED
evidence: All 14 rule series defined in packages/skilllint/plugin_validator.py. Only 3 of 14 (AS, FM, PA) have been moved to rules/ modules. 11 series remain in monolith unmigrated. CM001 (Cursor Memory) also present in monolith, undocumented in README.
source: packages/skilllint/plugin_validator.py (Grep for ErrorCode definitions)

New findings:

  • CM001 (Cursor Memory rule) exists in plugin_validator.py — unmigrated and undocumented
  • Migration status: 3 of 14 series migrated; 11 of 14 remain unmigrated
  • CU001/CU002 and CX001/CX002 emitted as raw dicts by adapters, bypassing @skilllint_rule

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions