Skip to content

feat(lint): add a Carp rule, and record four Carp reader defects - #95

Merged
takeokunn merged 1 commit into
mainfrom
feat/lint-carp
Aug 3, 2026
Merged

feat(lint): add a Carp rule, and record four Carp reader defects#95
takeokunn merged 1 commit into
mainfrom
feat/lint-carp

Conversation

@takeokunn

Copy link
Copy Markdown
Collaborator

RULE_COUNT 320 → 321. This completes dialect coverage in the strict sense: every one of the ten dialects now has a rule written for it specifically. (Carp wasn't previously absentself-recursive-tail-call and the macro-hygiene rules already list it — but nothing targeted it.)

Only one rule, and that's the interesting part

The Carp compiler already rejects almost everything worth linting. Ownership, @/& misuse, move-after-use and dangling references are each walked through in docs/Memory.md, and every one ends "the memory management system detects this and reports an error". fmt specifier/argument mismatch raises macro-error at expansion (core/Format.carp:8-22). Named holes ?x generate a type error. A rule for any of them would duplicate the compiler.

carp-deprecated-thread-macro survives precisely because it is the one that builds silently:

  • core/ControlMacros.carp:27,31 declares => and ==> deprecated.
  • But deprecated (core/Macros.carp:174) expands to meta-set! and nothing else, and that key is read in exactly two places — src/Primitives.hs:355 for the REPL's (info …), and src/RenderDocs.hs:197-219 for HTML docs. No compilation path touches it.
  • Carp's own core/Binary.carp:68,77 still uses ==>, which is the proof that nothing warns.

Fixable, because => and -> are byte-identical macro bodies in Carp's stdlib — the repair is a rename, not a rewrite. The fix is withheld (finding still reported) when the file defines its own ->/-->.

Four further candidates (Array.unsafe-nth, Debug.sanitize-addresses, Debug.trace, Unsafe.*) were dropped on zero or all-deliberate corpus occurrences rather than shipped and labelled unproven. Array.unsafe-nth in particular has ~20 legitimate uses inside core/Array.carp where the index is provably in bounds — a false-positive machine.

The larger result: four Carp reader defects

Found by parsing all 248 .carp files in carp-lang/Carp. Recorded in the package README, not fixed here.

  • @ and & are not reader prefixes, though docs/LanguageGuide.md:170-173 defines them under a literal "Reader Macros" heading. @(f x) splits into a bare @ atom plus a sibling, inflating the enclosing call's arity — 1493 such atoms across 116 of 248 files (47%). Byte spans survive, so round-trips are lossless; what breaks is structure, which makes any argument-counting analysis unsound for Carp. (This is why the shipped rule keys on the head symbol alone.)
  • @"…" silently splits string literals@ glues to the next token and swallows the opening quote, so (f @"a b") yields two atoms @"a and b" with no error. 46 split atoms across 10 files that otherwise parse cleanly. Clojure handles the same input correctly.
  • Character literals unrecognizedcharacter_literal_prefix_width has arms for Scheme/Racket/Clojure/EmacsLisp, none for Carp, so \{ \} \[ \] \( \) \" are read as real delimiters.
  • #"…" Pattern literals unrecognized.

Six of 248 files fail to parse outright, four of them in core/, each attributed by repairing one defect at a time until every file parsed. Same class as the Hy, LFE and Janet gaps; a repair belongs in core/syntax.

One benign observation pinned as a test rather than assumed away: Carp's unquote is %/%@, also unrecognized, so everything inside a ` template reads as data — which suppresses findings, the safe direction.

Corpus audit

248 files, 242 parsed. 9 findings / 9 candidates: 5 clear true positives (core/Binary.carp:68,77, examples/updating.carp:15, examples/langtons_ant.carp:86,92) and 4 deliberate uses inside test/produces-output/basics.carp, which exists to exercise the threading macros.

Honest limitation: the corpus contains zero ->/--> calls, so candidates equalled findings and gave no numerator/denominator separation. That separation comes from the permanent corpus test instead, which counts candidates over all four threading macros — counting only the deprecated pair would have made "0 findings / 0 candidates" structurally unavoidable, which is the false-clean this process exists to prevent.

Also honest: no carp binary exists on this machine, so the premises could not be executed. The compiler-behaviour claims come from reading the Haskell source and docs in the cloned tree — strong, but not the same as running it.

Cost, and a self-inflicted bug

Zero invocations on files without the deprecated spelling. Doubling ratios 1.99–2.04 against a shipped control timed in the same pass.

The first design cost 8,589,447 ns/call — the quote check and the shadowing check each called root_view(). Merging them into one node_context took it to 2.2M.

Mutation testing: 10 guards, 2 chased

One was a missing test (the engine's head index is paren-only, so the dispatcher never hands the rule a bracket list — but collect/candidate_count walk the tree themselves and do; without the guard, (def ops [=> ==> ->]) reads an array literal as three calls). One is genuinely dead but defensive, and documented as such since no input can distinguish the two spellings.

A third would have been dead code had the author not first added a shadow test inside a defmodule — Carp puts definitions in modules, and the original test was top-level only.

Wiring hit the PR #88 trap again

fixable_rules_match_the_fix_engine drives one fixture per dialect, so a Fixable rule with no fixture is a --fix that silently does nothing. Added a Carp fixture rather than relaxing the assertion.

Verification

cargo build --workspace, cargo test --workspace, cargo test --test cli (3083 passed), cargo fmt --all --check, cargo clippy --all-targets --all-features -- -D warnings — all exit 0. --list-rules --preset all reports 321.

Verified through the real binary: fires on (=> …), silent on (-> …), and fix plan reports fix_count: 1.

RULE_COUNT 320 -> 321. This completes dialect coverage in the strict
sense: every one of the ten dialects now has a rule written for it
specifically. (Carp was not previously absent from the catalogue --
`self-recursive-tail-call` and the macro-hygiene rules already list it
-- but nothing targeted it.)

Only one rule, and the reason is the interesting part: **the Carp
compiler already rejects almost everything worth linting.** Ownership,
`@`/`&` misuse, move-after-use and dangling references are each walked
through in `docs/Memory.md`, and every one ends "the memory management
system detects this and reports an error". `fmt` specifier/argument
mismatch raises `macro-error` at expansion (`core/Format.carp:8-22`).
Named holes `?x` generate a type error. A rule for any of them would
duplicate the compiler.

`carp-deprecated-thread-macro` survives precisely because it is the one
that **builds silently**. `core/ControlMacros.carp:27,31` declares
`=>` and `==>` deprecated, but `deprecated` (`core/Macros.carp:174`)
expands to `meta-set!` and nothing else, and that key is read in exactly
two places -- `src/Primitives.hs:355` for the REPL's `(info ...)` and
`src/RenderDocs.hs:197-219` for HTML docs. No compilation path touches
it. Carp's own `core/Binary.carp:68,77` still uses `==>`, which is the
proof that nothing warns.

Fixable, because `=>` and `->` are byte-identical macro bodies in Carp's
stdlib -- the repair is a rename. The fix is withheld (finding still
reported) when the file defines its own `->`/`-->`.

Four candidate rules were dropped on zero or all-deliberate corpus
occurrences rather than shipped and labelled unproven.

**The batch's larger result is four Carp reader defects**, recorded in
the package README:

- `@` and `&` are not reader prefixes, though `docs/LanguageGuide.md`
  defines them under a literal "Reader Macros" heading. `@(f x)` splits
  into a bare `@` atom plus a sibling, **inflating the enclosing call's
  arity** -- 1493 such atoms across 116 of 248 files (47%). Byte spans
  survive, so round-trips are lossless; what breaks is structure, which
  makes any argument-counting analysis unsound for Carp.
- `@"..."` silently splits string literals, because `@` glues to the
  next token and swallows the opening quote. 46 split atoms across 10
  files that otherwise parse cleanly.
- Character literals are unrecognized, so `\{ \} \[ \] \( \) \"` are
  read as real delimiters.
- `#"..."` Pattern literals are unrecognized.

Six of 248 files fail to parse outright, four of them in `core/`, each
attributed by repairing one defect at a time. Same class as the Hy, LFE
and Janet gaps; a repair belongs in `core/syntax`.

Wiring hit the same trap PR #88 did: `fixable_rules_match_the_fix_engine`
drives one fixture per dialect, so a `Fixable` rule with no fixture is a
`--fix` that silently does nothing. Added a Carp fixture rather than
relaxing the assertion; `fix plan` reports fix_count 1.
@takeokunn
takeokunn merged commit 985b825 into main Aug 3, 2026
10 checks passed
@takeokunn
takeokunn deleted the feat/lint-carp branch August 3, 2026 10:34
takeokunn added a commit that referenced this pull request Aug 3, 2026
`docs/LanguageGuide.md:170-173` defines `&` and `@` under a literal
"Reader Macros" heading -- `&x ;; same as (ref x)`, `@x ;; same as
(copy x)` -- but Carp routed through `classify_legacy`, which implements
neither. So `@(f x)` split into a bare `@` atom **plus a sibling**,
inflating the enclosing call's arity in **116 of 248 files (47%)**, 1493
atoms in all. Byte spans survived, so round-trips were lossless; what
broke was structure, which makes any argument-counting analysis unsound
for Carp.

Also fixed: `@"..."` silently split string literals, because `@` glued
to the next token and swallowed the opening quote; character literals
were unrecognized, so `\{ \} \[ \] \( \) \"` were read as real
delimiters; and `#"..."` Pattern literals were unrecognized.

Three more the original brief missed, all real: **`~` (deref) is also a
reader macro** (76 glued atoms in 15 files); **`$[...]` static arrays**
(55 bare `$` atoms, the same arity-inflating defect); and **`,` is
whitespace in Carp, not unquote** (39 sites, where the legacy reader
gave `max` and `val` phantom `Unquote` prefixes in `[min, max, val]`).

`src/Parsing.hs` settles the shape: `readerMacro` is `string macroStr`
then `expr <- sexpr`, recursing into `sexpr` rather than `atom`, so
prefixes bind any form, `@@x` stacks, and `@"..."` is a copy of a
genuine string literal. `validCharacters` excludes `& ~ @ % $ #`, so a
sigil can never be inside a symbol. `aChar` is `\` then one character
with no terminator, so `\{` and `\ ` are valid.

Corpus differential over all 248 files: parse failures **6 -> 0**, 171
trees changed, **1623 child slots removed** and reconciled
token-by-token, **0 files gained atoms**.

`%`/`%@` (Carp's real unquote) is deliberately deferred: it needs two
new `ReaderPrefix` variants threaded through 40+ sites and *adds*
findings inside macro templates, so it wants its own FP audit. Leaving
it keeps templates inert, which is the suppressing direction. `r"..."`
has zero corpus occurrences.

`Ref`/`Copy`/`Deref`/`StaticArray` are new `ReaderPrefix` variants
rather than reuses of `Function`, whose `as_source()` is `#'` and which
feeds three real re-emission paths -- reusing it would have been latent
corruption in place of the old kind.

No `carp` binary was obtainable (removed from nixpkgs 2026-02-05 as
broken; no ghc/stack/cabal), so the semantics are **grammar-derived from
`src/Parsing.hs`, not oracle-checked** -- 20 hand-verified cases. Said
plainly rather than implied.

Rebased across #96, #98, #100 and #102. The `classify_reader_macro`
dispatch table was resolved arm by arm: ours had `Unknown | Hy | Carp`
with `Lfe` pulled out by #98, theirs had `Unknown | Lfe | Hy` with
`Carp` pulled out -- each side's group line still named the other's
dialect, so taking either would have silently deleted a dialect's reader
support. A pristine `origin/main` control binary was built and diffed
over 36 repo files x 10 dialects plus a 28-case battery: **444
comparisons, 0 differences**. Nothing outside Carp changed.

Three things in `lint-carp-idiom` (#95) needed updating, because the
reader change was authored before that package existed and it documents
these defects as *current behaviour*. One was a real test failure:
`the_correct_corpus_exercises_the_readers_arity_inflation` pinned the
bug with `assert!(bare > 0)`. Inverted rather than deleted -- it now
asserts `bare == 0` *and* `prefixed_lists == 8`, keeping the
corpus-coverage pin it existed for, with the 8 derived from the corpus
text rather than read off the parser.

`edit format` on `@(unsafe-nth world i)` is confirmed fixed by #100 --
that was the original agent's top-ranked follow-up, and #100's
`carries_reader_prefix` fix is dialect-agnostic, so Carp's new variants
were covered automatically.
takeokunn added a commit that referenced this pull request Aug 3, 2026
`docs/LanguageGuide.md:170-173` defines `&` and `@` under a literal
"Reader Macros" heading -- `&x ;; same as (ref x)`, `@x ;; same as
(copy x)` -- but Carp routed through `classify_legacy`, which implements
neither. So `@(f x)` split into a bare `@` atom **plus a sibling**,
inflating the enclosing call's arity in **116 of 248 files (47%)**, 1493
atoms in all. Byte spans survived, so round-trips were lossless; what
broke was structure, which makes any argument-counting analysis unsound
for Carp.

Also fixed: `@"..."` silently split string literals, because `@` glued
to the next token and swallowed the opening quote; character literals
were unrecognized, so `\{ \} \[ \] \( \) \"` were read as real
delimiters; and `#"..."` Pattern literals were unrecognized.

Three more the original brief missed, all real: **`~` (deref) is also a
reader macro** (76 glued atoms in 15 files); **`$[...]` static arrays**
(55 bare `$` atoms, the same arity-inflating defect); and **`,` is
whitespace in Carp, not unquote** (39 sites, where the legacy reader
gave `max` and `val` phantom `Unquote` prefixes in `[min, max, val]`).

`src/Parsing.hs` settles the shape: `readerMacro` is `string macroStr`
then `expr <- sexpr`, recursing into `sexpr` rather than `atom`, so
prefixes bind any form, `@@x` stacks, and `@"..."` is a copy of a
genuine string literal. `validCharacters` excludes `& ~ @ % $ #`, so a
sigil can never be inside a symbol. `aChar` is `\` then one character
with no terminator, so `\{` and `\ ` are valid.

Corpus differential over all 248 files: parse failures **6 -> 0**, 171
trees changed, **1623 child slots removed** and reconciled
token-by-token, **0 files gained atoms**.

`%`/`%@` (Carp's real unquote) is deliberately deferred: it needs two
new `ReaderPrefix` variants threaded through 40+ sites and *adds*
findings inside macro templates, so it wants its own FP audit. Leaving
it keeps templates inert, which is the suppressing direction. `r"..."`
has zero corpus occurrences.

`Ref`/`Copy`/`Deref`/`StaticArray` are new `ReaderPrefix` variants
rather than reuses of `Function`, whose `as_source()` is `#'` and which
feeds three real re-emission paths -- reusing it would have been latent
corruption in place of the old kind.

No `carp` binary was obtainable (removed from nixpkgs 2026-02-05 as
broken; no ghc/stack/cabal), so the semantics are **grammar-derived from
`src/Parsing.hs`, not oracle-checked** -- 20 hand-verified cases. Said
plainly rather than implied.

Rebased across #96, #98, #100 and #102. The `classify_reader_macro`
dispatch table was resolved arm by arm: ours had `Unknown | Hy | Carp`
with `Lfe` pulled out by #98, theirs had `Unknown | Lfe | Hy` with
`Carp` pulled out -- each side's group line still named the other's
dialect, so taking either would have silently deleted a dialect's reader
support. A pristine `origin/main` control binary was built and diffed
over 36 repo files x 10 dialects plus a 28-case battery: **444
comparisons, 0 differences**. Nothing outside Carp changed.

Three things in `lint-carp-idiom` (#95) needed updating, because the
reader change was authored before that package existed and it documents
these defects as *current behaviour*. One was a real test failure:
`the_correct_corpus_exercises_the_readers_arity_inflation` pinned the
bug with `assert!(bare > 0)`. Inverted rather than deleted -- it now
asserts `bare == 0` *and* `prefixed_lists == 8`, keeping the
corpus-coverage pin it existed for, with the 8 derived from the corpus
text rather than read off the parser.

`edit format` on `@(unsafe-nth world i)` is confirmed fixed by #100 --
that was the original agent's top-ranked follow-up, and #100's
`carries_reader_prefix` fix is dialect-agnostic, so Carp's new variants
were covered automatically.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant