feat: React frontend IR — TSX routing + component semantic lowering

benchmarks/results/python-det-v2.json

@@ -5,25 +5,25 @@
       "bucket": "large(>10k)",
       "fidelity_missing": [],
       "file": "skills/kern/scripts/kern_cache.py",
-      "source_tokens": 15181,
+      "source_tokens": 15467,
       "tiers": {
         "L1": {
           "fidelity_missing": [],
           "fidelity_ok": true,
-          "ratio": 3.0,
-          "tokens": 5061
+          "ratio": 3.01,
+          "tokens": 5139
         },
         "L2": {
           "fidelity_missing": [],
           "fidelity_ok": true,
           "ratio": 2.38,
-          "tokens": 6370
+          "tokens": 6494
         },
         "L3": {
           "fidelity_missing": [],
           "fidelity_ok": true,
           "ratio": 1.18,
-          "tokens": 12867
+          "tokens": 13066
         }
       }
     }

docs/deterministic-compiler.md

@@ -372,3 +372,141 @@ the generated benchmark record instead of assuming a fixed savings percentage.
    size; publish per-language results under `benchmarks/results/`.
 5. **Fallback:** corpus runs with the pinned tree-sitter set absent and with syntax-broken files
    must degrade to the labeled generic baseline, never crash.
+
+---
+
+## React frontend (tsx/jsx)
+
+Step 1's grammar table lists `tree-sitter` for TypeScript and JavaScript, but two
+gaps sat under that row: `.tsx` files were routed to the plain TypeScript grammar
+(no JSX support, so JSX-bearing `.tsx` degraded to a parse error), and `.jsx`
+parsed fine but rendered every component as a truncated `FN` — hooks, state,
+effects, events, and render structure were invisible. `kern_react.py` closes both
+gaps as a post-pass over the same tree-sitter tree: no new parser, no new format.
+`COMPONENT` is a symbol kind inside the existing `ModuleIR`/`Symbol`/`emit_il`
+pipeline, so spans, slice hashes, tiers, faults, verify, cache, and redaction are
+inherited verbatim.
+
+### Grammar routing
+
+| suffix | grammar |
+| --- | --- |
+| `.js` `.jsx` `.mjs` `.cjs` | `tree_sitter_javascript` (JSX built in) |
+| `.ts` | `tree_sitter_typescript.language_typescript()` |
+| `.tsx` | `tree_sitter_typescript.language_tsx()` |
+
+`parse_tsjs(text, typescript: bool = False, tsx: bool = False)` selects the
+grammar (`tsx=True` wins over `typescript=True`), and `tsjs_available()` takes
+the same flags for per-grammar capability probes. Both call sites
+(`kern_cache.py` compile and verify paths) route this way. The React adapter
+runs only on the JSX-capable grammars (JavaScript and TSX; plain TypeScript has
+no JSX productions). The `generator=` header line reports
+`lang=typescript frontend=tree-sitter+react` for a `.tsx` module when the
+adapter fires, and plain `frontend=tree-sitter` otherwise — a strict no-op on
+non-component code, verified by `TestNoOpOnPlainCode` in `tests/test_react.py`.
+
+A symbol is upgraded to `kind="component"` when it is a function declaration,
+function expression, or arrow function, its name matches `^[A-Z]`, and a
+`return` (or arrow expression body) contains a `jsx_element`,
+`jsx_self_closing_element`, or `jsx_fragment`. `memo(Fn)` / `forwardRef(Fn)`
+wrappers are unwrapped: the inner function is lowered, the wrapper noted.
+Capitalized functions with no JSX stay plain `FN`; lowercase functions
+returning JSX also stay `FN` (not components by React convention).
+
+### Extraction vocabulary
+
+```text
+COMPONENT UserCard L3-18 #a3f9c2d1
+  PROPS user, onClose?=noop
+  STATE open=false
+  STATE [state, dispatch]=useReducer(reducer, init)
+  CTX theme=useContext(ThemeContext)
+  REF inputRef
+  HOOK data=useUserData(id)
+  EFFECT deps=[user.id]
+  EVENT Card.onClick -> set open=true
+  RENDER
+    Card
+      Avatar src=user.avatar
+      span {user.name}
+      IF open > UserDetails user=user
+      FOR item in items > Row key=item.id
+```
+
+Rules:
+
+- **PROPS** — first parameter. Destructured pattern lists names with defaults
+  (`onClose?=noop` when a default exists; `?` when the TS type marks it optional
+  and that is syntactically visible). Non-destructured param renders as its name
+  (`props`).
+- **STATE** — `const [x, setX] = useState(init)` → `STATE x=init`; setter name
+  recorded internally for EVENT lowering. `useReducer` renders the pair and
+  arguments.
+- **CTX / REF / HOOK** — `useContext`, `useRef`, and custom `use[A-Z]\w*` calls
+  respectively. Custom hooks are opaque: call text only, no cross-file
+  resolution.
+- **EFFECT** — `useEffect` / `useLayoutEffect`. Dependency array rendered
+  verbatim (`deps=[user.id]`, `deps=[]`); a missing array renders
+  `deps=EVERY-RENDER`. Effect body: L2 normally shows the head only but retains
+  risk-bearing operations required by the fault contract; L3 summarizes via
+  existing `flow()`.
+- **EVENT** — JSX attribute matching `on[A-Z]\w+={expr}`. If the handler body is
+  a single known-setter call, lower to `set <state>=<arg>`; otherwise render the
+  callee name or `flow()`-style summary at L3.
+- **RENDER** — JSX tree, indentation = nesting:
+  - `{cond && <X/>}` → `IF cond > X`; ternary → `IF cond > X ELSE > Y`
+  - `.map(` callback returning JSX → `FOR param in receiver > X`
+  - Host elements (lowercase) are structure; text/expression children render as
+    `{expr}` capped by existing `ntext` (secret redaction inherited)
+  - Capitalized JSX names are component dependencies; names that match imports
+    cross-link naturally through the existing import lines
+- Non-hook, non-render statements in the component body flow through the
+  existing L3 flow-op rendering unchanged (hook calls are skipped there —
+  they already surface as STATE/CTX/REF/HOOK/EFFECT heads). Components also
+  emit the same `EFFECTS` provenance line as plain functions (effect classes
+  plus `unknown-calls=N`) at L2 and L3. L2 retains only risk-bearing body ops
+  so their inline markers and `FAULT-BEFORE` entries cannot disappear.
+- All line math is `\n`-only, matching the repo rule (never `str.splitlines()`).
+
+### Tier mapping
+
+| Tier | Component detail |
+| --- | --- |
+| L1 | `COMPONENT name (props) span #hash` — one line, like current FN heads |
+| L2 | + STATE/CTX/REF/HOOK/EFFECT/EVENT heads, risk-bearing body ops, and the `EFFECTS` provenance line; RENDER collapsed to components-only tree (host elements and attributes dropped; IF/FOR structure kept) |
+| L3 | Full render tree with attributes, effect bodies, handler bodies, and non-hook body statements via `flow()` |
+
+### Faulting
+
+Ambiguity never disappears silently. Seven markers reuse the existing
+`!FAULT(...)` inline channel and FAULTS footer:
+
+| Construct | Marker |
+| --- | --- |
+| Hook called via alias or namespace (`R.useState`, renamed import) | `!FAULT(aliased-hook)` |
+| Dynamic component (`<Tag/>` where Tag is a lowercase variable or member expression) | `!FAULT(dynamic-component)` |
+| Spread props as the sole prop source (`{...rest}`) | rendered `...rest` + `!FAULT(spread-props)` |
+| Render prop / children-as-function | `!FAULT(render-prop)`, body summarized as flow |
+| Hook call inside conditional | `!FAULT(conditional-hook)` |
+| Multiple or nested JSX return paths collapsed to one render root | `!FAULT(render-control-flow)` |
+| Render tree exceeding op budget | explicit `…+N` + `!FAULT(render-truncated)` |
+
+Frontend IR remains a reasoning representation, not the write authority: edits
+still require faulting exact current source and verifying the slice hash, per
+the existing contract (Step 5).
+
+### Corpus run
+
+Compiled at L2 over two real corpora — KERN's own `app/` directory (4 files:
+`layout.jsx`, `page.jsx`, `docs/layout.jsx`, `docs/page.jsx`) and a fresh
+shallow clone of [`vercel/commerce`](https://github.com/vercel/commerce) (65
+`.js`/`.jsx`/`.ts`/`.tsx` files, `node_modules` excluded):
+
+```text
+files=69 crashes=0 components=61 faults=8
+ratio min=1.9 median=3.3 max=10.1
+```
+
+Zero crashes, components detected on both corpora, ambiguous constructs (spread
+props, dynamic components, render-prop children) surfaced as faults rather than
+silently dropped — the done bar this spec set for Stage 1 MVP.

docs/superpowers/specs/2026-07-11-react-frontend-ir-design.md

@@ -0,0 +1,127 @@
+# React Frontend IR — Design
+
+**Date:** 2026-07-11
+**Scope:** TSX/JSX grammar routing fix + React semantic extraction (Stage 1 MVP). Next.js adapter, type enrichment, and other frameworks are later specs.
+
+> **Historical note:** this spec predates the compiler-hardening merge from main. The shipped implementation adopts main's `parse_tsjs(text, typescript=, tsx=)` boolean API instead of the `dialect` parameter described below; the semantic vocabulary, tiers, and faulting are as specified. Current behavior: `docs/deterministic-compiler.md`.
+
+## Problem
+
+KERN-IL/0.2 lowers TS/JS through Tree-sitter, but:
+
+1. `.tsx` files are routed to the plain TypeScript grammar (`kern_cache.py` passes `typescript=True`; `parse_tsjs` loads `language_typescript()`). JSX is a syntax error in that grammar, so JSX-bearing `.tsx` files degrade to a parse-error IR. `tree_sitter_typescript.language_tsx()` exists and is unused.
+2. `.jsx` parses (the JavaScript grammar includes JSX) but the extractor has no JSX semantics: components render as `FN` symbols whose return statements are truncated JSX text. Hooks, state, effects, events, and render structure — the behavioral core of a React file — are invisible.
+
+React source is a verbose encoding of compact behavior. A deterministic semantic lowering recovers that behavior at a fraction of the tokens.
+
+## Decision summary
+
+- **No new parser.** Tree-sitter JS and TSX grammars provide syntax; KERN supplies React meaning. Writing or committing a custom React grammar is explicitly out of scope — React is semantics on top of JS/TSX syntax, not a new syntax.
+- **Extend KERN-IL, not a new format.** `COMPONENT` becomes a new symbol kind inside the existing `ModuleIR`/`Symbol`/`emit_il` pipeline. Spans, slice hashes, tiers, faults, verify, cache, and redaction are inherited verbatim. The runtime model learns one format.
+- **Adapter seam, not query packs.** React lowering lives in a new module (`skills/kern/scripts/kern_react.py`) as a post-pass over the same Tree-sitter tree, in the codebase's existing manual-visitor style. Future frameworks (Next.js, Vue, Svelte) become sibling adapters emitting the same vocabulary. Declarative `.scm` query packs are deferred until a second framework proves the need.
+- **Tiers stay the compression lever.** Component detail maps onto existing L1/L2/L3; no new knobs.
+
+## Architecture
+
+```text
+TSX/JSX source
+   ↓
+parse_tsjs(text, dialect)          # dialect: "js" | "ts" | "tsx"
+   ↓ tree-sitter CST + base symbols (FN/const/class/import)
+kern_react.py post-pass            # same tree, upgrades qualifying FN → COMPONENT
+   ↓
+ModuleIR → emit_il (existing)      # tier rendering, faults, hashes
+```
+
+### Grammar routing
+
+| suffix | grammar |
+| --- | --- |
+| `.js` `.jsx` `.mjs` `.cjs` | `tree_sitter_javascript` (JSX built in) |
+| `.ts` | `tree_sitter_typescript.language_typescript()` |
+| `.tsx` | `tree_sitter_typescript.language_tsx()` |
+
+`parse_tsjs(text, typescript: bool)` becomes `parse_tsjs(text, dialect: str)`. Both call sites (`kern_cache.py` compile and verify paths) updated. `generator=` header line reports `lang=tsx frontend=tree-sitter+react` when the adapter fires.
+
+### Component detection (deterministic)
+
+A symbol is upgraded to `kind="component"` when:
+
+- it is a function declaration, function expression, or arrow function, AND
+- its name matches `^[A-Z]`, AND
+- a `return` (or arrow expression body) contains `jsx_element`, `jsx_self_closing_element`, or `jsx_fragment`.
+
+Also detected: `memo(Fn)`, `forwardRef(Fn)` wrappers — the inner function is lowered, the wrapper noted. Capitalized functions with no JSX stay plain `FN`. Lowercase functions returning JSX stay `FN` (not components by React convention).
+
+## Extraction vocabulary
+
+```text
+COMPONENT UserCard L3-18 #a3f9c2d1
+  PROPS user, onClose?=noop
+  STATE open=false
+  STATE [state, dispatch]=useReducer(reducer, init)
+  CTX theme=useContext(ThemeContext)
+  REF inputRef
+  HOOK data=useUserData(id)
+  EFFECT deps=[user.id]
+  EVENT Card.onClick -> set open=true
+  RENDER
+    Card
+      Avatar src=user.avatar
+      span {user.name}
+      IF open > UserDetails user=user
+      FOR item in items > Row key=item.id
+```
+
+Rules:
+
+- **PROPS** — first parameter. Destructured pattern lists names with defaults (`onClose?=noop` when a default exists; `?` when the TS type marks it optional and that is syntactically visible). Non-destructured param renders as its name (`props`).
+- **STATE** — `const [x, setX] = useState(init)` → `STATE x=init`; setter name recorded internally for EVENT lowering. `useReducer` renders the pair and arguments.
+- **CTX / REF / HOOK** — `useContext`, `useRef`, and custom `use[A-Z]\w*` calls respectively. Custom hooks are opaque: call text only, no cross-file resolution.
+- **EFFECT** — `useEffect` / `useLayoutEffect`. Dependency array rendered verbatim (`deps=[user.id]`, `deps=[]`); a missing array renders `deps=EVERY-RENDER`. Effect body: L2 shows the head plus any risk-bearing ops required by the fault contract; L3 summarizes via existing `flow()`.
+- **EVENT** — JSX attribute matching `on[A-Z]\w+={expr}`. If the handler body is a single known-setter call, lower to `set <state>=<arg>`; otherwise render the callee name or `flow()`-style summary at L3.
+- **RENDER** — JSX tree, indentation = nesting:
+  - `{cond && <X/>}` → `IF cond > X`; ternary → `IF cond > X ELSE > Y`
+  - `.map(` callback returning JSX → `FOR param in receiver > X`
+  - Host elements (lowercase) are structure; text/expression children render as `{expr}` capped by existing `ntext` (secret redaction inherited)
+  - Capitalized JSX names are component dependencies; names that match imports cross-link naturally through the existing import lines
+- Non-hook, non-render statements in the component body flow through the existing L3 flow-op rendering unchanged; L2 retains only risk-bearing ops.
+- All line math is `\n`-only, matching the repo rule (never `str.splitlines()`).
+
+## Tier mapping
+
+| Tier | Component detail |
+| --- | --- |
+| L1 | `COMPONENT name (props) span #hash` — one line, like current FN heads |
+| L2 | + STATE/CTX/REF/HOOK/EFFECT/EVENT heads and risk-bearing body ops; RENDER collapsed to components-only tree (host elements and attributes dropped; IF/FOR structure kept) |
+| L3 | Full render tree with attributes, effect bodies and handler bodies via `flow()` |
+
+## Faulting
+
+Ambiguity never disappears silently. Reuses the existing `!FAULT(...)` inline channel and FAULTS footer:
+
+| Construct | Marker |
+| --- | --- |
+| Hook called via alias or namespace (`R.useState`, renamed import) | `!FAULT(aliased-hook)` |
+| Dynamic component (`<Tag/>` where Tag is a lowercase variable or member expression) | `!FAULT(dynamic-component)` |
+| Spread props as the sole prop source (`{...rest}`) | rendered `...rest` + `!FAULT(spread-props)` |
+| Render prop / children-as-function | `!FAULT(render-prop)`, body summarized as flow |
+| Hook call inside conditional | `!FAULT(conditional-hook)` |
+| Multiple or nested JSX return paths collapsed to one render root | `!FAULT(render-control-flow)` |
+| Render tree exceeding op budget | explicit `…+N` + `!FAULT(render-truncated)` |
+
+Frontend IR remains a reasoning representation, not the write authority: edits still require faulting exact current source and verifying the slice hash, per the existing contract.
+
+## Testing
+
+- **`tests/test_react.py`** — golden fixtures per construct: useState, useReducer, useContext, useRef, custom hook, effect with/without deps, events (setter and non-setter), IF/FOR render lowering, fragments, memo/forwardRef, spread props, dynamic component, aliased hook, conditional hook, `.jsx` and `.tsx` dialects, TSX generics ambiguity (`<T,>` arrow).
+- **Regression** — existing `test_tsjs.py` fixtures (no React constructs) must emit byte-identical IR; the adapter must be a strict no-op on non-component code.
+- **Corpus** — run compile over KERN's own `app/*.jsx` and a cloned real-world Next.js repo: zero crashes, all ambiguous constructs surface as faults (grep the FAULTS footer), compression ratios recorded per tier for the record. No hard ratio gate — compression is a large-file property and the `min_ir_tokens=600` source-cheaper stub already handles small files.
+
+## Non-goals (this spec)
+
+- Next.js concepts: routes, layouts, `use client`/`use server` boundaries, server actions, metadata (Stage 2 spec)
+- Type-aware enrichment via the TypeScript compiler API or LSP (Stage 3 spec)
+- Vue / Svelte / Angular / Astro adapters
+- Declarative `.scm` query-pack architecture
+- A custom Tree-sitter React grammar (never needed)

skills/kern/scripts/kern_cache.py

@@ -6,6 +6,7 @@
 import argparse
 import copy
 import hashlib
+import importlib
 import importlib.metadata
 import json
 import os
@@ -24,7 +25,7 @@
 BASELINE_GENERATOR = "kern-det/0.2"
 DERIVATION_MARKER = "kern-derivation/semantic16-v1"
 TSJS_SUFFIXES = {".js", ".jsx", ".mjs", ".cjs", ".ts", ".tsx"}
-ADDRESSABLE_SYMBOL_KINDS = {"function", "class", "type", "enum", "namespace", "module", "export"}
+ADDRESSABLE_SYMBOL_KINDS = {"function", "class", "component", "type", "enum", "namespace", "module", "export"}
 CACHE_DIRNAME = ".kern"
 DEFAULT_CONFIG: dict[str, Any] = {
     "schema": SCHEMA,
@@ -546,6 +547,31 @@ def _distribution_version(name: str) -> str:
         return "unknown"
 
 
+def _module_source_fingerprint(module_name: str, fallback: Path | None = None) -> str:
+    """Return a module source digest, or a deterministic availability marker.
+
+    Optional compiler adapters are still fingerprint inputs when they cannot be
+    imported: if their source is present, hash the source directly; otherwise
+    record a stable marker instead of making cache validation fail.
+    """
+    path = fallback
+    try:
+        module = importlib.import_module(module_name)
+        module_file = getattr(module, "__file__", None)
+        if module_file:
+            path = Path(module_file)
+    except Exception:
+        pass
+    if path is None:
+        return "unavailable"
+    try:
+        return sha256_file(path)
+    except FileNotFoundError:
+        return "missing"
+    except (OSError, ValueError):
+        return "unavailable"
+
+
 def compiler_fingerprint(
     source: Path,
     config: dict[str, Any],
@@ -562,6 +588,10 @@ def compiler_fingerprint(
         "python": f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}",
     }
     if suffix in TSJS_SUFFIXES:
+        try:
+            react_fallback = compiler_path.with_name("kern_react.py")
+        except ValueError:
+            react_fallback = None
         capabilities.update(
             {
                 "tree-sitter": _distribution_version("tree-sitter"),
@@ -573,6 +603,9 @@ def compiler_fingerprint(
             capabilities["tsjs"] = kern_compile.tsjs_capability_fingerprint()
         except Exception:
             capabilities["tsjs"] = "unavailable"
+        capabilities["kern-react-sha256"] = _module_source_fingerprint(
+            "kern_react", react_fallback
+        )
     payload = {
         "codec": CODEC_VERSION,
         "generator": BASELINE_GENERATOR,