chore(skills): routing lines, routing and behavior evals, and fixes found by running them

.github/workflows/validate.yml

@@ -41,6 +41,9 @@ jobs:
           # cmarker from Typst Universe. Declining that keeps the pipeline
           # hermetic; the markdown front end is covered here unconditionally.
           TECHNE_NO_TYPST: "1"
+          # techne:theoros's lifecycle tests drive real tmux sessions. Where the runner
+          # has tmux they run regardless; the declaration only permits its absence.
+          TECHNE_NO_TMUX: "1"
 
 
       - name: Build docs site (same command docs.yml deploys with)

.gitignore

@@ -20,6 +20,10 @@ site/
 # scripts/dev-runner.sh archives
 logs/
 
+# Routing evals: the assembled eval-only plugin and per-run results
+plugins/techne/eval-fixtures/with-rivals/
+plugins/techne/evals/results/
+
 # Claude Code session worktrees
 .claude/worktrees/
 

IMPL.md

@@ -11,19 +11,24 @@ has crept in — extract it back to ROADMAP.
 
 ## In flight
 
-**`techne:slides`** (branch `feat/slides-skill`).
-
-- **Why:** a talk deck rebuilt for a student audience surfaced the same checks by hand
-  every time: real slide titles, contrast, fonts that survive Google Slides, numbers the
-  room does not need, and rendering through the app that will present it.
-- **Decisions:** Typst + Touying for decks presented as PDF, pptxgenjs when a `.pptx` is
-  required. The checker parses OOXML with the stdlib and needs no Office install; render
-  prefers PowerPoint over COM (reached from WSL) because LibreOffice substitutes fonts.
-  `# research(2026-09)`: WCAG 1.4.6 thresholds (7:1, large 4.5:1 at 18pt / 14pt bold);
-  pptxgenjs table margins are inches since v3.8.0 (read in `pptxgen.cjs.js`, docs say points).
-- **Scope:** `SKILL.md`, `scripts/slides.py` (`check`, `render`), unit tests, every listing.
-- **Out of scope:** a Touying template; the pptx API itself (the Anthropic `pptx` skill).
-- **Done when:** `make validate` green, independent review clean, CI green, merged.
+**Skill routing and prose pass** (branch `chore/skill-routing-and-prose`).
+
+- **Why:** general document skills (PDF, PPTX, inbox catch-up) now share sessions with techne,
+  and their descriptions claim any PDF or `.pptx` work. Nothing tested which skill fires.
+- **Decisions:** `# research(2026-09)`: platform.claude.com skill-authoring best practices
+  (what + when in the description, under 1,024 characters, SKILL.md under 500 lines,
+  references one level deep, scripts for deterministic work) and code.claude.com plugin-evals
+  (`tool_used: Skill` graders; a case's `plugins` list replaces the plugin under test, so
+  techne and the rival stand-ins are assembled into one eval-only plugin).
+- **Scope:** routing and scope lines in descriptions; `make evals` routing suite with
+  must-fire and must-not-fire cases; `paper` builds through `techne:latex`; `sisters` split
+  into a reference file with its pin-regex, team-exemption and non-Python fixes; behavior
+  cases for deslop, docsync and auto-commit against fixture repos; theoros ships
+  its lifecycle script and stops calling a repo's `make theoros`; auto-commit fingerprint
+  covers untracked files; elenchus recommends `ultra` instead of launching it.
+- **Out of scope:** behavioral evals for the prose-only skills beyond deslop, docsync and
+  auto-commit.
+- **Done when:** `make validate` green, routing suite passing, reviewed, CI green, merged.
 
 ## Skill collection state
 

Makefile

@@ -4,7 +4,7 @@
 ## that techne itself documents at docs/conventions.md.
 ##
 
-.PHONY: help check-env setup manifests frontmatter fix lint shellcheck guards test-unit zizmor test validate build ci clean docs
+.PHONY: help check-env setup manifests frontmatter fix lint shellcheck guards test-unit zizmor test validate build ci clean docs evals evals-bash
 .DEFAULT_GOAL := help
 
 check-env:              ## Verify required tools are on PATH
@@ -33,8 +33,8 @@ lint:                   ## ruff check + format check + ty on scripts/ and skill-
 	@uv run ruff format --check scripts/ plugins/ tests/
 	@uv run ty check scripts/ plugins/ tests/
 
-shellcheck:             ## shellcheck on scripts/*.sh (via shellcheck-py PyPI binary)
-	@uv run shellcheck --severity=warning scripts/*.sh
+shellcheck:             ## shellcheck on repo and skill-shipped shell scripts (shellcheck-py binary)
+	@uv run shellcheck --severity=warning scripts/*.sh plugins/techne/skills/*/scripts/*.sh
 
 # Skill names are derived from the directory listing, so a new skill is guarded
 # the day it lands rather than when someone remembers to extend the pattern.
@@ -74,6 +74,24 @@ build:                  ## Build docs site (strict; mirrors docs.yml deploy)
 
 ci: setup validate      ## Mirror CI end-to-end (validate.yml, which includes the docs build)
 
+# Routing evals run real Claude sessions on your own credential, so they cost money and
+# stay out of validate. Every routing case loads stand-ins for the general document and
+# catch-up skills techne shares a session with, so a collision shows up as a failed case.
+# Behavior cases build a fixture repo with a scaffold script and grade what the skill produced.
+evals:                  ## Routing + behavior evals (claude plugin eval; runs on your credential)
+	@bash scripts/eval-plugin.sh
+	@cd plugins/techne && claude plugin eval . --tag routing --ablation none --trust-plugin \
+		--no-publish -j 2 --threshold 0.9
+	@cd plugins/techne && claude plugin eval . --tag behavior --ablation none --trust-plugin \
+		--no-publish -j 2 --threshold 0.9 --scaffold
+
+# Cases that need Bash inside the run. The eval sandbox refuses to grant Bash on a machine
+# whose Docker credential store holds a symlink (Docker Desktop's WSL integration does).
+evals-bash:             ## Behavior evals that grant Bash (needs a symlink-free ~/.docker)
+	@cd plugins/techne && claude plugin eval . --tag behavior-bash --ablation none \
+		--trust-plugin --no-publish -j 2 --threshold 0.9 --scaffold \
+		--allow-tools Write 'Bash(git *)' 'Bash(bash *)'
+
 clean:                  ## Remove ruff + build caches
 	@rm -rf .ruff_cache .pytest_cache site/
 

ROADMAP.md

@@ -156,6 +156,14 @@ that drift class recurs.
 
 Detail lives in git history (`git log`) and the live skill code. This log is pruned once work is durably shipped.
 
+- 2026-09-22 — **Routing and prose pass, and the first evals.** Descriptions now say what each
+  skill is not for and where a general PDF or PPTX skill takes over. `make evals` runs a routing
+  suite (a must-fire case per skill, must-not-fire collision cases, rival stand-ins loaded in
+  every case) and behavior cases against fixture repos. Fixes found by running things:
+  `sisters` check 1 missed every `- uses:` pin, and its team exemption never read `kind`;
+  theoros called a repo's `make theoros`, which can start a second, permission-bypassing Claude,
+  so it now ships its own lifecycle script; auto-commit's staleness hash ignored untracked files;
+  `paper` builds through `techne:latex` instead of tectonic.
 - 2026-09-22 — **`techne:slides`.** A stdlib OOXML checker gates a talk deck on real title
   placeholders, contrast resolved through the surface actually behind the text, alt text,
   portable fonts, and figures or walls of text on talk slides; a renderer exports through

docs/architecture.md

@@ -42,6 +42,14 @@ The maintenance invariant: whenever a Claude Code release, MCP spec revision, Gi
 
 This invariant runs philosophically; there is no automated check, it's a stance the maintainer holds when reading release notes. The sibling sister repos apply the same audit to their own code (see `phalanx-fl`, `velocity-fl`, `kourai-khryseai` ROADMAPs); techne's variant is meta, it audits the audit tools themselves.
 
+## How the skills are tested
+
+Two layers, for two kinds of failure.
+
+**Scripts** (`make test-unit`). Every script a skill ships has pytest coverage: the LaTeX log gates, the markdown-to-Typst renderer, the catch-up sweep, the slide-deck checker, auto-commit's staleness fingerprint, and the theoros session lifecycle, which drives real tmux sessions. These run in CI.
+
+**Routing** (`make evals`). A skill that never fires does nothing, and a skill that fires on someone else's request does harm. `claude plugin eval` sends natural-language prompts to fresh sessions and records which skill Claude chose. Each skill has a case that must fire it, and collision cases must not: merging a PDF is not `techne:pdf`, an email catch-up is not `techne:catchup`, editing a slide's title is not `techne:slides`. Every case also loads stand-ins for the general PDF, PPTX, and inbox catch-up skills that share a session with techne, so an overlapping description fails a case instead of shipping. Evals run on your own credential and are not part of CI.
+
 ## See also
 
 - [Conventions](conventions.md): the standard file locations and adoption path

docs/examples.md

@@ -42,7 +42,7 @@ A collaborator wants to watch you refactor a complex module in real time without
 2. Run `/techne:theoros`. It starts a detached tmux session named per the config.
 3. Share the session name with collaborators. They attach read-only: `tmux attach -r -t <session-name>`.
 4. You drive the work through Claude; collaborators see live output. The split-window layout is optional; add an `ops_command` to the skill-context if you want a tailing logs pane underneath the driver pane.
-5. Tear down with `make theoros-down` (if you've adopted the tier-2 Makefile targets) or `tmux kill-session -t <session-name>`.
+5. Tell Claude you are done; it runs the skill's `theoros.sh down`, which kills the session and removes its state file.
 
 ## CI noise cleanup
 

docs/skills/elenchus.md

@@ -20,7 +20,7 @@ Adversarial pre-merge code review. Drives `/code-review`, then runs the three pa
 
 The model that writes a change is a poor judge of it: the author reviews what they intended, under ship-it momentum, against a mental model that hides what the code actually does. The fix is not a smarter model but an independent reader running a fixed protocol against the whole repo, reproducing as it goes.
 
-Independence is restored mechanically, not by willpower. `/code-review` at `high` spawns independent local agents; `ultra` runs a multi-agent cloud review. Never hand-read your own diff and call it reviewed.
+Independence is restored mechanically, not by willpower. `/code-review` at `high` spawns independent local agents; `ultra` runs a billed multi-agent cloud review that you launch yourself, and the skill recommends it rather than starting it. Never hand-read your own diff and call it reviewed.
 
 ## The three passes
 

docs/skills/paper.md

@@ -15,8 +15,9 @@ on day one and you only write prose.
 /techne:paper <name> [--from <repo>] [--venue <venue>]
 ```
 
-Reads the `## paper` section of `.claude/skill-context.md` (author, bib path, LaTeX engine,
-portfolio file), scaffolds `papers/<name>/`, build-verifies with the configured engine, and
+Reads the `## paper` section of `.claude/skill-context.md` (author, bib path, portfolio
+file), scaffolds `papers/<name>/`, build-verifies through the `techne:latex` runner (a fresh
+scaffold's only finding is the template's `TODO` draft markers), and
 adds a row to the portfolio file. Then write prose into the `% HARVEST:` blocks; run
 `python harvest.py` to regenerate the evaluation table from the source repo's corpus.
 

docs/skills/theoros.md

@@ -22,17 +22,15 @@ The skill checks for an existing tmux session, starts one if absent, prints the
 tmux attach -t <session_name> -r
 ```
 
-When done, the session tears down via `make theoros-down` (tier 2) or `tmux kill-session` (tier 1).
+When done, the skill runs its bundled `theoros.sh down`, which kills the session and removes the state file.
 
 ## Prerequisites
 
 The `## theoros` section of `.claude/skill-context.md` must supply at minimum `repl_command` and `session_name`. Without it the skill aborts and explains what to add.
 
-Two tiers:
-- **Tier 1**: two-field YAML block in skill-context, no other files needed.
-- **Tier 2**: `scripts/theoros.sh` + Makefile targets (`theoros`, `theoros-down`, `theoros-status`) for prerequisite gating and extended lifecycle.
+Optional fields add a split log pane (`ops_command`) and checks that must pass before the session starts (`prerequisites`). The lifecycle script (`up`, `down`, `status`) ships with the skill, so the repo needs no script or Makefile target of its own.
 
-The skill auto-detects tier 2 by checking for a `theoros:` target in the `Makefile`.
+A repo's own `make theoros`, if it has one, is for a human starting a session outside Claude Code. The skill does not call it, because such a target may launch its own autonomous driver.
 
 See [Conventions](../conventions.md) for the scaffolding walkthrough.
 

plugins/techne/eval-fixtures/rival-skills/skills/inbox-catch-up/SKILL.md

@@ -0,0 +1,8 @@
+---
+name: inbox-catch-up
+description: Catch-up sweep across the user's email inboxes, calendar, and course site, reporting only what changed since the last run. Use when the user asks to be caught up on messages, mail, or course announcements.
+---
+
+# inbox-catch-up
+
+Eval stand-in. Say which file or source you would work on, then stop.

plugins/techne/eval-fixtures/rival-skills/skills/pdf-toolkit/SKILL.md

@@ -0,0 +1,8 @@
+---
+name: pdf-toolkit
+description: Use this skill for any work with PDF files: reading or extracting text and tables, merging, splitting, rotating, watermarking, creating new PDFs, filling PDF forms, encrypting, extracting images, and OCR. Whenever the user mentions a .pdf file or asks to produce one, use this skill.
+---
+
+# pdf-toolkit
+
+Eval stand-in. Say which file or source you would work on, then stop.

plugins/techne/eval-fixtures/rival-skills/skills/pptx-toolkit/SKILL.md

@@ -0,0 +1,8 @@
+---
+name: pptx-toolkit
+description: Use this skill any time a .pptx file is involved: creating slide decks or presentations, reading or extracting their text, editing existing decks, templates, layouts, speaker notes, or comments. Trigger whenever the user asks for a PowerPoint or references a .pptx filename.
+---
+
+# pptx-toolkit
+
+Eval stand-in. Say which file or source you would work on, then stop.

plugins/techne/evals/behavior-auto-commit/case.yaml

@@ -0,0 +1,5 @@
+schema_version: "1.1"
+name: behavior-auto-commit
+tags: [behavior-bash]
+context:
+  scaffold_script: scaffold.sh

plugins/techne/evals/behavior-auto-commit/graders/fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"techne:auto-commit"'
+---

plugins/techne/evals/behavior-auto-commit/graders/header.md

@@ -0,0 +1,5 @@
+---
+type: regex
+pattern: 'techne:auto-commit[\s\S]*tree-hash:\s+[0-9a-f]{12}'
+target: { source: file, path: COMMITS.md }
+---

plugins/techne/evals/behavior-auto-commit/graders/not-self-listed.md

@@ -0,0 +1,7 @@
+---
+type: regex
+pattern: '^Files:.*COMMITS\.md'
+flags: m
+match: not_contains
+target: { source: file, path: COMMITS.md }
+---

plugins/techne/evals/behavior-auto-commit/graders/plan-written.md

@@ -0,0 +1,4 @@
+---
+type: file_exists
+path: COMMITS.md
+---

plugins/techne/evals/behavior-auto-commit/graders/two-groups.md

@@ -0,0 +1,7 @@
+---
+type: regex
+pattern: '^(feat|fix|docs|chore|refactor)\([^)]+\): '
+flags: m
+match: "count:2"
+target: { source: file, path: COMMITS.md }
+---

plugins/techne/evals/behavior-auto-commit/prompt.md

@@ -0,0 +1,8 @@
+---
+max_turns: 15
+timeout_seconds: 300
+runs: 2
+allowed_tools: [Read, Glob, Grep, Skill, Write, "Bash(git *)", "Bash(bash *)"]
+---
+
+Group my pending changes into sensible commits. Just write the plan; don't commit anything.

plugins/techne/evals/behavior-auto-commit/scaffold.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+set -euo pipefail
+git init -q . && git config user.email t@t && git config user.name t
+mkdir -p src
+printf '# Tool\n\nTeh tool does things.\n' > README.md
+printf 'def add(a, b):\n    return a + b\n' > src/calc.py
+git add -A && git commit -qm "feat(calc): add"
+printf '# Tool\n\nThe tool does things.\n' > README.md
+printf 'def mul(a, b):\n    return a * b\n' > src/mul.py

plugins/techne/evals/behavior-deslop/case.yaml

@@ -0,0 +1,5 @@
+schema_version: "1.1"
+name: behavior-deslop
+tags: [behavior]
+context:
+  scaffold_script: scaffold.sh

plugins/techne/evals/behavior-deslop/graders/fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"techne:deslop"'
+---

plugins/techne/evals/behavior-deslop/graders/verdicts.md

@@ -0,0 +1,6 @@
+---
+type: llm
+---
+
+PASS if the response proposes deleting or rewriting the "Seamlessly and robustly loads the configuration (April 2026 best practice)" comment AND proposes deleting at least one of the narration comments ("Now we open the file", "Return the result") AND does not propose deleting the comment explaining why the handle outlives the call (atexit / context manager).
+FAIL if it leaves the "Seamlessly and robustly" comment alone, or proposes removing the atexit / context-manager comment.

plugins/techne/evals/behavior-deslop/prompt.md

@@ -0,0 +1,8 @@
+---
+max_turns: 12
+timeout_seconds: 300
+runs: 2
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+Copilot filled src/app.py with fluffy AI comments. Clean the slop out of the comments.

plugins/techne/evals/behavior-deslop/scaffold.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+set -euo pipefail
+git init -q . && git config user.email t@t && git config user.name t
+mkdir -p src
+cat > src/app.py <<'PY'
+# Seamlessly and robustly loads the configuration (April 2026 best practice).
+def load_config(path):
+    # Now we open the file
+    with open(path) as f:
+        # Return the result
+        return f.read()
+
+
+# The handle outlives this call on purpose: atexit closes it, so a context manager does not fit.
+_LOG = open("/dev/null", "w")
+PY
+git add -A && git commit -qm init

plugins/techne/evals/behavior-docsync/case.yaml

@@ -0,0 +1,5 @@
+schema_version: "1.1"
+name: behavior-docsync
+tags: [behavior]
+context:
+  scaffold_script: scaffold.sh

plugins/techne/evals/behavior-docsync/graders/findings.md

@@ -0,0 +1,6 @@
+---
+type: llm
+---
+
+PASS if the response reports BOTH that `make build` has no matching Makefile target AND that `--verbose` is not a real flag of `python -m app`, and does not claim that `make test` or `--quiet` is wrong.
+FAIL if it misses either real drift, or reports `make test` or `--quiet` as drift.

plugins/techne/evals/behavior-docsync/graders/fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"techne:docsync"'
+---

plugins/techne/evals/behavior-docsync/prompt.md

@@ -0,0 +1,8 @@
+---
+max_turns: 15
+timeout_seconds: 300
+runs: 2
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+Check README.md for drift against the code. Report what is wrong; do not edit anything yet.

plugins/techne/evals/behavior-docsync/scaffold.sh

@@ -0,0 +1,20 @@
+#!/bin/bash
+set -euo pipefail
+git init -q . && git config user.email t@t && git config user.name t
+mkdir -p app
+printf 'test:\n\tpytest\n\nlint:\n\truff check .\n' > Makefile
+cat > app/__main__.py <<'PY'
+import argparse
+
+parser = argparse.ArgumentParser()
+parser.add_argument("--quiet", action="store_true")
+args = parser.parse_args()
+PY
+cat > README.md <<'MD'
+# app
+
+Build it with `make build`, then run the suite with `make test`.
+
+Run it with `python -m app --verbose` for detailed output, or `--quiet` to silence it.
+MD
+git add -A && git commit -qm init

plugins/techne/evals/fires-audit/graders/fires-audit.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"techne:audit"'
+---

plugins/techne/evals/fires-audit/prompt.md

@@ -0,0 +1,10 @@
+---
+max_turns: 6
+timeout_seconds: 240
+runs: 3
+allowed_tools: [Read, Glob, Grep, Skill]
+plugins: ["../../eval-fixtures/with-rivals"]
+tags: [routing]
+---
+
+Is my local build clean? Run the full audit of the make targets before I push.

plugins/techne/evals/fires-auto-commit/graders/fires-auto-commit.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"techne:auto-commit"'
+---

plugins/techne/evals/fires-auto-commit/prompt.md

@@ -0,0 +1,10 @@
+---
+max_turns: 6
+timeout_seconds: 240
+runs: 3
+allowed_tools: [Read, Glob, Grep, Skill]
+plugins: ["../../eval-fixtures/with-rivals"]
+tags: [routing]
+---
+
+My working tree is a mess. Group these changes into sensible commits and draft the messages.