Stall detection now fires while agent is busy (#252)

* feat: apply display-length clip for goal progress summary at render time

Goal progress renderer now clips run summaries to display length (120 chars),
moved from orchestrator so stored events retain full text while terminal output
stays single-line. Blocker and reason prose are deliberately never clipped.

Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>

* feat: stall detection now fires while agent is busy

Previously a stall was only detectable on a continuation turn that made zero
tool calls. This missed the dominant real-world failure mode: the three worst
stalls in the corpus (31, 24, and 54 turns) made tool calls every turn while
the goal had become unsatisfiable, so the detector never fired.

A second trigger, independent of tool activity, now runs alongside the
original. It uses a cheap deterministic pre-filter over recent evaluator
reasons every turn (no model call) and only consults the stall judge when the
pre-filter trips, preserving the invariant that a mechanical condition alone
never trips a stall.

The stall judge was reframed from binary yes/no into a verdict taxonomy:
resolvable, time-locked, structure-locked, or history-locked. The last three
all mean stop now and name the dead end, which users need to rewrite the
goal. The judge's prompts were split by trigger, since the original prompt
hard-codes the assertion that the assistant took no tool actions.

The CLI's stalled-state wording now derives from that verdict instead of from
a distinct-blocker count. The old counter compared strings with only
whitespace and case normalization, so an evaluator that rephrased a blocker
each turn produced a distinct signature per turn, yielding a false "flailing"
verdict.

The evaluator reason list is now capped before being passed to the summary
model; it previously grew unbounded and a 54-turn run shipped all 54 reasons.

Replayed against the three recorded reason chains: new pre-filter first
consults judge at turn 14 (stalled at 31), turn 12 (stalled at 24), and turn
18 (stalled at 54). On achieved organic runs from the same sessions: zero
false trips.

Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>

* feat: ship goalify skill inside CLI as version-locked packaged data

goalify composes a /goal stop-condition from the current conversation
and lints it against known termination-failure patterns before showing
it to the user. It has a hard prerequisite — it is only meaningful when
the CLI provides the /goal command — so co-locating it with the CLI
satisfies that prerequisite by construction rather than by configuration.

The skill lives at amplifier_app_cli/data/skills/goalify/ and is
registered by appending a package-relative path in
_ensure_default_skills_dirs(). Resolution is by installed package
location, never by git URI: a git reference would have co-located the
files in the repo while still versioning them independently of the code
they depend on, which defeats the purpose. The wheel build declares
packages = ["amplifier_app_cli"], so repo-root files are excluded
from the distribution entirely.

The skill directory is appended in Python rather than declared through
a bundle's tool-skills config, because bundle-level skills config
replaces rather than appends — the same hazard _ensure_default_skills_dirs
was originally written to work around. Verified that workspace and user
skills still load alongside the packaged one.

The skill body contains only instructions to the invoking agent. Authoring
rationale and the lint-rule evidence base were moved to sibling
PROVENANCE.md, which load_skill does not read.

The lint output table now reports "no known pattern detected" rather
than "PASS", because a column of PASS reads as a validation claim the
linter cannot support — it detects known patterns from a finite corpus.

A repo-root AGENTS.md records the decision rule for what may live in
amplifier_app_cli/data/ versus an external bundle: the asset must depend
on something the CLI uniquely provides, no non-CLI host would want it,
and unconditional triggers belong in the bundle layer while gated ones
stay in Python. Assets resolve by package path only, and this location
is auto-loaded for every user in every session, so its token budget
discipline is stricter than anywhere else.

Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>

---------

Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>

Author: bkrabach

AGENTS.md

@@ -0,0 +1,36 @@
+# AGENTS.md — amplifier-app-cli
+
+## Boundary rule: `amplifier_app_cli/data/` vs. an external bundle
+
+`amplifier_app_cli/data/` ships inside the CLI's own wheel — anything placed there loads
+for every user, every session, version-locked to the installed CLI, with no bundle
+composition step in between. That reach is exactly why it must stay small. Before adding
+anything here, run it through these three tests, in order:
+
+1. **Does it depend on something the CLI uniquely provides, that cannot move?** A slash
+   command, a settings key, a terminal affordance — something with no home outside this
+   process. If no → it belongs in an external bundle, not here.
+2. **Would a non-CLI host ever want it?** If yes → external bundle. The CLI may still
+   *include* it (compose the bundle), but must not *own* it — ownership belongs wherever
+   the capability is portable to.
+3. **Is the trigger unconditional?** If the asset is gated on settings, an env var, a flag,
+   or runtime state, the asset itself may still live here, but the compose/injection
+   *decision* stays in Python (see `runtime/config.py::_ensure_default_skills_dirs` for the
+   pattern) — never encode conditional loading in a bundle YAML that lives alongside it.
+
+If the answer to 1 is "no" or the answer to 2 is "yes," it's an external bundle question,
+not a `data/` question.
+
+**Resolution rule:** assets under `amplifier_app_cli/data/` are always resolved by
+**package-relative path** (e.g. `Path(__file__).parent.parent / "data" / "..."`), **never**
+by git URI. A git URI decouples the asset's version from the installed wheel's version —
+defeating the reason for co-locating it here in the first place. If it needs independent
+versioning, it isn't a `data/` asset.
+
+**Token budget:** this location is auto-loaded for every user, every session — its budget
+discipline is stricter than anywhere else in the ecosystem. No always-on context files
+here without an explicit, named exception recorded in this section. Prefer mechanisms
+that load on demand (skills, agent-scoped context) over anything injected unconditionally.
+
+This section exists to keep `data/` from becoming a junk drawer — re-run the three tests
+before adding, not after.

amplifier_app_cli/data/skills/goalify/PROVENANCE.md

@@ -0,0 +1,69 @@
+# goalify — rule provenance
+
+Not loaded by `load_skill`. This file exists so the lint rules in `SKILL.md`
+can be audited and re-derived. Read it when changing a rule, not when using
+the skill.
+
+## Where the rules come from
+
+Every BLOCKER traces to an observed `/goal` run that failed to terminate.
+That property is the ruleset's credibility. **Do not add a rule without a
+named run or a corpus measurement behind it.**
+
+| Rule | Origin |
+|---|---|
+| L1 — ordering/provenance constraint | A run whose condition required `PROVEN with evidence you produced yourself`. The evaluator crystallised this into a constraint on the transcript's own past and stated it could not be retroactively repaired. Stalled after 54 turns; unreachable from roughly turn 45. |
+| L2 — universal quantifier with exempt members | A run requiring evidence for `all 9 sites` in a uniform structure, where one site could not structurally produce that evidence. Stalled after 24 turns. |
+| L3 — elapsed wall-clock requirement | A run whose only available proof standard was real-world use over time. Stalled after 31 turns. |
+| L4 — human-in-the-loop dependency | A condition containing `Stop and ask me if you need a decision from me`, which contradicts unattended continuation. 12 turns consumed in 35 seconds. |
+| L5 — open enumeration | `all editing features of Publisher` — never terminated. Its sibling run in the same session, same codebase, same day (`until I have a usable app`) achieved on turn 1. |
+| L0 — cross-clause consistency | The L2 run above carried a textbook four-verdict exit vocabulary and stalled anyway, because one other sentence silently overrode it. |
+| L6 — missing disjunctive exit | Advisory, not blocking. See below. |
+
+## Why L6 is a WARNING, not a BLOCKER
+
+A lint regression over 30 scored real runs with known outcomes measured L6
+firing on 37% of all real conditions at a 9% hit rate — the largest single
+source of false positives of any rule, consistent across the tuning split,
+the held-back split, and the full set. L6 judges exit-clause presence in
+isolation and cannot see turn position or residual scope, so it routinely
+flagged short finisher-style conditions written late in a long session.
+
+On that corpus, presence of a disjunctive exit was only weakly correlated
+with actual termination — several conditions with strong exit language
+stalled anyway, for reasons L1/L2/L0 cover independently.
+
+Demoting L6 raised measured precision from 20% to 43%, recall unchanged
+at 100%. L0 and L1–L5 each fired only 1–2 times in that evaluation — too few
+observations to justify changing their classification, so none was changed.
+
+## How to read the precision number
+
+The 20%/43% figures were measured on a corpus dominated by **human-authored**
+conditions, which terminate about 96% of the time. On a population that
+rarely fails, any linter's precision is bounded by arithmetic — a rule that
+flags a condition which succeeded anyway counts as a false positive.
+
+This skill lints **agent-authored** conditions, which in the same corpus
+terminated about 60% of the time. Same rules, roughly ten times the base
+rate of true positives. **Treat 43% as a floor measured on the easiest
+available population, not as the operating precision.** Re-scoring against
+the agent-authored subset alone is the outstanding measurement.
+
+The asymmetry also justifies the operating point: a false positive costs one
+in-session rewrite pass; a false negative costs a 24-to-54-turn unrepairable
+stall. High recall at moderate precision is the correct trade here.
+
+## Standing caveats
+
+- **The corpus ages.** These rules encode `/goal` evaluator semantics as
+  observed at the time of measurement. If the goal loop's evaluator changes,
+  nothing will automatically flag that the rules have rotted.
+- **Effective sample size is smaller than it looks.** The scored runs came
+  from roughly 8 distinct sessions; runs within a session share an author, a
+  project, and phrasing habits.
+- **The skill drafts; the human edits.** Automating goal authoring is itself
+  the thing that raises failure rates (60% vs 96%). The lint is the bet that
+  it closes that gap, and that bet has not been measured end to end. The
+  human review step is load-bearing, which is why the skill offers and never
+  auto-runs.

amplifier_app_cli/data/skills/goalify/SKILL.md

@@ -0,0 +1,188 @@
+---
+name: goalify
+description: >
+  Compose a /goal stop-condition from the current conversation and lint it
+  against known termination-failure patterns before showing it to the user.
+  Use when the user wants to turn the current task into a /goal loop, asks to
+  "goalify this", wants a stop condition for autonomous work, says "write a
+  goal condition", "make this a /goal", "turn this into a goal", or asks for
+  help wording a condition for /goal.
+user-invocable: true
+version: 1.1.0
+license: MIT
+---
+
+Run this procedure yourself, in the current conversation. Do not delegate it
+to a sub-agent or forked session — Phase 1 reads the live transcript.
+
+$ARGUMENTS
+
+If the user supplied focus text above, use it to scope Phase 1. If empty,
+extract from conversation alone — do not ask the user to restate what they
+already said.
+
+---
+
+## Phase 1 — Extract
+
+From the conversation so far, determine:
+
+- **The target end state.** What does "finished" concretely look like? It
+  must be a state that can be checked, not an activity that can be performed
+  indefinitely. ("a usable app" is checkable; "finish building the project"
+  is not — there is no test for "finished building".)
+- **What is already done.** Re-read the transcript for completed sub-tasks,
+  passing tests, merged changes, or resolved questions. These become
+  candidates for the KNOWN section (Phase 2) or for narrowing scope
+  (SCOPE-OUTS).
+- **What is explicitly NOT required.** Anything the user has ruled out,
+  deferred, or said isn't needed. This is the raw material for SCOPE-OUTS.
+
+If the end state genuinely cannot be determined from context (not merely
+effortful to determine), ask one direct question. Otherwise proceed —
+guessing and then showing your extraction for correction is faster than
+front-loading a question the transcript already answers.
+
+## Phase 2 — Compose
+
+Emit a candidate condition using this structure. Every element is required
+unless marked optional.
+
+1. **One-sentence outcome** naming a checkable end state (not an activity).
+2. **Disjunctive exit**: the condition must be satisfiable by *either*
+   reaching the end state *or* conclusively demonstrating it cannot be
+   reached (naming the blocker). Never phrase a condition with only one exit.
+3. **Per-item negative terminal**, if the condition lists multiple items
+   (tasks, sites, phases, experiments). Each item must be able to resolve to
+   its own PASS / FAIL / BLOCKED-with-named-reason — a blocker on one item
+   converts *that item* to a residual; it must not block the whole goal.
+4. **SCOPE-OUTS section** — an explicit list of what is *not* required. Write
+   this by directly converting anything from Phase 1's "not required" list
+   into a plain negative statement (e.g. "No production soak time required."
+   / "Uniformity across all N items is NOT the goal.").
+5. **KNOWN section (optional)** — facts already established, so the actor
+   doesn't re-derive them. Label it explicitly as a speed aid: it prevents
+   wasted turns, it does not by itself prevent stalls, so it never replaces
+   items 1–4.
+
+## Phase 3 — Lint
+
+Check the full composed document against every rule below. Work through all
+BLOCKERS first; a document with any BLOCKER triggered is not ready to show.
+Then check WARNINGS, which are advisory and do not block presentation.
+
+**Read the whole document for each check.** Several of these rules are only
+detectable by considering the document as one system — a single clause
+elsewhere can silently defeat a correct-looking rule everywhere else. Do not
+scan for keywords in isolation and stop at the first clean-looking match.
+
+### BLOCKERS — fix all before presenting
+
+- **L1 — Ordering/provenance constraint on the transcript's own history.**
+  Any phrasing that requires evidence to precede, or be produced independent
+  of, events that already exist in the transcript (e.g. "verify it yourself,
+  then state what you verified", "proof must precede the claim", "evidence
+  you produced yourself" applied to something already reported by a
+  sub-agent or prior turn). This class of requirement is **unrepairable** —
+  no later turn can change what already happened earlier in the transcript,
+  so it cannot be fixed by adding more work. If the condition constrains
+  ordering, it must constrain only *future* actions, never re-litigate what
+  is already in the history.
+
+- **L2 — Universal quantifier over a set with possibly-exempt members.**
+  "all N", "every X", "each of the Y", "uniform/uniformity", "complete
+  parity", applied to a set, is a blocker **unless** each item individually
+  carries a negative terminal (see Compose #3) or the condition names which
+  members are exempt and why. Without one of those, a single member that
+  cannot structurally produce the required evidence makes the whole
+  condition permanently unsatisfiable.
+
+- **L3 — Elapsed wall-clock requirement.** Anything that requires real time
+  to pass beyond the current session: "production soak", "after N days of
+  use", "monitor over time", "verify in real-world use". A single session
+  cannot advance wall-clock time; this can never be satisfied in-session.
+
+- **L4 — Human-in-the-loop or external-actor dependency mid-loop.**
+  "stop and ask me if you need a decision", "once a reviewer merges this",
+  "wait for approval before continuing". This directly conflicts with
+  unattended continuation — the loop will halt waiting on an event that a
+  condition-checking loop cannot itself produce.
+
+- **L5 — Open enumeration.** Scope phrased as an unbounded or unenumerated
+  set: "all editing features of X", "complete parity with Y", "everything
+  needed to fully support Z". An evaluator can always name one more item
+  under this phrasing, so it never terminates. Convert to a closed, named
+  list, or to a single representative artifact.
+
+- **L0 — Cross-clause consistency (meta-rule).** *An escape hatch is only as
+  strong as the strictest other clause in the same document.* After
+  confirming L1–L5 pass individually and a disjunctive exit exists, re-read
+  the document once more asking only: **is there any other sentence, anywhere
+  in the document, that is stricter than the stated exit and would override
+  it?** A document can have a textbook-perfect exit clause and still be
+  unsatisfiable because one unrelated sentence elsewhere re-imposes an L1–L5
+  style constraint the exit clause doesn't cover. Confirming an exit clause
+  exists is not sufficient — confirm nothing else in the document is
+  stricter than it.
+
+### WARNINGS — advisory, do not block presentation
+
+- **L6 — Missing disjunctive exit.** The document should state achievement
+  *or* a way to conclusively end in "not achievable, here is why" (see
+  Compose #2). Flag and fix its absence where practical, but do not block
+  presentation on it alone.
+- **W1** — Multiple items are listed but not all of them carry their own
+  negative terminal (some do, some don't).
+- **W2** — No clause asking the actor to show evidence inline in the
+  transcript as it's produced, rather than only asserting a result.
+- **W3** — Scope reads like more than one session's worth of work (multi-week
+  rollout language, coordination across many independent repos/teams,
+  phased production deployment).
+- **W4** — The condition contains a cautionary anecdote or narrative about a
+  failure mode (e.g. "don't repeat what went wrong last time", "make sure
+  this doesn't stall like before") rather than a plain instruction. Any such
+  narrative addressed to the actor is read by the evaluator too, and can
+  silently become a criterion the evaluator judges against instead of
+  guidance the actor merely follows. State requirements as plain criteria,
+  never as stories.
+
+  **This applies to the condition you are composing right now.** Write every
+  clause as a direct instruction to the actor, never as a story about a past
+  run. If you catch yourself writing "so that we don't repeat X", rewrite it
+  as the direct requirement it implies, with no reference to the incident.
+
+### If a BLOCKER cannot be cleared
+
+Rewrite and re-check. Allow up to three rewrite passes. If a BLOCKER still
+fires after three passes, stop and surface the specific tension to the user
+by name (e.g. "the user's own request requires enumerating an open-ended set
+— L5 fires no matter how I phrase it; how would you like to bound this?").
+Do not present a condition that still fails a BLOCKER.
+
+---
+
+## Output format
+
+Always output the condition inside a fenced code block — terminal reflow
+will otherwise destroy its multi-line structure. Follow it with the lint
+report as a table, then offer (do not auto-run) `/goal`.
+
+```
+<the condition text>
+```
+
+| Rule | Result | Note |
+|------|--------|------|
+| L0 | no known pattern detected | ... |
+| L1 | no known pattern detected | ... |
+| L2 | no known pattern detected | ... |
+| L3 | no known pattern detected | ... |
+| L4 | no known pattern detected | ... |
+| L5 | no known pattern detected | ... |
+| L6, W1–W4 | (list only the ones that fired) | ... |
+
+A clean table means no *known* failure pattern was detected — not that the
+condition is validated. Say so if the user reads it as a guarantee.
+
+Then: "Pass this to `/goal` to start the loop — want me to run it now, or
+would you like to adjust anything first?"

amplifier_app_cli/data/skills/goalify/examples/known-bad-L1-ordering.md

@@ -0,0 +1,25 @@
+# Known-bad example — L1 (ordering/provenance constraint)
+
+Used to sanity-check the lint in `SKILL.md` Phase 3. This condition is
+modeled on a real run that stalled for many turns because the ordering
+constraint it imposes on the transcript's own history became impossible to
+satisfy once evidence had already been reported by a sub-agent.
+
+## Condition text
+
+```
+Done when the migration is PROVEN complete, with evidence you produced
+yourself. A sub-agent's report is not proof. Verify it yourself, then state
+what you verified — or show a named blocker and stop.
+```
+
+## Expected lint result
+
+- **L1: FAIL** — "with evidence you produced yourself" combined with "verify
+  it yourself, then state what you verified" imposes an ordering constraint
+  on the transcript's own history. If a sub-agent already reported a result
+  earlier in the transcript, no later turn can retroactively make that
+  report "verified by you first" — the requirement is unrepairable once the
+  transcript already contains the sub-agent's report.
+- L6 present (disjunctive exit exists: "or show a named blocker and stop"),
+  but L1 alone should be sufficient to fail this condition.

amplifier_app_cli/data/skills/goalify/examples/known-bad-L2-quantifier.md

@@ -0,0 +1,30 @@
+# Known-bad example — L2 / L0 (universal quantifier + cross-clause override)
+
+Used to sanity-check the lint in `SKILL.md` Phase 3. Modeled on a real run
+that stalled because a uniform requirement applied to every member of a set
+made the condition permanently unsatisfiable once one member could not
+structurally produce the required evidence.
+
+## Condition text
+
+```
+Done when all 9 sites are migrated in a uniform structure, verified working,
+or proven impossible with a named blocker.
+```
+
+## Expected lint result
+
+- **L2: FAIL** — "all 9 sites" + "uniform structure" is a universal
+  quantifier over a set, with no per-item negative terminal and no named
+  exemption for a site that cannot physically satisfy "uniform structure."
+- **L0: FAIL** — the document has a disjunctive-looking exit ("or proven
+  impossible with a named blocker"), but that exit applies to the *goal as a
+  whole*, not per-site. The stricter clause ("uniform structure" across all
+  9) is not covered by the escape hatch, because the escape hatch only
+  fires once — it can't let 8 sites succeed uniformly while 1 is blocked and
+  still call the set "uniform." Confirming the exit clause exists is not
+  enough; it does not cover the stricter clause.
+- L6 nominally present (a disjunctive-shaped phrase exists) but does not
+  actually rescue the condition — this is exactly the L0 case: an escape
+  hatch that reads as satisfied but is overridden by a stricter clause
+  elsewhere in the same document.