feat: add /provider command for mid-session model pinning (experimental) - #267
Merged
Brian Krabach (bkrabach) merged 1 commit intoAug 13, 2026
Merged
Conversation
- New /provider (status), /provider use <name> (pin), /provider auto (unpin) - Refuses loudly when orchestrator doesn't register capability (never silently fails) - Persistent [📌 <name>] prompt indicator, composing with existing [mode] indicator - (experimental) tagging on help/status/confirmations; usage-accuracy caveat in status view - Fixed latent bug: _create_prompt_session carried duplicate get_prompt closure that ignored pin getter * 48 unit tests passed while feature did not work * Added wiring tests that fail against broken code (6 of 11 fail when bug reintroduced) * Deleted duplicate; now properly invokes pin getter - Tests: 1372 passing Verified end-to-end in DTU with real interactive TUI and multi-vendor API calls. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Brian Krabach (bkrabach)
marked this pull request as ready for review
August 13, 2026 08:23
Brian Krabach (bkrabach)
deleted the
feat/provider-pin-command-experimental
branch
August 13, 2026 08:23
Brian Krabach (bkrabach)
pushed a commit
that referenced
this pull request
Aug 13, 2026
…, terse confirmations
Addresses the feedback from testing that the /provider use/auto confirmations
were 320-char paragraphs explaining that the pin 'takes effect on the NEXT
turn' and that users should check the token-usage line to confirm. That copy
compensated for uncertainty that no longer exists — three persistent surfaces
already do the confirming:
• Prompt indicator: [📌 name]> rendered one line down, persistent
• Per-turn footer badge: · 📌 pinned
• /provider status: full scope + usage caveat
The transition line was the least important of four surfaces and by far the
longest. This redesign reduces noise and applies progressive disclosure:
NEW BEHAVIOR (matches Option 4 from the design review):
First /provider use in a session (two lines, taught once):
📌 pinned: anthropic-fable
experimental · scope: this conversation only · /provider for details
Every subsequent pin (one line):
📌 pinned: openai-fast
Already pinned to that provider (one line):
📌 already pinned: openai-fast
Unpin when something was pinned (one line):
unpinned (was openai-fast)
Unpin when nothing was pinned (one line):
not pinned
CHANGES:
1. /provider use: 320 chars / 5 wrapped lines → 25 chars / 1 line (first pin
also includes 2-line teach block, one time per session).
2. /provider auto: 284 chars / 4 wrapped lines → 26 chars / 1 line.
3. Three design decisions locked in code:
a) DELIBERATE ASYMMETRY: use is confirmed by a signal APPEARING, auto by
one DISAPPEARING (weaker evidence). Unpinning destroys the prompt
indicator, the only other record of what was pinned. The (was X)
clause is the single genuinely non-redundant fact in either message.
Locked by a test with the reason documented.
b) FORWARD-LOOKING TENSE IS BANNED: every string is past-tense or
state-descriptive ('pinned', 'unpinned', 'already pinned', 'not pinned')
— each one true at the instant it prints, because it describes what
the system was TOLD, not what a model DID. Removed 'takes effect on
NEXT turn' entirely rather than rewording it. Enforced by runtime
sweeps and AST checks on the string constants.
c) NO-OP HONESTY: /provider auto with nothing pinned previously printed
'unpinned' (untrue), re-pinning an already-pinned provider printed a
fresh 'pinned' (untrue). Both were plausible-but-false confirmations —
exactly the failure class this feature exists to prevent. They now
report 'not pinned' and 'already pinned: X'. The already-pinned path
still calls pin() so an unmounted provider surfaces its loud error
instead of a false confirmation.
4. Messages render dim (color carries weight, reads as a receipt not content).
Provider names are markup-escaped so '[' can't open a style tag.
TESTS: 83 passed (was 71). Full suite: 1384 passed, 1 skipped, 13
deselected, 1 xfailed. All five states verified end-to-end in a real TUI
session; teach line correctly appears once per session; no old verbose text;
no de-emphasis attribute capture (pyte limitation) but luminance difference
measured at ~63% of normal (technically correct, visually dim).
Fixes follow-up to PR #267.
🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)
Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Brian Krabach (bkrabach)
added a commit
that referenced
this pull request
Aug 13, 2026
…, terse confirmations (#268) Addresses the feedback from testing that the /provider use/auto confirmations were 320-char paragraphs explaining that the pin 'takes effect on the NEXT turn' and that users should check the token-usage line to confirm. That copy compensated for uncertainty that no longer exists — three persistent surfaces already do the confirming: • Prompt indicator: [📌 name]> rendered one line down, persistent • Per-turn footer badge: · 📌 pinned • /provider status: full scope + usage caveat The transition line was the least important of four surfaces and by far the longest. This redesign reduces noise and applies progressive disclosure: NEW BEHAVIOR (matches Option 4 from the design review): First /provider use in a session (two lines, taught once): 📌 pinned: anthropic-fable experimental · scope: this conversation only · /provider for details Every subsequent pin (one line): 📌 pinned: openai-fast Already pinned to that provider (one line): 📌 already pinned: openai-fast Unpin when something was pinned (one line): unpinned (was openai-fast) Unpin when nothing was pinned (one line): not pinned CHANGES: 1. /provider use: 320 chars / 5 wrapped lines → 25 chars / 1 line (first pin also includes 2-line teach block, one time per session). 2. /provider auto: 284 chars / 4 wrapped lines → 26 chars / 1 line. 3. Three design decisions locked in code: a) DELIBERATE ASYMMETRY: use is confirmed by a signal APPEARING, auto by one DISAPPEARING (weaker evidence). Unpinning destroys the prompt indicator, the only other record of what was pinned. The (was X) clause is the single genuinely non-redundant fact in either message. Locked by a test with the reason documented. b) FORWARD-LOOKING TENSE IS BANNED: every string is past-tense or state-descriptive ('pinned', 'unpinned', 'already pinned', 'not pinned') — each one true at the instant it prints, because it describes what the system was TOLD, not what a model DID. Removed 'takes effect on NEXT turn' entirely rather than rewording it. Enforced by runtime sweeps and AST checks on the string constants. c) NO-OP HONESTY: /provider auto with nothing pinned previously printed 'unpinned' (untrue), re-pinning an already-pinned provider printed a fresh 'pinned' (untrue). Both were plausible-but-false confirmations — exactly the failure class this feature exists to prevent. They now report 'not pinned' and 'already pinned: X'. The already-pinned path still calls pin() so an unmounted provider surfaces its loud error instead of a false confirmation. 4. Messages render dim (color carries weight, reads as a receipt not content). Provider names are markup-escaped so '[' can't open a style tag. TESTS: 83 passed (was 71). Full suite: 1384 passed, 1 skipped, 13 deselected, 1 xfailed. All five states verified end-to-end in a real TUI session; teach line correctly appears once per session; no old verbose text; no de-emphasis attribute capture (pyte limitation) but luminance difference measured at ~63% of normal (technically correct, visually dim). Fixes follow-up to PR #267. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
10 tasks
Brian Krabach (bkrabach)
added a commit
that referenced
this pull request
Aug 13, 2026
Adds comprehensive user-facing documentation for the experimental /provider model-pinning feature, covering: - Mental model: each provider entry combines one account + one model - Worked example with two real Anthropic accounts and four models - Configuration, naming guidance, and default precedence via priority - Interactive usage: /provider, /provider use <name>, /provider auto - Limits: same-vendor only, session-only, top-level-only, orchestrator-dependent - Comparison table for three related but distinct 'provider' surfaces All instructions verified against live container with two real Anthropic accounts. Closes #267 and #268. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
9 tasks
Brian Krabach (bkrabach)
added a commit
that referenced
this pull request
Aug 14, 2026
) * docs: add provider pinning quickstart guide Adds comprehensive user-facing documentation for the experimental /provider model-pinning feature, covering: - Mental model: each provider entry combines one account + one model - Worked example with two real Anthropic accounts and four models - Configuration, naming guidance, and default precedence via priority - Interactive usage: /provider, /provider use <name>, /provider auto - Limits: same-vendor only, session-only, top-level-only, orchestrator-dependent - Comparison table for three related but distinct 'provider' surfaces All instructions verified against live container with two real Anthropic accounts. Closes #267 and #268. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> * feat: add CLI expertise bundle overlay with on-demand expert agent Add a composable bundle overlay that provides every CLI session with an expert agent that answers questions about the CLI itself. The expert reads from six existing docs (PROVIDER_PINNING, CONTEXT_LOADING, INTERACTIVE_MODE, GOAL_COMMAND, OUTPUT_FORMATS, SPAWN_PRECEDENCE), loaded on-demand when spawned (16K tokens), paired with a thin awareness routing pointer (317 tokens) loaded every session to prevent guessing. Bundle structure (repo root, matching layout of amplifier/core/foundation): - behaviors/cli-expertise.yaml: composed behavior including the agent - agents/cli-expert.md: context-sink expert with @-mentions of six docs - context/cli-awareness.md: routing pointer, deliberately bounded token budget - bundle.md: marker file for nested-bundle detection Modified: - pyproject.toml: force-include bundle dirs into installed wheel at amplifier_app_cli/_bundle/ (not package root, to avoid shadow effects) - runtime/config.py: _build_app_cli_behaviors() resolves bundle from installed package location, composes behavior only (never bundle.md) Fix: commands/run.py error messages now point to valid commands (/provider list, /provider add, /provider --help instead of /provider use) Known costs documented in PR body: docs/ duplication in wheel, root bundle.md affects repo as nested-bundle, sdist untested. Refs: #267, #268, #269 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --------- Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Brian Krabach (bkrabach)
added a commit
that referenced
this pull request
Aug 14, 2026
Add two new read-only diagnostic subcommands to the in-session /provider slash
command, closing a diagnostic dead end when provider pins fail.
BEFORE: /provider supported exactly three forms—status, use <name>, auto. When a
pin failed with ✗ cannot pin conversation provider 'fable': it is not mounted in
this session, the user had no in-session way to find out WHY. Bad API key? Wrong
model name? They had to leave the session and run amplifier provider test /
amplifier provider models in a shell.
AFTER: Two new read-only diagnostic forms, callable mid-conversation:
• /provider test [name] — connectivity check against the live mounted
providers. No name tests all, concurrently, bounded at 15s per provider.
• /provider models [name] — lists models a provider actually offers. No name
uses the active provider.
FILES
• amplifier_app_cli/provider_diagnostics.py (new, 114 lines)
Shared primitives: invoke_list_models(), ProviderTestResult,
test_provider_connectivity(), format_model_line(). Async-aware;
15s timeout per provider; single source of truth for connectivity semantics.
• amplifier_app_cli/provider_loader.py (+13/-4)
get_provider_models() now delegates to invoke_list_models() rather than
keeping its own inline implementation. One less independent implementation
of the same mechanic.
• amplifier_app_cli/main.py (+171/-4)
Slash command dispatch wiring, two new handlers (_handle_provider_test,
_handle_provider_models), usage string generation, error messages.
Updated /help description; unknown-subcommand error lists all five forms.
• tests/test_provider_test_and_models_command.py (new, 402 lines, 29 tests)
Unit and integration coverage for both commands, all error paths.
THREE DESIGN DECISIONS
1. MOUNTED PROVIDER REALITY, NOT DISK CONFIG
These operate on THIS SESSION'S MOUNTED PROVIDERS, not settings.yaml.
The existing amplifier provider test/models CLI commands read config from
disk and instantiate a throwaway provider. The slash versions read
coordinator.get("providers") — the live mounted instances. That is the
entire reason to run them mid-conversation instead of in a shell: if the
mounted set and the on-disk config disagree, the session's reality is what
the user needs. The session's mounted instances are never closed after
testing (asserted in tests); only the CLI's throwaway instances are.
2. NOT GATED ON CONVERSATION.PROVIDER_PIN CAPABILITY
/provider use and /provider auto require pin capability and refuse loudly
without it. These two are pure read-only diagnostics and are useful
precisely WHEN pinning is unavailable or failing — gating them would remove
the diagnostic exactly when it is needed most. Dispatched before the
capability gate; tested with pin=None.
3. SHARED LOGIC AT THE RIGHT SEAM
The existing click commands in commands/provider.py were left untouched —
they are disk-config-driven and their tests mock at that boundary. What is
genuinely shared is the lower mechanic ("call list_models() on a provider
object, async-aware") and the definition of "connectivity is OK", both of
which now live once in provider_diagnostics.py. So the two surfaces cannot
drift on what "OK" means.
CONCURRENCY & UX
/provider test with no name runs all targets via asyncio.gather, each
wrapped in asyncio.wait_for(..., 15s) so one hung provider cannot freeze
the REPL. Progress line prints before await. Wall clock tracks the slowest
provider (~1.1s cold, ~0.5s warm), not the sum.
Markup safety: [name] in usage strings was parsed as Rich markup and
silently vanished. Switched to <name>, matching /provider use <name> style.
EVIDENCE
• 29 new tests; 1413 passed / 1 skipped / 13 deselected / 1 xfailed
Same pass count as baseline (verified re-run against original stash).
ruff clean; pyright error count identical to baseline (none in new code).
• Verified end-to-end in live TUI (build 2026.08.14-25e9944), 8 checks:
✓ /help lists all five forms
✓ /provider models anthropic-sonnet — real network, 3 models, aligned
✓ /provider models (no name) — defaults to priority winner, byte-identical
✓ /provider test anthropic-sonnet — ✓ 0.1s, singular grammar
✓ /provider test (all four) — 4/4 ✓, 3+3+37+31 models across vendors
✓ Concurrency confirmed — wall clock ≠ sum(individual)
✓ /provider test nope — clean refusal, no traceback
✓ Pin regression intact: pin → pinned footer → unpin → priority winner
OUT OF SCOPE
add, edit, remove, install, manage were NOT added. They mutate persistent
config, a different concern from "who is answering me right now", and
mixing them in would blur the line: amplifier provider changes what is
available tomorrow; /provider changes who is answering now.
Closes: #267, #268, #269, #271 (extends /provider feature)
Generated with [Amplifier](https://github.com/microsoft/amplifier)
Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
/providercommand for mid-session model pinning within a conversation scope. Marked experimental.Key changes:
/provider(status),/provider use <name>(pin),/provider auto(unpin)conversation.provider_pincapability[📌 <name>]prompt indicator, properly composed with existing[mode]indicator(experimental)tag on help/status/confirmations; usage-accuracy caveat in status view_create_prompt_sessioncontained a duplicateget_prompt()closure that ignored the pin getterVerification: Proven end-to-end in DTU with real interactive TUI and real API calls. Prompt indicator renders correctly in all states (unpinned, pinned, pinned+mode). Cross-reference: amplifier-module-loop-streaming (sibling: capability + event), amplifier-module-hooks-streaming-ui (sibling: footer marker).