meta: track llm-here extraction to unify provider detection across noether, agentspec, caloron-noether

[alpibrupa]

Problem

"Which LLM CLI is installed, which env vars apply, what fallback order" is currently reimplemented in three places:

• caloron-noether: stages/phases/_llm.py
• agentspec: src/agentspec/resolver/resolver.py
• noether (grid): crates/noether-grid-*/src/cli_provider.rs (research branch)

Observed drift:

• caloron knows about the Nix sub-30s timeout cap; grid does not.
• agentspec handles Vertex AI routing; grid does not.
• Each project has its own understanding of fallback precedence.

The docs/research/llm-here.md design doc in this repo already proposes the fix: a single Rust binary llm-here, JSON in/out, callable both via subprocess (Python) and as a library crate (Rust).

Proposed scope

1. Create a new standalone repo alpibrusl/llm-here (not a crate inside noether — the utility's audience is broader than Noether and shouldn't be coupled to Noether's release cadence).
2. Consolidate provider detection, env-var priority, fallback order, timeout handling in one place.
3. Ship both llm-here binary and llm-here-rs crate.
4. Migrate the three callers:
   • noether-grid: replace cli_provider.rs with llm-here-rs dep.
   • agentspec: subprocess call (Python stays pure-Python).
   • caloron-noether: subprocess call.

Acceptance criteria (for this meta-issue)

• alpibrusl/llm-here repo exists with initial crate structure and CI.
• One caller migrated as proof of concept (suggest: noether-grid since it's the youngest).
• Follow-up issues filed in agentspec and caloron-noether to track their migrations.
• docs/research/llm-here.md updated to "implemented, see alpibrusl/llm-here".

Priority

High — highest-ROI cross-project refactor in the audit. Blocks nothing, but every month of delay compounds the drift.

Related

• alpibrusl/agentspec — callers to migrate.
• alpibrusl/caloron-noether — callers to migrate.
• Research doc: docs/research/llm-here.md.

[alpibrupa]

Progress update — repo created

Repo: https://github.com/alpibrusl/llm-here

v0.1.0 scaffold is live. What ships:

• `llm-here detect` — JSON report of reachable providers (4 CLIs + 4 API keys).
• `llm-here run` — stubbed with the stable error JSON shape so callers can code against v0.1 today; transport lands in v0.2.
• `llm-here-core` crate for Rust callers that want to skip the subprocess boundary.
• `SCHEMA.md` defines the semver'd wire format.

Supported providers: claude-cli, gemini-cli, cursor-cli, opencode, anthropic-api, openai-api, gemini-api, mistral-api.

Milestones

• v0.2 — wire CLI subprocess dispatch for `run --provider ` and `run --auto` (CLI-only).
• v0.3 — HTTPS transport for the four API providers.
• v0.4 — migration of noether-grid (`crates/noether-engine/src/llm/cli_provider.rs`), agentspec (`src/agentspec/resolver/resolver.py`), caloron-noether (`stages/phases/_llm.py`). Cross-repo regression fixtures.

Migration tracking issues will be filed in agentspec and caloron-noether so each project can close its piece independently.

Deferring this meta-issue closure until v0.4 lands, since the problem statement here is "triplication across the stack" — creating the repo is step one.

[alpibrupa]

v0.2.0 released — CLI dispatch works end-to-end

https://github.com/alpibrusl/llm-here/releases/tag/v0.2.0

What's shipping:

• `llm-here run --provider ` — subprocess dispatch with 25s default timeout, child-kill on expiry.
• `llm-here run --auto` — walks reachable CLI providers in fallback order.
• `llm-here-core::dispatch` crate exposing `CommandRunner` trait; noether-grid can depend on this directly instead of shelling out.
• 31 tests green in CI; fmt + clippy `-D warnings` pass.

Still stubbed: API providers (v0.3), then the cross-repo migrations (v0.4).

Useful for noether-grid right now: `crates/noether-engine/src/llm/cli_provider.rs` can migrate its `probe_subscription_clis()` to `llm_here_core::detect()` today, even before v0.3, because the detect API has been stable since v0.1. The subscription-CLI dispatch part of `cli_provider.rs` could also migrate to `llm_here_core::dispatch::run_cli_provider` right now. Happy to open a migration tracking issue in `alpibrusl/noether` if useful.

[alpibrupa]

v0.3.0 released — full caloron-settled fallback chain

https://github.com/alpibrusl/llm-here/releases/tag/v0.3.0

All four API providers (anthropic, openai, gemini, mistral) now dispatch via HTTPS. `run --auto` walks CLIs first, then APIs in REGISTRY order. End-to-end smoke against live Mistral API succeeded during development.

Ready for the noether-grid migration. `crates/noether-engine/src/llm/cli_provider.rs` can now delegate both probing and dispatch to `llm-here-core::{detect, dispatch, api}` — same subprocess pattern as the other two sibling projects, but you get to link the crate directly instead of going through the binary. Happy to open a migration PR against noether-grid if you'd like; otherwise will file a tracking issue here to match the ones already open in agentspec (#28) and caloron-noether (#21).

One small thing worth noting: the `run_auto` core function grew a third generic parameter `H: HttpClient` in v0.3. `run_auto_real` still works as-is for consumers who don't want to mock the HTTP boundary. This may matter if noether-grid ends up wanting to inject its own budgeted HTTP client for cost accounting later.

[alpibrupa]

Noether-grid migration PR is open: #54

#54

Status: branch feat/llm-here-migration, all 6 CI checks green (clippy, rustfmt, doc tests, coverage, test on ubuntu + macos).

Replaces in-tree subprocess dispatch in `crates/noether-engine/src/llm/cli_provider.rs` with a ~50-LOC adapter over `llm-here-core` v0.4.0. Public API unchanged — `providers.rs` and `noether-grid-worker/main.rs` compile without modification.

Deploy note: artifact deploy is in flight on v0.7.3; PR is held until that settles. Once it lands, this can merge without a version bump (internal refactor).

One follow-up that's not in this PR:

• crates.io switch. Currently `llm-here-core` is pulled via git dep pinned to `v0.4.0`. Once llm-here lands on crates.io, the Cargo.toml line switches to a version spec. Tracking separately.

After this merges, the remaining migration surface is:

• alpibrusl/agentspec#28 — Python; unblocked since llm-here v0.1.
• alpibrusl/caloron-noether#21 — Python; unblocked since llm-here v0.3.

Plan is to tackle those after the noether deploy settles and this PR lands, so each project's migration lives alongside the others in time rather than straddling releases.

[alpibrupa]

noether slice done in #54 — noether-grid's cli_provider.rs now delegates to llm-here-core v0.4.0 (git dep pinned to tag until llm-here lands on crates.io). Public API preserved, 583 tests green.

Remaining migrations per the meta-plan:

• agentspec feat(M2): resolver-normalisation pass #28 (unblocked since llm-here v0.1)
• caloron-noether fix: P2 batch — find_nix PATH walk, CLAUDE.md phase table, quickstart CI (closes #14 #17 #19) #21 (unblocked since llm-here v0.3)

When llm-here-core publishes to crates.io, switch the noether-engine dep from tag = "v0.4.0" to a semver requirement (test-plan item 3 on #54).

[alpibrupa]

All three migrations are up

Repo	PR	Status
noether	#54	Merged (2026-04-20)
agentspec	#29	Open
caloron-noether	#22	Open

Design choice for the Python migrations: soft dependency on llm-here with in-tree fallback when it's not on PATH. All existing tests stay green; new tests exercise the llm-here primary path. For agentspec this is the complete migration — it only delegates detection, not dispatch, and the detection surface is narrow. For caloron this is Phase 1; Phase 2 (delete the in-tree providers entirely, hit the ~30-LOC target from caloron#21) lands as a follow-up once llm-here is ubiquitous on caloron deployments.

The meta-tracker can close once the two open PRs merge, unless you want to keep it open until Phase 2 lands in caloron — happy either way.