Skip to content

feat(stage_router): corroborative tanh-scored routing with windowed self-clearing signals - #68

Merged
sabhatinas merged 21 commits into
mainfrom
sabhatinas/stage-router-scorer-corroborative-tanh
Jul 24, 2026
Merged

sabhatinas merged 21 commits into
mainfrom
sabhatinas/stage-router-scorer-corroborative-tanh

Conversation

@sabhatinas

@sabhatinas sabhatinas commented Jul 14, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does

Reworks the stage_router scorer, picker, and signals into a cleaner, honestly-documented router — and moves the routing logic into one shared Rust core.

  • One shared Rust core. The scorer + picker now live in switchyard-components/stage_router.rs (a short priority ladder: escalate → de-escalate → scorer → fall-open). Python calls it via a binding (stage_pick_tier); the Rust profile calls it directly. One source of truth. The dead prompt-classification path is deleted. (SWITCH-1056)
  • Two-axis scorer. Error signals (a hard error, spinning, or exploring) push toward the strong model; making progress (writing code) pushes toward the weak model. Squashed with tanh, and confidence = |score|.
  • Corroborative. One signal alone rarely flips the decision — it takes a signal plus backup. A critical error overrides everything.
  • Cleaner signals. spinning and exploring are mutually exclusive. Dropped repeated_cmd_ratio from scoring — it fired on nearly every shell command and never caught a real loop.
  • Compaction holds strong. Once the context is compacted, the router pins the strong model.
  • Handoff notes. Optional short note at a tier switch telling the next model why it was switched (off by default).
  • Per-tier system prompts. Optional prompt injected for the weak or strong tier.

Docs and tests updated; the old calibration scripts are replaced by the scorer skill.

Signals live in libsy. Tool-signal extraction (tool_signals) now lives in switchyard_libsy::ToolSignals; switchyard-components re-exports it — no divergence. The scorer/picker core stays in switchyard-components/stage_router.rs.
count_tokens was split out to its own PR (SWITCH-1048).

Linear: SWITCH-1056 (Rust consolidation), SWITCH-939 (scorer), SWITCH-983 (compaction), SWITCH-982 (handoff notes).

@sabhatinas
sabhatinas requested a review from a team as a code owner July 14, 2026 21:08
@sabhatinas sabhatinas self-assigned this Jul 14, 2026
@sabhatinas
sabhatinas marked this pull request as draft July 14, 2026 21:13
@coderabbitai

coderabbitai Bot commented Jul 14, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The stage router now separates tests_passed decisions, uses recent-window progress gates, routes settled runs directly to EFFICIENT, preserves critical overrides, and computes tanh-squashed scores with updated tests.

Changes

Stage router behavior

Layer / File(s) Summary
Parameterized tanh scoring
switchyard/lib/processors/stage_router/scorer.py, tests/test_stage_router_scorer.py
Grouped signal weights and tanh scoring replace fixed weights and hard clipping; tests cover direction, saturation, confidence, and raw contributions.
Routing gates and settled-run precedence
switchyard/lib/processors/stage_router/decision_log.py, switchyard/lib/processors/stage_router/dimensions.py, switchyard/lib/processors/stage_router/picker.py, tests/test_stage_router_pickers.py
Recent-window gates and the dedicated tests_passed source support early EFFICIENT routing, while critical severity continues to force CAPABLE; picker scenarios are updated accordingly.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Poem

I’m a bunny with tests in my ear,
Hopping through scores that grow smooth and clear.
Settled runs dash to EFFICIENT light,
Critical thumps keep CAPABLE in sight.
Tanh curls the signals, soft as spring—
New paths bloom while green checks sing.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 88.24% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly captures the main stage_router changes: tanh scoring, corroborative routing, and windowed self-clearing signals.

Comment @coderabbitai help to get the list of available commands.

@sabhatinas
sabhatinas force-pushed the sabhatinas/stage-router-scorer-corroborative-tanh branch 2 times, most recently from 0279579 to 220c474 Compare July 17, 2026 19:13
@sabhatinas
sabhatinas marked this pull request as ready for review July 22, 2026 20:22
@sabhatinas
sabhatinas force-pushed the sabhatinas/stage-router-scorer-corroborative-tanh branch from 03ffe96 to b098dfd Compare July 22, 2026 23:30
@sabhatinas
sabhatinas enabled auto-merge (squash) July 22, 2026 23:44
@github-actions

github-actions Bot commented Jul 23, 2026 •

Copy link
Copy Markdown
PR Preview Action v1.8.1

🚀 View preview at
https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-68/

Built to branch gh-pages at 2026-07-24 18:23 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@ryan-lempka

Copy link
Copy Markdown
Collaborator

@sabhatinas is the plan to refactor this to align with libsy design at a later date / does this take libsy into account?

Comment thread switchyard/lib/endpoints/anthropic_messages_endpoint.py Outdated
Comment thread crates/switchyard-py/src/component_bindings/dimension_collector.rs Outdated
Comment thread benchmark/score_staged_run.py
Comment thread benchmark/score_run.py Outdated
Comment thread switchyard/lib/processors/stage_router_request_processor.py
Comment thread switchyard/server/switchyard_app.py Outdated
@sabhatinas

Copy link
Copy Markdown
Contributor Author

@sabhatinas is the plan to refactor this to align with libsy design at a later date / does this take libsy into account?

Yes, I plan to migrate to libsy once I have this checked in. I want to have a clean submission of the working algorithm to reproduce results from our submissions.

@sabhatinas
sabhatinas force-pushed the sabhatinas/stage-router-scorer-corroborative-tanh branch 3 times, most recently from e0c9e7d to eb5dc07 Compare July 24, 2026 00:01
…elf-clearing signals

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…signal

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…indow

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…r pattern

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…ring signals; window=5

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…scalation signal

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
… churn

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…fault, configurable)

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…ons were real loops; spurious Opus bias)

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
@messiaen
messiaen force-pushed the sabhatinas/stage-router-scorer-corroborative-tanh branch from 96be575 to 3076cf8 Compare July 24, 2026 14:32
…jection

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
sabhatinas and others added 9 commits July 24, 2026 11:20
Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…er; replace calibration scripts with scorer skill

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…ython calls it via bindings; drop dead prompt-classification path

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…rom the PR

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
…s, drop patterns test assertions

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…run→score_staged_run, stage_score_signal, escalate/de-escalate)

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…d weights); trim verbose comments

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
@sabhatinas
sabhatinas force-pushed the sabhatinas/stage-router-scorer-corroborative-tanh branch from 7cb8a7b to 96fc5e3 Compare July 24, 2026 18:22
@sabhatinas
sabhatinas merged commit 3269dce into main Jul 24, 2026
20 checks passed
@sabhatinas
sabhatinas deleted the sabhatinas/stage-router-scorer-corroborative-tanh branch July 24, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants