Skip to content

Safety: correctness gate for swapped local/compiled intent checkpoints (sensitivity under-estimation downgrades a control) #146

Description

@se-jo-ma

Context (real unguarded security path — exists today). An operator can point analysis.mode=llm-only at any fine-tuned checkpoint via LocalInferenceProviderSpec and route every policy decision through it, yet the only safety net is health_check (a deliberate no-op for local, local_provider.py:59-71) and the determinism harness — which checks self-consistency, not correctness. Because estimated_sensitivity feeds the deny/scope decision, a checkpoint that under-estimates sensitivity downgrades a security control with no operator signal.

Proposal. Document the gap, then add a correctness harness alongside the determinism one.

  • The locked fixture tests/fixtures/llm_determinism/intent_prompts_100.jsonl already carries ground-truth labels (expected_data_types/estimated_sensitivity). Replay it through the candidate provider's real client (not the fake) and score agreement on the security-critical fields (data_types_needed, estimated_sensitivity).
  • Asymmetric penalty for sensitivity UNDER-estimation (mirroring intent_v1.txt rule 6's minimal-interpretation bias).
  • Surface a pass/fail threshold operators must clear before flipping to llm-only.

Where.

  • local_provider.py:59-71 (health_check no-op) + :38-54 (verbatim-model acceptance); tests/integration/test_llm_determinism_harness.py (determinism-only); model on rkm/validator/scoring.py:score (which has no correctness analog for providers).

Acceptance criteria.

  • Correctness harness replays the labeled fixture through the candidate's real client; reports per-field agreement.
  • Sensitivity under-estimation penalized asymmetrically; a documented threshold gates llm-only promotion.

Caveat. Too-strict blocks good checkpoints; too-loose gives false assurance. Anchor on the deterministic fixture labels (the graph is the oracle), not frontier-generated labels, to avoid the closed-loop-judge trap.


Source: arXiv:2605.22502v1 — "Compiling Agentic Workflows into LLM Weights" (subterranean agents). Distilled from arXiv-research/2605.22502v1/analysis.md; file refs verified against current main by the analysis pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priority: load-bearing v1 gaparea/coreBroker core, CLIPS routing, session stateenhancementNew feature or requestsecuritySecurity or attestation gapsize/M<2 days: multi-file feature

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions