Skip to content

VALUE-04A: distinguish decision-grade device evidence - #148

Draft
daniele21 wants to merge 7 commits into
devfrom
feat/value-04-device-evidence-prep
Draft

VALUE-04A: distinguish decision-grade device evidence#148
daniele21 wants to merge 7 commits into
devfrom
feat/value-04-device-evidence-prep

Conversation

@daniele21

Copy link
Copy Markdown
Owner

Parent: #145

Outcome

Prevent contextual telemetry from being promoted to model-resource evidence merely because it has HOST/RUNTIME provenance.

This isolated VALUE-04A slice introduces the canonical application classification boundary for decision evidence:

  • black-box CLIENT performance remains attributable to the frozen execution and can participate after runtime comparability;
  • current host-stdlib-v1 telemetry remains context-only because it measures the Performance Lab process/host rather than attributable model-server resource consumption;
  • current local-llm-server-status-v1 activity/concurrency telemetry remains context-only;
  • HOST/RUNTIME resource metrics require explicit opt-in from an owning versioned telemetry contract before they can establish the Resources dimension as evidence-available;
  • zero completed candidate Runs no longer become evidence-available through all([]) semantics.

Campaign compatibility now exposes an additive evidence_note explaining why a dimension is unavailable. The TypeScript boundary accepts the additive field without forcing existing mocked payloads to manufacture it.

Files

  • src/performance_lab/application/device_evidence.py
  • src/performance_lab/application/campaign_queries.py
  • src/performance_lab/application/campaign_models.py
  • frontend/src/api/campaign-types.ts
  • tests/test_device_evidence.py
  • tests/test_campaign_queries.py

Invariants

  • Quality / Performance / Resources remain separate.
  • Unknown/unattributable resource evidence remains unavailable, never zero.
  • Hardware/runtime fingerprint compatibility is still checked before any future resource comparison.
  • The current strict-quality-dominance recommendation policy is unchanged; VALUE-04C will version policy semantics separately.
  • No guessed sensor or metric contract is introduced.

Dependency / publication gate

Prepared from dev@0b9c376f2dbcbb167f8df8488374c1b0c5d0ace2 under the adopted repo-template-sw 0.9.1 baseline. Keep draft and do not integrate until VALUE-02D #132 PASS / VALUE-02 DONE.

Evidence boundary

Iteration CI proves deterministic classification/projection semantics only. Representative resource claims remain VALUE-04D / REAL_ENVIRONMENT evidence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant