Skip to content

feat(evaluation): add source-backed execution profile registry - #407

Draft
daniele21 wants to merge 1 commit into
devfrom
eval/execution-profile-registry
Draft

feat(evaluation): add source-backed execution profile registry#407
daniele21 wants to merge 1 commit into
devfrom
eval/execution-profile-registry

Conversation

@daniele21

Copy link
Copy Markdown
Owner

Summary

Adds the canonical execution-profile registry needed by EVAL-U-13 and runner preflight without inventing ad-hoc profile values.

  • derives evaluation profile refs and generation semantics directly from versioned Qwen35GenerationProfiles;
  • pairs every profile with the versioned Qwen3.5 runtime tuning source and exposes its CANDIDATE/MEASURED evidence status;
  • requires an explicit product composition tier resolver instead of guessing model tier from filename/size;
  • adds registry lookup and a preflight adapter that fails closed with UNSUPPORTED_EXECUTION_PROFILE;
  • covers source identity, direct/non-thinking and thinking semantics with focused tests.

The current Qwen3.5 runtime tuning remains candidate evidence; this PR exposes that truth rather than certifying it as measured.

@daniele21
daniele21 force-pushed the eval/execution-profile-registry branch from fa77902 to b1b4069 Compare August 24, 2026 16:17
@daniele21
daniele21 force-pushed the eval/execution-profile-registry branch from b1b4069 to f7de9e8 Compare August 24, 2026 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant