Skip to content

Serialize the full OptKnob spec into the IR + fix KNOB_MUTATE - #1499

Merged
akashgit merged 2 commits into
mainfrom
knob-specs-knob-mutate
Sep 11, 2026
Merged

akashgit merged 2 commits into
mainfrom
knob-specs-knob-mutate

Conversation

@akashgit

Copy link
Copy Markdown
Owner

What

Two changes that make KNOB_MUTATE (the highest-weight outer-loop operator, 0.25) actually contribute to the swarm:

  1. knob_specs in the IRWorkflow now carries a full knob_specs table (kind / node_id / default / bounds / expandable / expansion_hint / description) beside knob_values / knob_bounds. to_dict() serializes it, from_dict() reconstructs it, and Package.compile() populates it.

  2. KNOB_MUTATE survives mutation + novelty filtering — two defects silently discarded every knob-only mutation:

    • _deep_copy_workflow dropped knob_specs (and task / declared_capabilities), so children lost their OptKnob declarations.
    • structural_hash excluded knob_values and NoveltyFilter.is_novel rejected any workflow whose graph-edit-distance to an archived candidate was below threshold — even when only knob values differed. Now structural_hash includes knob_values, and the near-duplicate check only applies when knob_values match.

Why

Without these, KNOB_MUTATE was selected ~25% of the time but its results were always rejected as "duplicates", so the operator was dead weight.

Tests

  • pytest tests/test_outer_loop/test_similarity.py tests/test_outer_loop/test_mutations.py tests/test_outer_loop/test_engine.py tests/test_outer_loop/test_seed_diversity.py tests/test_outer_loop/test_reflector.py174 passed.
  • A downstream harness confirms all 8 operators now fire and the archive/reflection/evaluate path works end to end.

Two defects stopped the highest-weight operator (KNOB_MUTATE, 0.25) from ever
contributing to the swarm:

- _deep_copy_workflow dropped knob_specs (and task/declared_capabilities) when
  copying a workflow for mutation, so children lost their OptKnob declarations.
- structural_hash excluded knob_values and NoveltyFilter.is_novel rejected any
  workflow whose graph-edit-distance to an archived candidate was below
  threshold, even when only knob values differed, so every knob-only mutation
  was discarded as a near-duplicate.

Include knob_values in structural_hash, preserve knob_specs across deep copies,
and only apply the edit-distance near-duplicate check when knob_values match.
@github-actions

Copy link
Copy Markdown

Sentrux Quality Report

Absolute

Scanning ....
[scan] git ls-files: 651 total, 637 kept, 14 dropped (ext:14, meta:0, big:0)
[build_project_map] 637 files, 108 unique dirs, 100 cache misses, 2.6ms
[resolve] 1296 resolved, 1680 unresolved (of 2976 total specs)
[resolve_imports] project_map 2.7ms, suffix_idx 0.8ms, suffix_resolve 12.4ms, total 15.9ms
[build_graphs] 637 files | maps 1.6ms, imports 16.1ms, calls+inherit 4.9ms, total 22.6ms | 1295 import, 10472 call, 4 inherit edges
sentrux check — 3 rules checked

Quality: 4415

✗ [Error] max_cc: 9 function(s) exceed max cyclomatic complexity of 30
    factory/workflow/executor.py:_execute_data (cc=53)
    factory/outer_loop/reflector.py:_extract_eval_patterns (cc=46)
    factory/cli/_ceo_helpers.py:_validate_ceo_flags (cc=43)
    factory/cli/_ceo_helpers.py:_execute_ceo (cc=43)
    factory/outer_loop/mutations.py:_try_mutation (cc=40)
    examples/chess_evolve.py:main (cc=33)
    factory/cli/_task_builder.py:_build_ceo_task (cc=32)
    factory/outer_loop/reflector.py:_llm_reflect (cc=32)
    factory/outer_loop/mutations.py:validate_and_repair (cc=31)

✗ 1 violation(s) found

Diff (vs base branch)

Scanning ....
[scan] git ls-files: 651 total, 637 kept, 14 dropped (ext:14, meta:0, big:0)
[build_project_map] 637 files, 108 unique dirs, 100 cache misses, 2.7ms
[resolve] 1296 resolved, 1680 unresolved (of 2976 total specs)
[resolve_imports] project_map 2.8ms, suffix_idx 0.9ms, suffix_resolve 14.7ms, total 18.4ms
[build_graphs] 637 files | maps 1.7ms, imports 18.6ms, calls+inherit 5.5ms, total 25.8ms | 1295 import, 10472 call, 4 inherit edges
sentrux gate — structural regression check

Quality:      4415 -> 4415
Coupling:     0.81 → 0.81
Cycles:       3 → 3
God files:    3 → 3

Distance from Main Sequence: 0.35

✓ No degradation detected

@akashgit
akashgit merged commit f0d28a0 into main Sep 11, 2026
3 checks passed
@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.74%. Comparing base (4f83d6c) to head (4295cf2).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1499      +/-   ##
==========================================
+ Coverage   83.62%   83.74%   +0.12%     
==========================================
  Files         225      225              
  Lines       25447    25757     +310     
  Branches     4138     4209      +71     
==========================================
+ Hits        21280    21571     +291     
- Misses       3202     3204       +2     
- Partials      965      982      +17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant