Skip to content

feat: add optimize-sorting Python workflow with gap detection and I/O profiling - #1435

Closed
colehurwitz wants to merge 2 commits into
mainfrom
factory/run-a26d6e43
Closed

colehurwitz wants to merge 2 commits into
mainfrom
factory/run-a26d6e43

Conversation

@colehurwitz

Copy link
Copy Markdown
Collaborator

Changes

  • New file: .factory/workflows/optimize_sorting.py — Project-local Python workflow definition encoding the full optimize-sorting workflow graph (31 nodes, 39 edges) with two new capabilities:

    • Gap Detection: Enhanced researcher_profile_pipeline prompt with gap analysis that compares speed_seconds.mean to the sum of stage means, using a 10% threshold, and outputs a gap_analysis JSON section identifying unexplained overhead
    • I/O Profiling: Modified FnNode commands for lock_baseline, run_benchmark_t*, and confirm_benchmark_t* to capture /proc/self/io before and after benchmark runs, computing read/write byte deltas with graceful fallback (available: false) when procfs is unavailable
  • New file: tests/test_optimize_sorting_workflow.py — 61 tests covering:

    • Workflow loads correctly with expected node/edge counts
    • All 31 nodes exist with correct types (8 AgentNode, 11 FnNode, 12 GateNode)
    • Gate nodes have evaluator commands; agent nodes have post_checks
    • Gap detection keywords present in researcher prompt
    • I/O capture logic present in benchmark scripts with graceful fallback
    • Archivist nodes use haiku model with 300s timeout
    • Graph validation passes with no structural issues
    • Gap arithmetic threshold tests

… profiling

- Create .factory/workflows/optimize_sorting.py encoding the full 31-node,
  39-edge optimize-sorting workflow graph as a project-local Python workflow
- Change 1 (Gap Detection): Enhanced researcher_profile_pipeline prompt with
  gap analysis comparing speed_seconds.mean to sum of stage means (10%
  threshold), outputting gap_analysis JSON section
- Change 2 (I/O Profiling): Modified FnNode commands for lock_baseline,
  run_benchmark_t*, and confirm_benchmark_t* to capture /proc/self/io
  before/after benchmark runs with graceful fallback when unavailable
- Add 61 tests covering workflow structure, node types, gap detection prompts,
  I/O capture in benchmark scripts, graceful fallback, and graph validation
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

Sentrux Quality Report

Absolute

Scanning ....
[scan] git ls-files: 631 total, 617 kept, 14 dropped (ext:14, meta:0, big:0)
[build_project_map] 617 files, 112 unique dirs, 103 cache misses, 5.4ms
[resolve] 1162 resolved, 1575 unresolved (of 2737 total specs)
[resolve_imports] project_map 5.6ms, suffix_idx 1.1ms, suffix_resolve 19.3ms, total 25.9ms
[build_graphs] 617 files | maps 2.3ms, imports 26.1ms, calls+inherit 6.5ms, total 34.9ms | 1161 import, 8388 call, 1 inherit edges
sentrux check — 3 rules checked

Quality: 4447

✗ [Error] max_cc: 6 function(s) exceed max cyclomatic complexity of 30
    factory/cli/_ceo_helpers.py:_validate_ceo_flags (cc=43)
    factory/cli/_ceo_helpers.py:_execute_ceo (cc=43)
    examples/chess_evolve.py:main (cc=33)
    factory/cli/_task_builder.py:_build_ceo_task (cc=32)
    factory/cli/run.py:cmd_run (cc=32)
    factory/outer_loop/mutations.py:validate_and_repair (cc=31)

✗ 1 violation(s) found

Diff (vs base branch)

Scanning ....
[scan] git ls-files: 631 total, 617 kept, 14 dropped (ext:14, meta:0, big:0)
[build_project_map] 617 files, 112 unique dirs, 103 cache misses, 5.8ms
[resolve] 1162 resolved, 1575 unresolved (of 2737 total specs)
[resolve_imports] project_map 5.9ms, suffix_idx 1.3ms, suffix_resolve 18.8ms, total 26.0ms
[build_graphs] 617 files | maps 2.3ms, imports 26.2ms, calls+inherit 6.8ms, total 35.4ms | 1161 import, 8388 call, 1 inherit edges
sentrux gate — structural regression check

Quality:      4447 -> 4447
Coupling:     0.80 → 0.80
Cycles:       4 → 4
God files:    3 → 3

Distance from Main Sequence: 0.35

✓ No degradation detected

The optimize-sorting workflow's SKILL.md was stale after the Python
workflow definition was added (93d2cd4). The export-skills output at
workflow-optimize-sorting/ now includes gap detection and I/O profiling
content matching the workflow definition.

Also refreshed the skills/ cache (gitignored) and reverted unrelated
workflow-design-v2 SKILL changes.

Generated files exceed 500-line limit (SKILL.md: 2521 lines,
SKILL.annotations.yaml: 805 lines) — splitting would break export-skills
tooling.
@colehurwitz

Copy link
Copy Markdown
Collaborator Author

SKILL.md Regeneration Fix

Added the regenerated workflow-optimize-sorting/SKILL.md and SKILL.annotations.yaml — these were missing from the original commit.

Before: skills cache SKILL.md (59KB, 841 lines) — no gap detection or I/O profiling content
After: regenerated SKILL.md (104KB, 2521 lines) — includes Gap Detection (1 match) and capture_io (42 matches)

Also reverted unrelated workflow-design-v2/ SKILL changes and refreshed the local skills/ cache.

@colehurwitz

Copy link
Copy Markdown
Collaborator Author

Superseded by unified PR — see #1443

@colehurwitz colehurwitz closed this Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant