chore: align env_vars/env_vars_release key order across all build configs

[Jammy2211]

Overview

Every buildable workspace carries two env profiles under config/build/: env_vars.yaml (the smoke profile driving the per-PR CI gate) and env_vars_release.yaml (the release-fidelity profile). They are meant to be read side by side — the release file's whole job is to say "same knobs, different values". Today they cannot be, because the defaults: blocks list the same keys in different orders.

autolens_workspace is the clearest case: smoke runs TEST_MODE, SKIP_FIT_OUTPUT, SKIP_VISUALIZATION, SKIP_CHECKS, SMALL_DATASETS, DISABLE_JAX, FAST_PLOTS, while release runs TEST_MODE, SMALL_DATASETS, FAST_PLOTS, SKIP_FIT_OUTPUT, SKIP_VISUALIZATION, SKIP_CHECKS, DISABLE_JAX, SKIP_WORKSPACE_VERSION_CHECK. Working out which knob actually differs between profiles means scanning, not diffing.

This applies one canonical key order to every config/build/env_vars*.yaml in the workspace, so the two profiles line up row-for-row and a diff shows only genuine value/comment differences. It is provably a no-op: defaults is applied by iterating a plain dict (PyAutoHands/autobuild/env_config.py:68), so its order cannot affect the resolved environment. Only overrides: is order-sensitive, and overrides: is not touched.

Plan

• Adopt one canonical key order for the defaults: block, shared by both profiles in every workspace.
• Reorder the five env_vars_release.yaml files that actually deviate from it.
• Align the ragged trailing-comment column (column 40) in the remaining twelve config files so the pairs diff cleanly.
• Add no keys, remove no keys, change no values, keep each key's existing comment text, and leave every overrides: block untouched.
• Verify by resolving both profiles for every script before and after and diffing the results — the diff must be empty in every repo.

Detailed implementation plan

Affected Repositories

• autolens_workspace (primary)
• autogalaxy_workspace
• autofit_workspace
• autolens_workspace_test
• autogalaxy_workspace_test
• autofit_workspace_test
• HowToLens
• HowToGalaxy
• HowToFit
• euclid_strong_lens_modeling_pipeline
• autocti_workspace_test

Branch Survey

Repository	Current Branch	Dirty?
./autolens_workspace	main	clean
./autogalaxy_workspace	main	clean
./autofit_workspace	main	clean
./autolens_workspace_test	main	clean
./autogalaxy_workspace_test	main	clean
./autofit_workspace_test	main	clean
./HowToLens	main	clean
./HowToGalaxy	main	clean
./HowToFit	main	clean
./euclid_strong_lens_modeling_pipeline	main	1 modified (test_report.md, unrelated/unregistered)
./autocti_workspace_test	main	clean

No worktree claim in active.md covers any of these repos (worktree_list_claimed checked).

Suggested branch: feature/env-vars-profile-key-order

Canonical order

Union of every key currently in use, ordered smoke-first:

PYAUTO_TEST_MODE
PYAUTO_SKIP_FIT_OUTPUT
PYAUTO_SKIP_VISUALIZATION
PYAUTO_SKIP_CHECKS
PYAUTO_SMALL_DATASETS
PYAUTO_DISABLE_JAX
PYAUTO_FAST_PLOTS
PYAUTO_SKIP_WORKSPACE_VERSION_CHECK
JAX_ENABLE_X64
MPLBACKEND
NUMBA_CACHE_DIR
MPLCONFIGDIR

Per-file contract:

• list only the keys the file already has — no keys added or removed;
• no values changed; each key keeps its own existing comment text;
• present keys appear in the canonical relative order;
• trailing # comment column aligned at column 40 within each defaults: block;
• overrides: untouched (ordered list of pattern rules — order is load-bearing there).

Implementation Steps

1. Snapshot: for each repo, resolve both profiles for every script from an empty base and dump to JSON, reusing resolve_clean(script, cfg) in PyAutoHands/autobuild/validate_env_profiles.py.
2. Reorder the defaults: block of the five deviating release profiles:
   • autolens_workspace/config/build/env_vars_release.yaml
   • autogalaxy_workspace/config/build/env_vars_release.yaml
   • autofit_workspace/config/build/env_vars_release.yaml
   • autolens_workspace_test/config/build/env_vars_release.yaml (FAST_PLOTS/DISABLE_JAX swap)
   • autogalaxy_workspace_test/config/build/env_vars_release.yaml (same swap)
3. Comment-column alignment only in the remaining twelve config/build/env_vars*.yaml (the six smoke siblings, autofit_workspace_test/config/build/env_vars_release.yaml, and the standalone configs in HowToLens / HowToGalaxy / HowToFit / euclid_strong_lens_modeling_pipeline / autocti_workspace_test).
4. Re-dump the resolved env and diff against the step-1 snapshot — must be empty per repo. A non-empty diff means a key was dropped, renamed, or a value changed; treat as a hard failure.
5. Run validate_env_profiles.py against each edited repo (exit-1 tier catches unparseable YAML, non-mapping profiles, malformed overrides).
6. Sanity check the point of the exercise: diff each repo's env_vars.yaml against its env_vars_release.yaml and confirm the surviving hunks are value/comment differences only, with keys lining up row-for-row.
7. Pre-flight git diff --stat in each worktree before committing — nothing but config/build/env_vars*.yaml may appear.

No script execution and no CI dispatch is needed to prove correctness; steps 1, 4 and 5 are decidable from config alone.

Key Files

• <repo>/config/build/env_vars.yaml — the smoke profile (per-PR CI gate); 11 of these.
• <repo>/config/build/env_vars_release.yaml — the release-fidelity profile; 6 of these.
• PyAutoHands/autobuild/env_config.py:68 — applies defaults by dict iteration; the reason this reorder is a no-op.
• PyAutoHands/autobuild/validate_env_profiles.py — resolve_clean() (empty-base resolution) + the exit-1 config-error tier; the verification harness. Not modified.

Notes on classification

The Brain Feature Agent returned too-large → split-into-4-phases, scored off the 11-repo count alone. Overridden by judgment: this is a comment-column tidy plus a key reorder in five files, with no API surface and a provable no-op guarantee. Single task, one small PR per repo.

Original Prompt

Click to expand starting prompt

Make env_vars and env_vars_release same order, e.g. the two configs are hard to compare side by isde cause the top entries are in different orders,

Formalised as PyAutoMind/draft/maintenance/workspaces/env_vars_profile_key_order.md.

[Jammy2211]

Shipped — 10 PRs open, awaiting merge

All 17 config/build/env_vars*.yaml files across the workspace now share one canonical defaults: key order with an aligned comment column. autocti_workspace_test needed no change (already canonical, no inline comments), so 10 repos get PRs rather than 11.

Repo	PR
autolens_workspace	#322
autogalaxy_workspace	PyAutoLabs/autogalaxy_workspace#148
autofit_workspace	PyAutoLabs/autofit_workspace#110
autolens_workspace_test	PyAutoLabs/autolens_workspace_test#202
autogalaxy_workspace_test	PyAutoLabs/autogalaxy_workspace_test#84
autofit_workspace_test	PyAutoLabs/autofit_workspace_test#67
HowToLens	PyAutoLabs/HowToLens#49
HowToGalaxy	PyAutoLabs/HowToGalaxy#39
HowToFit	PyAutoLabs/HowToFit#28
euclid_strong_lens_modeling_pipeline	PyAutoLabs/euclid_strong_lens_modeling_pipeline#33

All carry pending-release (verified).

Verification

• Resolved-env diff before/after: identical in all 11 repos. Every script × both profiles, resolved from an empty base via resolve_clean(). For a config-only change with zero scripts/ edits this is strictly stronger than a smoke run — it proves every script's environment is unchanged rather than sampling a subset. Smoke was therefore not re-run as the gate.
• validate_env_profiles.py verdicts byte-identical to main. The 5 env_vars_release.yaml: missing errors (HowToLens/HowToGalaxy/HowToFit/euclid/autocti carry only the smoke profile) and the 42 PYAUTO_DISABLE_JAX derivation-rule warnings are all pre-existing and untouched.
• Every repo's git diff --stat shows nothing but config/build/env_vars*.yaml.

Heart gate

Shipped under the corrective-PR exception, explicitly authorized by the human against the verbatim RED reason:

release validation FAILED (stage integrate)

(verdict: red, score: 40, ts: 2026-07-23T14:11:21Z; yellow: workspace validation not passing (13 failed, 2026-07-21T19-05-22Z), 33 stale parked script(s).) The RED is unrelated to this diff — it is integrate:fail on v2026.7.22.1.dev67201 from the 2026-07-22 release-validation run. Per the exception this stops at PR-open: no merges, no release.

Note for the reviewer

Path correction to the issue body above: the modules are PyAutoHands/autohands/env_config.py and PyAutoHands/autohands/validate_env_profiles.py — the issue says autobuild/, which predates the autobuild→autohands rename.

Follow-up found, not fixed here

The alignment exposed a genuine config gap: autofit_workspace_test/config/build/env_vars_release.yaml omits PYAUTO_SMALL_DATASETS and PYAUTO_FAST_PLOTS entirely, while the release-profile header doctrine (see the autolens_workspace release header) requires every var the profile cares about to carry an explicit value rather than being left absent — an absent key silently falls through to whatever the calling process already had. Worth its own prompt; deliberately out of scope for a no-op reordering.