Background
In benchmarks/m3/eval.sh, the ERR trap fires on evaluator failure regardless of set -e/set +e state in bash. Because cleanup() unconditionally ends with exit $exit_code, the script never reaches EVAL_EXIT=$? or the else branch after the evaluator invocation. As a result, the "✗ M3 evaluation failed (exit code: ...)" banner is unreachable.
This was identified during review of PR #4 (comment: #4 (comment)).
Functional impact
None — the crash-salvage behavior (create_bundle via ERR/EXIT trap) is already correct. This is a cosmetic issue only.
Proposed fix
Selectively suppress ERR around each uv run evaluator invocation so the script falls through to the explicit success/failure branches instead of immediately trapping:
trap '' ERR
uv run python -m benchmarks.m3.eval_m3 ...
EVAL_EXIT=$?
trap cleanup ERR
This change needs careful testing against the crash-salvage path to ensure the bundle is still created correctly on early interrupt/exception.
Requested by
@haroldship
Background
In
benchmarks/m3/eval.sh, theERRtrap fires on evaluator failure regardless ofset -e/set +estate in bash. Becausecleanup()unconditionally ends withexit $exit_code, the script never reachesEVAL_EXIT=$?or theelsebranch after the evaluator invocation. As a result, the "✗ M3 evaluation failed (exit code: ...)" banner is unreachable.This was identified during review of PR #4 (comment: #4 (comment)).
Functional impact
None — the crash-salvage behavior (
create_bundleviaERR/EXITtrap) is already correct. This is a cosmetic issue only.Proposed fix
Selectively suppress
ERRaround eachuv runevaluator invocation so the script falls through to the explicit success/failure branches instead of immediately trapping:This change needs careful testing against the crash-salvage path to ensure the bundle is still created correctly on early interrupt/exception.
Requested by
@haroldship