diff --git a/scripts/eval/README.md b/scripts/eval/README.md index 521d2a970d..b689fab9f6 100644 --- a/scripts/eval/README.md +++ b/scripts/eval/README.md @@ -132,12 +132,21 @@ Seven runs per controller: a profile run, three single-axis sweeps and three two-axis grids, all on the evaluation plant, 60 s a command with the first 8 s discarded. About twelve minutes a controller on an RTX 3060. -The RL controllers run first. Their checkpoints are checked for existence -before anything starts, but a checkpoint that exists can still fail to load, -and putting a quintic run first would hide that failure behind twenty minutes -of work. The grids carry both the combined-axis tracking and — through -`fall_time` — the stability envelope, so nothing is collected twice. Figures -land in `logs/eval/comparison/figures` as 300 dpi PNG and PDF. +The command envelope lives in two places in that script, deliberately. The +`*_MIN` / `*_MAX` / `*_STEP` variables set the sweep and grid axes, which feed +figures 03 to 07; the `PROFILE_*` variables set the amplitudes of the profile +lanes, which are figures 01 and 02 and nothing else. Widening the sweeps does +not widen the profile, and it should not: a sweep is meant to overshoot what a +controller can do so the stability envelope has an outside, while a profile is +meant to show a controller tracking a moving command, which it cannot do at a +speed it falls over at. + +The RL half runs first. The checkpoint is checked for existence before anything +starts, but a checkpoint that exists can still fail to load, and putting the +quintic half first would hide that failure behind twenty minutes of work. The grids carry +both the combined-axis tracking and — through `fall_time` — the stability +envelope, so nothing is collected twice. Figures land in +`logs/eval/comparison/figures` as 300 dpi PNG and PDF. Every environment in a sweep is a distinct command rather than a replica: the plant is deterministic and the domain randomisation is off, so replicas of one diff --git a/scripts/eval/collect_comparison.sh b/scripts/eval/collect_comparison.sh index d1637e092f..e96b4679be 100755 --- a/scripts/eval/collect_comparison.sh +++ b/scripts/eval/collect_comparison.sh @@ -247,6 +247,16 @@ WZ_MIN=${WZ_MIN:--3.0} WZ_MAX=${WZ_MAX:-3.0} WZ_STEP=${WZ_STEP:-0.05} +# Command amplitudes for the profile runs -- figures 01 and 02. +PROFILE_VX=0.35 +PROFILE_VY=0.20 +PROFILE_WZ=0.60 +# The combined lanes hold one axis while another moves, so each component is +# lower than the single-axis amplitude above it. +PROFILE_COMBINED_VX=0.25 +PROFILE_COMBINED_VY=0.15 +PROFILE_COMBINED_WZ=0.40 + # Grid axes (used for plotting) VX_GRID_MIN=${VX_GRID_MIN:--2.0} VX_GRID_MAX=${VX_GRID_MAX:-2.0} @@ -315,6 +325,12 @@ run_controller() { echo "=== ${name} (${engine}): velocity profile ===" uv run python scripts/eval/eval_velocity_profile.py \ --engine "${engine}" "${extra[@]}" \ + --profile.vx "${PROFILE_VX}" \ + --profile.vy "${PROFILE_VY}" \ + --profile.wz "${PROFILE_WZ}" \ + --profile.combined-vx "${PROFILE_COMBINED_VX}" \ + --profile.combined-vy "${PROFILE_COMBINED_VY}" \ + --profile.combined-wz "${PROFILE_COMBINED_WZ}" \ --profile.hold 6.0 --profile.ramp 1.5 --profile.replicas 4 \ --output-dir "${OUT}" --tag "profile_${name}"