Virtual leader teleop + episode recording for the SO-101

docs/hooks/repo_links.py

@@ -51,6 +51,7 @@
     "perception/camera-calibration.md": "mote_perception/config/README.md",
     "perception/benchmarks.md": "mote_perception/benchmarks/README.md",
     "arm/index.md": "mote_arm/README.md",
+    "arm/teleop.md": "mote_arm/TELEOP.md",
     "arm/bench.md": "mote_arm/BENCH.md",
     "fleet/package.md": "mote_fleet/README.md",
     "simulation/benchmark.md": "mote_simulation/tools/benchmark/README.md",

mkdocs.yml

@@ -129,6 +129,7 @@ nav:
       - Camera driver evaluation: research/usb_cam_evaluation.md
   - SO-101 arm:
       - Overview: arm/index.md
+      - Teleop and episodes: arm/teleop.md
       - Bench validation: arm/bench.md
   - Fleet:
       - Operator runbook: fleet/README.md

mote_arm/BENCH.md

@@ -24,7 +24,7 @@ recalibration. What is still open:
 - **Does the homing offset apply to commanded goals, or only to feedback?**
   The read side is proven — positions moved by exactly the predicted delta on
   every joint, which is what "written and confirmed" checks. The write side
-  shows up on the first `arm-jog` move after calibrating: if a commanded angle
+  shows up on the first `arm-teleop` move after calibrating: if a commanded angle
   lands roughly one offset away from where you asked, `config.rad_to_counts`
   has to compensate. Try this first.
 - **Does `wrist_roll` have real stops?** It swept 5.89 rad, 94% of a turn, and
@@ -45,14 +45,14 @@ needed. One process owns that port and it is the controller_manager, so the arm
 now comes up with the base rather than instead of it.
 
 Only one thing on this bench still needs the base stopped: the tools that open
-the bus directly — `arm-check` and `arm-gains`. Run `pixi run kill` before
+the bus directly — `arm-setup check` and `arm-setup gains`. Run `pixi run kill` before
 those; they refuse to start otherwise, naming the process that holds the port.
-`arm-jog` and `arm-pose` command the controller and need no such care.
+`arm-teleop` and `arm-pose` command the controller and need no such care.
 
 ## Step 2 — enumerate + health check
 
 ```
-pixi run arm-check
+pixi run arm-setup check
 ```
 
 **Expected:** a table with all six joints — `shoulder_pan`, `shoulder_lift`,
@@ -73,7 +73,7 @@ falls. Stop the driver and the robot base first (`pixi run kill`): this tool
 opens the serial bus directly.
 
 ```
-pixi run arm-calibrate
+pixi run arm-setup calibrate
 ```
 
 ### Phase 1 — record the ranges
@@ -122,14 +122,14 @@ we assume and would catch a wrong sign encoding. Success is a count rather than
 a list because a servo that fails stops the run by name, below.
 
 **If it stops partway** it names the servos already changed and points at
-`pixi run arm-offsets restore`, which puts them back from the snapshot taken
+`pixi run arm-setup offsets restore`, which puts them back from the snapshot taken
 before the first write. Do that before re-running.
 
 If a stop reports a position that looks nothing like the expected one, check
 whether the number it read equals the offset just written in sign-magnitude
 form (`abs(offset) | 0x800` for a negative one). That means the read picked up
 the previous register's reply rather than the position — the same
-read-races-the-EEPROM-write hazard documented for `arm-gains`. Reads now clear
+read-races-the-EEPROM-write hazard documented for `arm-setup gains`. Reads now clear
 the input buffer first and the post-write check requires two agreeing reads, so
 this should not recur; if it does, the settle delay needs raising further.
 
@@ -145,7 +145,7 @@ ranges were on screen a moment ago, the limits are those pulled inward by
 
 **If the save fails** — validation rejects the document, or the file cannot be
 written — the servos have already been centred, so the arm is calibrated and the
-file is not. It says so and names `pixi run arm-offsets restore`. Do one or the
+file is not. It says so and names `pixi run arm-setup offsets restore`. Do one or the
 other before `pixi run arm`: until then the soft limits describe a frame the
 servos have stopped using.
 
@@ -155,7 +155,7 @@ outside the new limits is named — that one was taught somewhere the arm cannot
 now reach and needs a decision.
 
 ```
-pixi run arm-check          # rad column reads ~0.000 at the centred pose
+pixi run arm-setup check          # rad column reads ~0.000 at the centred pose
 ```
 
 Nothing in the repo changes — the calibration is per-robot state under
@@ -182,13 +182,37 @@ its line, so pasting the block never silently reverts a joint to a guess. A join
 whose sweep is unusable also does not get its zero moved — the usable set is
 decided before any EEPROM is touched.
 
+### The goal-range fence
+
+Phase 2 shows registers 9 and 11 on every joint — the band of goal positions
+the servo will accept — under the same confirmation as the zeros, and then
+rewrites both. A fence binds only under torque, so the sweep you just did went
+straight through it: a fence left behind describes travel the arm will refuse to
+make, silently, stopping at the same angle every time as if it had run out of
+torque. This arm spent four months in exactly that state.
+
+Each joint's fence is written immediately after its own zero, so no joint is
+ever left without one. The new band is wider than the soft limits in `arm.yaml`
+by `--margin` at each end, so the soft limits always stop the arm first and the
+fence only acts if they have gone wrong. `--skip-homing` writes nothing to the
+servos, so it reports a cutting fence rather than correcting it.
+
+The as-found bands are snapshotted to `~/.mote/arm_limits_backup.yaml` before
+the first write, so:
+
+```
+pixi run arm-setup limits show      # read-only: the band, in counts and radians
+pixi run arm-setup limits clear     # hand the whole range back, outside a calibration
+pixi run arm-setup limits restore   # put the as-found bands back
+```
+
 ### The offsets themselves
 
 ```
-pixi run arm-offsets show      # read-only: raw register, decoded value, position
-pixi run arm-offsets backup    # snapshot before doing anything risky
-pixi run arm-offsets restore   # put the snapshot back
-pixi run arm-offsets set --joint shoulder_pan --value=2027
+pixi run arm-setup offsets show      # read-only: raw register, decoded value, position
+pixi run arm-setup offsets backup    # snapshot before doing anything risky
+pixi run arm-setup offsets restore   # put the snapshot back
+pixi run arm-setup offsets set --joint shoulder_pan --value=2027
 ```
 
 `show` prints the raw register next to the decoded value deliberately: the
@@ -237,15 +261,17 @@ arm is part of the mission stack now, and this step works during a mission).
 Terminal C:
 
 ```
-pixi run arm-jog
+pixi run arm-teleop
 ```
 
+Press `m` for step mode: one 0.05 rad increment per key press, which is what
+this step wants and what `arm-jog` used to give. `?` prints the key map.
+
 For **each** joint in turn (arm supported, ready to cut power):
 
-1. Select it by number (e.g. `0` for `shoulder_pan`). The status line shows its
-   measured position and soft limits.
-2. `step 0.05` to set a small increment.
-3. Jog `+` a few times, then `-` back — watch the joint move a small amount in
+1. Find its key pair in the help (`q`/`a` is joint 1, down to `y`/`h`).
+2. Press the raise key a few times, then the lower key back — watch the joint
+   move a small amount in
    the commanded direction, and `/joint_states` (Terminal B) track it.
    - If the joint moves the **wrong way**, set `invert: true` for it in
      `robot.yaml`, rebuild, and repeat.
@@ -272,8 +298,9 @@ came from Step 3.
    `pixi run arm-pose save <name>` (read-only capture).
 3. Repeat for each pose worth reaching.
 4. `pixi run arm-pose go <name>` moves between taught poses. It prints per-joint
-   travel and asks before moving; it refuses any move over `--max-travel`
-   (0.35 rad default).
+   travel and asks before moving. Expect a large travel on the first `go` after
+   a `save`: the arm is limp until a controller claims it, so it falls to rest
+   the moment you let go of it.
 
 `pixi run arm-pose limits` prints a `joints:` block spanning every taught pose
 plus a margin. It is **not** the calibration path — it widens outward from poses
@@ -285,27 +312,27 @@ over calibrated limits expecting to widen them: re-run Step 3 for that.
 ## Step 5c — measure the position-loop gains
 
 Gains live in servo EEPROM, so they are hardware config, not software config:
-`robot.yaml`'s `arm.gains` records them and `arm-gains` reconciles the two.
+`robot.yaml`'s `arm.gains` records them and `arm-setup gains` reconciles the two.
 Choose them from a measurement, not from a datasheet default.
 
-Stop the driver first (`arm-gains` opens the bus itself) and clear the joint's
+Stop the driver first (`arm-setup gains` opens the bus itself) and clear the joint's
 path — this step moves the arm. Park the arm in a pose it holds unsupported:
 each trial drops torque briefly to write the gains, so a raised pose would sag.
 
-1. `pixi run arm-gains show` — what the servos actually hold right now.
-2. `pixi run arm-gains sweep --joint elbow_flex --kp 16,32,64,128` — steps the
+1. `pixi run arm-setup gains show` — what the servos actually hold right now.
+2. `pixi run arm-setup gains sweep --joint elbow_flex --kp 16,32,64,128` — steps the
    joint -0.2 rad under each gain and prints error, load, settling, ripple and
    reversals per trial. **Expected:** error falls as `kp` rises while `kp*err`
    stays roughly constant and load stays far below 1000 (proportional droop);
    the verdict line says so. Watch and listen at the top of the range — ripple
    over a few counts, rising `rev`, or an audible buzz is the joint hunting, and
    that gain is too high whatever its error says.
 3. Optional, for the residual droop:
-   `pixi run arm-gains sweep --joint elbow_flex --kp <chosen> --ki 0,1,2`.
+   `pixi run arm-setup gains sweep --joint elbow_flex --kp <chosen> --ki 0,1,2`.
    Do it with the joint **unloaded** first: integral action stores the effort it
    needed to hold a load, so removing that load can produce a lunge.
 4. Put the winner in `robot.yaml`'s `arm.gains`, rebuild, then
-   `pixi run arm-gains apply` to write it to all six servos.
+   `pixi run arm-setup gains apply` to write it to all six servos.
 
 The sweep restores the gains it started with and leaves the joint limp, so a run
 on its own changes nothing — step 4 is what makes a choice stick. Each run writes
@@ -325,8 +352,8 @@ criterion 2.**
 
 ## Step 7 — torque-off on exit, and a clean exit
 
-In `arm-jog`, type `quit`. **Expected:** `limping arm (deactivating
-arm_controller) and exiting...`; the arm goes back-drivable immediately. Stop
+In `arm-teleop`, press `x`. **Expected:** `teleop stopped; the arm is limp.`
+and the arm goes back-drivable immediately. Stop
 `arm` (Ctrl-C) and confirm it also logs a clean shutdown and leaves the arm
 limp. **Nothing should move on startup or shutdown.**
 
@@ -335,7 +362,7 @@ the time the process falls over, so an abort here is invisible unless looked
 for:
 
 ```
-pixi run arm-jog        # 'quit' at the prompt
+pixi run arm-teleop     # 'x' at the prompt
 echo $?                 # expect 0
 pixi run arm-pose list
 echo $?                 # expect 0
@@ -346,6 +373,51 @@ exception` on stderr. A `134` is the destroy-while-spinning abort (see README,
 "Exits and arguments"); it means the tool did its job and then crashed on the
 way out.
 
+## Step 8 — keyboard teleop, recording and replay
+
+**Rehearse it headless first** — the whole loop runs against the mock follower
+with no hardware at all, and a failure there is a software bug, not a bench one:
+
+```
+pixi run arm-teleop-test
+```
+
+Then, on the arm, two terminals:
+
+```
+pixi run arm            # or `pixi run launch`, if you want the camera
+pixi run arm-teleop     # '?' prints the keys
+```
+
+Three of these are observations no test can make — the arm stopping at a limit,
+halting on a released key, going limp on panic — which is the whole reason a
+human is here. Keep a hand near SPACE throughout.
+
+1. **It follows.** Hold one joint's key. The arm moves smoothly, not in steps.
+2. **It stops at the soft limit.** Keep holding past the limit. It stops there
+   and goes no further. Check the angle it stopped at is the limit `?` printed:
+   stopping short of that is the servo's own fence, not the soft limit — see
+   [the goal-range limits](README.md#the-servos-own-goal-range-limits-which-are-not-the-soft-limits).
+3. **Releasing stops it.** Drive, then let go mid-move. It halts within a
+   fraction of a second, and does not coast on to where it was heading.
+4. **Panic drops torque.** Press SPACE. The arm goes limp — back-drivable by
+   hand — and stays limp while you keep pressing joint keys.
+5. **Clearing resumes without a jump.** Press `z`, then drive again. It picks up
+   from where the arm is, not from where the command had got to.
+6. **Step mode.** Press `m`, then tap a joint key: exactly one 0.05 rad
+   increment per press, and *holding* the key steps once rather than repeatedly.
+
+Then record, check, and replay:
+
+```
+pixi run arm-record -- --task "move the arm through a simple motion" --dataset bench
+python3 mote_arm/test/teleop_loop/check_capture.py ~/.mote/episodes/bench
+pixi run arm-replay -- ~/.mote/episodes/bench --episode 0   # stop teleop first
+```
+
+The export is off-board and verifies itself by loading the dataset back through
+LeRobot's own API. Full workflow and design: [TELEOP.md](TELEOP.md).
+
 ---
 
 ## Sign-off checklist
@@ -362,12 +434,12 @@ Already verified on the robot (2026-07-25):
 - [x] enabling torque holds the current pose instead of snapping
 - [x] `min`/`max` in `robot.yaml` derived from taught poses, not guessed
       (superseded by the calibration pass below — provisional until it runs)
-- [x] servo gains applied and verified (`pixi run arm-gains`), full
+- [x] servo gains applied and verified (`pixi run arm-setup gains`), full
       home<->reachy move completed both ways
 - [x] gains chosen from a sweep, not a default: Kp=64 applied to all six,
       residual on the full move now 0.012-0.028 rad (2026-07-28)
 
-- [x] **Step 3: one full `pixi run arm-calibrate` pass on the real arm**
+- [x] **Step 3: one full `pixi run arm-setup calibrate` pass on the real arm**
       (2026-07-28), saved to `~/.mote/arm.yaml`; taught poses migrated
       automatically rather than re-taught
 - [x] every servo's homing offset written and confirmed, and no joint reports an
@@ -380,16 +452,19 @@ Still open:
 - [ ] `shoulder_pan` can be commanded to 0 rad (its packaged band still excludes
       its own zero; the calibrated band on the arm does not)
 - [ ] the offset applies to commanded goals, not only to feedback — the first
-      `arm-jog` move settles it
+      `arm-teleop` move settles it
 - [ ] **the arm moving under ros2_control on the real robot** — everything
       about the fold is verified against a simulated bus
       (`mote_hardware/test/test_arm_bus.cpp`), not against servos: confirm
-      `arm-jog` still moves `elbow_flex` in the commanded direction, that the
+      `arm-teleop` still moves `elbow_flex` in the commanded direction, that the
       soft limits still hold, and that activating `arm_controller` takes hold
       without a snap
 - [ ] **the arm moving while the wheels are driving** — the point of the fold.
-      `pixi run robot`, drive a short goal, and jog the arm at the same time;
+      `pixi run robot`, drive a short goal, and teleop the arm at the same time;
       watch for wheel-odometry glitches that would mean the bus is oversubscribed
+- [ ] step 8: teleop, record, export/inspect and replay on the arm
+      (BENCH.md step 8) — verified headless against the mock
+      control stack, but not yet on hardware
 - [ ] the other five joints jogged and direction-checked (`invert`)
 - [ ] re-check the gain with a payload on the gripper — the sweep only measures
       an unloaded static hold, which is why Kp=64 was taken over a better-scoring