Conversation
Local changes to Webots sim setup, fastdds profiles, walk/vision/planning configs and tooling, preserved before importing robocup2026 branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
K1Camera.hpp auto-merged keeping the local Hcw_mutex/latest_Hcw members while the cpp took the robocup2026 timestamp-matched Hcws buffer; the old members are now unused. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment out VisualMesh/GreenHorizonDetector/FieldLineDetector as in robocup.role: Orin NX has no OpenCL so the mesh chain never ran on the robot, and FieldLocalisationNLopt consumes Yolo's FieldIntersections and Goals instead of FieldLines. Keeps sim vision identical to hardware for policy/behaviour transfer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The module-level list() platform helper shadows the builtin, so list(sorted(names)) on the multi-tag warning path raised TypeError. Use plain sorted() instead. Also pass the missing username argument to selected() in build(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Booster SDK ships a prebuilt libfastrtps.so.2.13 with a NEEDED entry for libtinyxml2.so.9, but the tinyxml2 layer only builds the static archive, so binaries fail to load at runtime. Add a shared tinyxml2 build in the late layer section to keep earlier layers cached. Also pass the missing username/uid/reset arguments to platform.build() on the ./b run --rebuild path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
platform.build() renames generic to generic_k1 when tagging, but target.py tagged from the unsuffixed name. When a stale :generic tag existed this silently selected the wrong (old) image; with no stale tag it fails with 'No such image'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
alsa-lib installs to the Jetson targetfs prefix regardless of platform, so FindALSA (required since whistle detection) fails on generic builds even though the built libraries are host-arch. Copy headers and libs to /usr/local where the generic toolchain searches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Not just alsa-lib: lame, libbacktrace, gperftools and the OpenCL loader also install to the Jetson targetfs prefix on every platform, so each one fails find_package in turn on generic. Blanket-copy include/ and lib/ (no-clobber) instead of fixing them one at a time. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TensorRT 11 (generic image) removed platformHasFastFp16 and the weakly-typed kFP16 builder flag; guard on NV_TENSORRT_MAJOR so the same code builds against Jetson TensorRT 8/10 and generic 11. VisualMesh probes OpenCL when its image layer builds and bakes VISUALMESH_DISABLE_OPENCL into its exported cmake config; copy the targetfs-stranded CL headers and loader into /usr/local first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The targetfs prefix has no include/CL directory (headers come from the opencl-c-headers apt package), so copying it failed the layer. Only the loader library is stranded and needed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tester emits FallRecovery at hardcoded priority 2 (as Soccer does, with play at 1), but find_ball ran at 4, so a fallen robot's walk branch kept the motors and GetUp never executed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tester never emitted FallRecovery, so a fallen robot in test behaviour kept walking and GetUp never ran. Port the upstream robocup2026 change (ec7d858): fall_recovery_priority from Tester.yaml, emitted above the behaviour tasks per the Soccer convention (recovery 2, play 1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BoosterGetUp was only sent on NEW_TASK, then the provider waited for IS_READY. A failed get-up or an immediate re-fall returns the state to HAS_FALLEN with the task still running: no Done, no re-request, no resend — the robot stays down forever. Re-request on a configurable cooldown while HAS_FALLEN persists; IS_GETTING_UP never retriggers it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jpptm
force-pushed
the
jmontano/nusim-local
branch
from
July 19, 2026 01:41
0e33438 to
2948a7f
Compare
Minimal 13-term scratch reward (2.1B steps) replaces v18. NUSim lying_front: stands in ~2s, walks stably after handoff. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…0-obs) Preserve the deployable ONNX artifacts and the kick module in-repo: the mujoco_playground training code and all checkpoints lived only on a RunPod pod whose volume is gone, so these exported policies are now irreplaceable. - K1KickPolicy module: v5 side-sweep policy (80-obs = gyro/grav/kick_dir/ ball/gait-phase/q/dq/last_action), gait-phase clock, ball-side sweep pick, and a walk->kick handoff cross-fade (fixes the freeze/topple at the switch). - Deployed: getup v21 984M, walk v1 617M (already tracked), kick v5 926M. - Fallback getup/kick ONNX committed alongside. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Snapshot of the reconstructed B-Human-reward kick run at 360M (reward ~53.9): stable + balanced (termination ~0, support_foot high) but contact still weak (contact_retain 0, low ball speed). Kept as an early copy; deployed model stays v5. Longer warm-continued run in progress.
ZeoNyph
requested changes
Jul 30, 2026
ZeoNyph
left a comment
ZeoNyph
left a comment
Collaborator
There was a problem hiding this comment.
in general it looks good! might wait till you test the policies on the K1 before I think about merging it into main
i'd also try to clean up some of these comments; the Claude slop is insane lmao
Collaborator
There was a problem hiding this comment.
we don't use SensorFilter anymore; just K1Sensors
Collaborator
There was a problem hiding this comment.
probably worth looking at my PR to fix generic, as that sets up these dependencies properly
Collaborator
There was a problem hiding this comment.
does this run on the K1? I notice you're using OpenVINO to load and get inference from the model, which I've tried to get working in the past but have failed
I'd also try and setup TensorRT in here as well since that will 100% work on the K1
Author
There was a problem hiding this comment.
Apparently it had something to do with the ENABLE_INTEL_CPU=OFF flag, which was meant to be on.
For some reason that is beyond my understanding, the ENABLE_INTEL_CPU flag is not just for intel processors but for risc v and arm processors as well. So this pr should work with cpu only. Given that the models are tiny we might be able to get away with just cpu inference
" - ENABLE_INTEL_CPU=ON both platforms → a CPU device exists at all.
- ENABLE_ARM_COMPUTE_CMAKE now ON only for orinnx → ACL is the aarch64 backend of that plugin; building ACL on x86 was pointless.
Old settings were INTEL_CPU=OFF + ARM_COMPUTE=ON always — i.e. ACL built but no plugin consuming it, and zero CPU device on either target.
Confirmed in the current image:
libopenvino_auto_plugin.so libopenvino_hetero_plugin.so libopenvino_auto_batch_plugin.so
No *_cpu_plugin.so at all. That's exactly the Device with "CPU" name is not registered error my run.py overlay hack worked around.
"
Collaborator
There was a problem hiding this comment.
see my comment on the GetUp policy
Collaborator
There was a problem hiding this comment.
also fixed in my PR for generic target ^-^
Collaborator
There was a problem hiding this comment.
this is also fixed in my PR
Comment on lines
+207
to
+225
| # The image's OpenVINO was built with ENABLE_INTEL_CPU=OFF (no CPU inference | ||
| # device), which breaks Yolo and the locomotion policy skills | ||
| # ("Device with \"CPU\" name is not registered" / "CompiledModel was not | ||
| # initialized"). If the official runtime has been extracted to | ||
| # ~/.cache/ov_overlay (preferred: survives reboots) or /tmp/ov_overlay | ||
| # (see NUSim docs/K1_MUJOCO_SETUP.md), mount it over the baked one and make the | ||
| # loader prefer it. Check for the CPU plugin itself, not just the directory — | ||
| # a docker-created empty mount stub at /tmp/ov_overlay must not count. | ||
| ov_env = [] | ||
| for ov_overlay in ( | ||
| os.path.expanduser("~/.cache/ov_overlay/runtime/lib/intel64"), | ||
| "/tmp/ov_overlay/runtime/lib/intel64", | ||
| ): | ||
| if os.path.isfile(os.path.join(ov_overlay, "libopenvino_intel_cpu_plugin.so")): | ||
| print(f"Auto-mounting OpenVINO CPU overlay from {ov_overlay}") | ||
| docker_args.extend(["--volume", f"{ov_overlay}:/usr/local/runtime/lib/intel64:ro"]) | ||
| ov_env = ["--env", "LD_LIBRARY_PATH=/usr/local/runtime/lib/intel64"] | ||
| break | ||
| docker_args.extend(ov_env) |
Collaborator
There was a problem hiding this comment.
this feels a bit hacky, but I'll have to look into building this only for generic target
Takes ZeoNyph's generic-target fixes and drops the local workarounds they supersede: - Dockerfile DEP_PREFIX/DEP_TOOLCHAIN installs source deps to /usr/local for generic (was /l4t/targetfs), so the two cp-from-targetfs layers are removed. - OpenVINO now builds with ENABLE_INTEL_CPU=ON and ARM_COMPUTE only for orinnx. - jobs build-arg threaded through target.py/platform.py/run.py/defaults.py. - TensorRT.cpp NV_TENSORRT_MAJOR guards taken from #38 (same logic). Kept local-only: shared tinyxml2 build (Booster SDK libfastrtps needs libtinyxml2.so.9), cache-from build_platform fix, sorted() fix in platform.selected, and the IPC/FastDDS/OpenVINO-overlay bits in run.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tag_source was left dangling when the duplicate _k1 suffix logic from the #38 merge was collapsed into one branch, so 'b target generic' died with NameError after a successful build. The overlay auto-mount in run.py existed only because the image was built with ENABLE_INTEL_CPU=OFF and had no CPU inference device at all. #38 turns that on and the image now ships libopenvino_intel_cpu_plugin.so, verified by a full nusim/behaviour run with the hack removed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K1WalkPolicy, K1GetUpPolicy and K1KickPolicy now build a TensorRT engine from their ONNX model and fall back to OpenVINO CPU when that fails, mirroring what vision::Yolo already does. All three models are single-input/single-output with a static batch of 1, which is what utility::vision::TensorRT requires. fp16 is disabled for the policies: they are small MLPs, so there is no speed to win, and the actions drive servos directly. Yolo's device is switched to GPU, which is also the flag that gates its TensorRT path (it refuses to probe CUDA otherwise, since that can crash inside the driver rather than throw). It was running at 44 ms/frame on the CPU with an idle GPU. To make that reachable, ./b run now defaults --gpus to all, but only when the nvidia container runtime and device nodes are both present -- passing --gpus all without them makes docker run fail outright, which would break every ./b run on a CPU-only machine. Every inference site degrades to OpenVINO CPU there anyway. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
NUSimCamera reads frames from NUSim's shared-memory seqlock writer instead of NUbridge's on-robot writer, so the same behaviour tree runs against the sim. K1Servos is the Director-arbitrated sink for the K1 low-level channel: the walk, kick and get-up policies all emit BoosterLowCmd, and without a single owning module they fight over the channel. roles/nusim/behaviour.role is the sim counterpart of test/behaviour -- same skills, policies and tree, swapping K1Camera for NUSimCamera. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
roles/test/behaviour swaps skill::KickWalk for skill::K1KickPolicy and adds actuation::K1Servos so walk/kick/get-up arbitrate over the low-level channel instead of fighting for it. Tester gains kick_policy_priority, which emits skill::Kick directly and bypasses PlanKick's approach/alignment gate -- the policy does its own approach. Left at 0 so the real FindBall -> WalkToKickBall -> PlanKick chain runs by default; it is an isolation-test knob. Priorities are reordered so fall_recovery (3) outranks kick_to (2) outranks walk_to_kick_ball (1), which is what lets the kick subsume the walk on approach and the get-up subsume everything on a fall. Also commits the v7 kick checkpoint the config already points at. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
NUClearNet was pointing at a broadcast address on a subnet this setup does not use; 239.226.152.162:7447 is the multicast group NUsight and the rest of the tooling expect. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…like tip toe and torso balance
…k policy The walk observation is now 79 floats. The three base-linear-velocity entries are gone: there is no measured linear velocity on the real K1 in CUSTOM mode, and the 82-obs contract filled them by differentiating rt/odometer_state and low-passing the result — an estimator never shown to be live in CUSTOM, whose "no new sample" case is indistinguishable from "zero velocity". It is a privileged quantity, so it now lives in the critic's observation in training and nowhere on this side. That removes this module's dependency on BoosterOdometry entirely. This is a removal because the signal is unsupportable at deployment, not because it was proven to be the cause: substituting the command for obs[0:3] on the robot did not remove the tip-toe. Instrumentation, all of it needed before the next CUSTOM entry: - At TRACE, every tick logs `WALKOBS <tick> mode=<K1Mode> t=<s> dt=<s>` plus all 79 observations. tools/analysis/segment_walk_log.py splits a capture into MOVING/FROZEN runs and reports per-segment. The first hardware log is 81.5% robot-standing-still — 150 moving ticks against 663 frozen, including a 540-tick stretch where the leg joints move by the encoder quantisation floor — and whole-file statistics from it (mean |action| 0.790, frac > 0.99 = 0.469) are not the moving-segment statistics (0.538 and 0.149). - The gait phase advances on the measured loop period, clamped to [5, 100] ms, instead of a hardcoded 0.02 s. Every<50, Per<seconds>> is best-effort on an Orin also running YOLO, and at a true 40 Hz a nominal 1.5 Hz gait advances at 1.20 Hz — outside the U(1.25, 1.75) training range. - Commanded positions are clamped to the trained model's joint ranges (new joint_lower/joint_upper in the config) and clamps are counted and logged. A saturated action already commands past them on several joints — the knee reaches -0.064 at a = -1 against a 0.0 hard stop — and MuJoCo absorbs that silently where a mechanical stop does not. model_path now points at k1_walk_v2_79obs.onnx, which the current training run produces; the 82-obs checkpoints will not load against this module. benchmarks/surfaces.md is where the tilt-test measurements go. Until it has them, the training friction range and every simulator sweep corner are bracketing guesses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ts_K1 into jmontano/nusim-local
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disclaimer: Current get-up policy is WIP, it doesnt work atm. Also, the robot shakes its hands vigorously when walking with low velocities which is something that will need fine tuning later on.