feat(alpasim-integration): continuation of closed loop integration

[Arseni10Lk]

This is a PR for PR #175 .

PR #166 was closed automatically due to a history revamp, and now this PR continues the work on the closed-loop integration.

Acceptance order:

• First, this PR has to be accepted to amend PR feat(alpasim-integration): add alpasim driver plugin for closed-loop operation #175.
• After this, PR feat(alpasim-integration): add alpasim driver plugin for closed-loop operation #175 can be accepted

[Arseni10Lk]

@riita10069

Please, take a look when you have a moment

[riita10069]

@Arseni10Lk thanks, really great your work,

1. I want to check Design Doc level, so could you add Desgin/reinforcement-learning.md? I will read it.
2. Could you run the reinforcement learning loop? I want to know whather it works or not.

[Arseni10Lk]

@riita10069

1. Will create the doc today🫡

2. And about the closed-loop run

It is impossible to run it as is as of now, because the usdz are not provided by KITScenes I think and the conversion PR is not there yet it seems (unless I missed smth).

The KIT camera names are hardcoded as of now (should this be swappable?)

However, I ran it using Physical AI dataset (by swapping the camera names), and it worked.

[Arseni10Lk]

@riita10069, the doc is here. The first half of it comes from KIT/Alpasim/AutoE2E integration (so this plugin + other stuff from issue #140 )

The second half comes almost exclusively from issue #123, where I was not really actively participating, so I hope I did not get anything wrong there

[Arseni10Lk]

@riita10069

In terms of production, this seems very clear.

In terms of reinforcement learning though, I don't know if this doesn't exceed NuRec capabilities. If I understand everything correctly, then what you are suggesting is:

1. Generating a trajectory (navigational)
2. Rendering a world based on this trajectory

But isn't this impossible in NuRec?

[riita10069]

@Arseni10Lk
I think we can make this much closer to the imitation-learning pipeline.

Instead of converting AlpaSim's route into LEFT / RIGHT / STRAIGHT and then reconstructing a synthetic mask from that command, I would keep the original route geometry and rasterize it directly into the AutoE2E route_mask.

Concretely, I would suggest:

AlpaSim route -> AutoE2E adapter -> same route rasterization logic as IL -> route_mask

So the driver should preserve the route received from AlpaSim, transform/crop it into the ego-local frame, and render the same fixed-width route corridor used during imitation learning. The discrete command can still be kept for other policies if needed, but AutoE2E should consume the route geometry directly.

This also means we do not need to provide the route to NuRec or generate the world from the route; NuRec can continue rendering from the updated ego pose as it does now.

My understanding is that whether it's a PoV or Robotaxi, fundamentally there should be navigation set up with the current location, destination, and Route displayed on a map.

My understanding is that what gets input into AutoE2E's Route is always a rasterized screenshot from that IVI.

I think we should create something equivalent to that. If the Vector Map in Alpasim is inadequate, we could also pull OSM data from GNSS and create a raster map in the same way as during Imitation.

Please point it out if I'm saying something strange.

[Arseni10Lk]

@riita10069, thanks for all your feedback. I really appreciate all the great ideas shared

So, I examined the navigation and its potential implementation here, and I wanted to confirm/discuss one thing.

Constructing a route mask shouldn't be an issue (at least not as much as map rasterization). However, when it comes to map rasterization, there is one critical difference from offline training that requires some thought.

• As far as I understand, in production, the maps are rasterized before the run (so not on the fly, meaning the model does not have to "wait" for the map to be available). This is possible because it is IL and we have an actual trajectory, so rendering in advance is not an issue.
• Here, however, rendering in advance is impossible (well, if we want it to be precise, at least). So we can only rasterize a map on the fly, which might become a bottleneck.

So, the key question I wanted to ask is whether you had a chance to test live rasterization before, and how much it slows down performance in combination with a 10 Hz frequency of operation of our Reactive model

[Arseni10Lk]

I think this actually can be resolved by simply pulling a "bigger" tile than needed first and then just cropping (not literally cropping, but rather rasterizing its chunk) it in accordance with the current position. So, now I will be working on that

[riita10069]

I think one important point is how much freedom we actually want to give the model during RL.

Since the 3DGS environment is reconstructed from camera data along the original vehicle trajectory, the agent cannot deviate too far from it. For example, if it moves more than ~3 m away, we could give a negative reward and terminate the episode, since we probably shouldn’t train on clearly degraded regions of the reconstruction.

So even with RL, the task is essentially to follow the same route as the original vehicle, while allowing some local freedom in how the agent drives.

Given this constraint, I think we can preprocess and pack a sufficiently large rasterized map and route with each scene, similar to IL. During RL, we would only need to sample the relevant region based on the agent’s current pose, rather than rasterizing the map from scratch at every step.

@Arseni10Lk cc. @m-zain-khawaja

[Arseni10Lk]

Today (9/2/2026), @ZaynabEM mentioned https://huggingface.co/SIMFORGE-AI/SIMFORGE-D1.5, which is a distilled version of Alpamayo.

I wanted to leave this here so that it can be looked into and added after we are done with the Reactive Branch

[gcordova10]

@Arseni10Lk Before reviewing the code I wanted to flag something mechanical that makes this PR much
harder to read than it needs to be, and that costs one click to fix.

The diff currently shows 93 files, +18,088 / -1,147. Against main it is 19 files, +3,178, and no
deletions at all. The difference is the base branch: this PR targets
restore/pr-166-closed-loop-integration, whose tip 338a455e is from 30 July, so GitHub compares
against a snapshot from before several things landed in main on 31 July. That is why
Design/navigation_training_objectives.md and Design/rollout_aligned_checkpoint_selection.md show
as new files even though they are already in main, byte-identical, and why
.claude/settings.local.json shows as deleted when main untracked it in 92043448. None of the
1,147 deletions are yours — against main the PR deletes nothing.

Retargeting the base to main (Edit, then change the base branch) should be enough, with no
rebase or force-push: the head already carries main up to 265d6c27 (12 August), and a trial
merge against current main comes out clean here. After that the diff is your work and only your
work — the driver plugin, parser.py, rewards.py, the two configs, the two test suites,
Design/reinforcement_learning.md and three lines in requirements.txt.

Please can you check that?

[Arseni10Lk]

@gcordova10, thank you for letting me know. It seems to be fixed now.

Also, just a heads-up: I initially started implementing the full Reward Registry, which was a bit problematic at the time because KIT scenes didn't provide bounding boxes, and they were required, and so on. But as @riita10069 said during the last meeting (9/9/2026), at first we can just use a normal deviation from the GT.

Then we can implement the full reward as you suggested it and I provided this suggestion of yours in Reinforcement_Learning.md in a separate PR (leaving it here as "Future work," as it is labeled in reinforcement learning. md now).

What do you think about that?

[gcordova10]

@Arseni10Lk Thanks for the 16 September commits — this review is against a9f9afc, so it already
takes them into account. They do what you proposed on the 14th: GroundTruthDeviationReward
(rewards.py:8-52) is the ground-truth anchor in trajectory space, OffRoadReward (:55-113) keeps
the drivable-area check, and Design/reinforcement_learning.md §6.2 moves the five-term formula to
Phase 2 with the attribution. That order looks right to me, and it is the answer to your question.

One consequence worth recording, because it closes §6.2.2's own concern: the stationary-ego case
cannot happen with R_track. If the ego stops, the recorded trajectory keeps moving, so ADE and
FDE grow (rewards.py:43-45) and the terminal penalty fires past 3 m (:48-50). The progress
incentive is carried by construction rather than by a separate term, and
test_3dgs_boundary_threshold_exceeded (test_rewards.py:391-408) pins it at −10.7.

I went through the plugin, the parser, the reward library, the two test suites and the design
document against the current model code.

Short version. Items 1 and 2 stop any real predict() from completing today, and item 1 is a
regression from e1da7412 rather than a long-standing gap. Items 3 and 4 are about the model
receiving something different from what the checkpoint was trained on, and 3 and 7 have the same
one-line cause. Item 5 is small but not a small effect. Item 6 is about the design document
describing as implemented something the driver cannot reach. I hope this help you!

Line numbers for repository files are against upstream/main (31b83bf); the plugin's own files,
the two test files and the design document are cited against this PR's head (a9f9afc), since they
do not exist on main. Bare filenames (plugin.py, parser.py, rewards.py, config.py) are
under Model/plugins/alpasim_driver/alpasim_autoe2e/.

What now works. The two-yaml camera inconsistency from 7 August is closed. The default is
checkpoint_path: ??? with mock in its own autoe2e_mock.yaml. The route is rasterized from real
geometry through the dataset's own NativeNavigationRasterizer and build_scene_navigation
(parser.py:49-79), which is the offline-parity contract rather than a reconstruction from a
discrete command. The egomotion contract matches the training pipeline exactly: 64 steps by 4
signals in the order speed, acceleration, yaw_rate, curvature, with
curvature = yaw_rate / max(speed, 0.1) the same speed floor as
Model/data_parsing/kit_scenes/egomotion.py:42-43,85-86. The calibration moving to
configs/calibration/kit_scenes.json makes the camera set swappable, which answers your own
question from 6 August. And the loader now fails loudly instead of silently — see item 3. On the
reward side, OffRoadReward is carefully covered: test_rewards.py:123-337 checks ego rotation and
translation, polygons with holes, 2D and 3D coordinates, polygons under three points, and STRtree
cache invalidation by map_version, with exact values.

1. camera_params now reaches AutoE2E.forward(), which rejects it by design.

parse_observation returns "camera_params" in its dict (parser.py:175) alongside "projection"
and "geometry_type": "pinhole" (:176-177), and predict() forwards the whole dict with
self.model(**tensors, mode="inference") (plugin.py:174-181). On main, AutoE2E.forward treats
that key as a hard error: _REMOVED_GEOMETRY_KWARGS (Model/model_components/auto_e2e.py:28-31)
contains camera_params, and :207-220 raises TypeError for any of them. The message explains
why it is an error and not a warning — otherwise **kwargs absorbs it and the model runs on
geometry_type='pseudo', a learned spatial prior instead of your calibration.

Until d813eb40 the plugin did exactly the right thing: tensors.pop("camera_params") before
building PinholeProjection (old plugin.py:261-264). e1da7412 moved the projection into the
parser, which is cleaner, but left the raw key in the returned dict. So the fix is one line — drop
"camera_params" from the dict parse_observation returns, since "projection" already carries it.

The suite cannot catch this: MockAutoE2EModel.forward(**kwargs) accepts anything
(test_alpasim_stream.py:41-43), so every test that exercises predict() passes with a model that a
real AutoE2E would reject.

2. The route path is unreachable from the wizard, so parse_observation raises before the model is
called.

AlpasimStreamParser only builds the rasterizer and the route when scene_id is set
(parser.py:49-79). AutoE2EDriver.__init__ accepts scene_id (plugin.py:56), but from_config
— the path AlpaSim uses — reads only model_cfg.checkpoint_path and never passes it
(plugin.py:97-113). AutoE2EAlpaSimConfig.scene_id exists (config.py:102) and nothing
instantiates it. With scene_id unset, parse_observation reaches raise ImportError("The rasterizer and/or route are missing, cannot render route mask.") (parser.py:160-162), and
predict() calls the parser before it branches on self.model (plugin.py:174), so this fires on
every real call including mock mode. AlpaSim does send a scene identifier in the request; wiring
scene_id through from_config would be enough.

3. The driver builds the model from constructor defaults instead of from the config the
checkpoint carries, and every other loader in the repo does the opposite.

_model_kwargs(config) (Platform/pipelines/workflows.py:246 on main) filters a saved config
down to the kwargs the current constructor accepts. Three call sites use
AutoE2E(**_model_kwargs(config)) followed by load_state_dict — inline at :6211 and :6378, in
two steps at :7414-7417. A fourth loader keeps its own copy of the helper:
Platform/pipelines/inference.py defines model_kwargs at :27-32 and rebuilds the same way at
:45-47, reading checkpoint["config"] first. The code that writes the checkpoint documents the
contract explicitly, at workflows.py:4038-4041:

Carried in the checkpoint so evaluation rebuilds the SAME architecture: _model_kwargs feeds this
dict into AutoE2E(**config), so without the key a non-default run would be reconstructed with
the constructor default and load mismatched weights.

plugin.py:78-81 builds AutoE2E(num_views=len(self._camera_ids), is_pretrained=False) and then
loads checkpoint["model_state_dict"], never reading checkpoint["config"]. Concretely:
map_context_channels is saved in that dict (workflows.py:4047, read from the pack manifests at
:3368, collapsed at :3385); anything packed through the navigation path carries 14, because
NavigationRaster.__post_init__ refuses any other count (Model/navigation/rasterizer.py:68-69,
MapChannel / MAP_CHANNEL_COUNT at Model/navigation/geometry.py:13-27 and :35); the
constructor default is 3 (AutoE2E.__init__, Model/model_components/auto_e2e.py:75); and the
navigation encoder is sized map_context_channels + route_channels (build_map_encoder call in
ReactiveE2E.__init__, Model/model_components/reactive_e2e.py:64-66). So load_state_dict fails
on shape.

The good news is that this is now a loud failure — dropping the try/except means the driver does
not come up on an untrained network.

Worth noting where the helper should come from. workflows.py imports flytekit at module level
(:13-17), so importing _model_kwargs from a plugin running inside the AlpaSim container is
probably not an option — which is presumably why inference.py keeps a five-line copy instead
(:27-32), with the AutoE2E import deferred into the function body. That is the shape that
transfers here: read checkpoint["config"], filter it against
inspect.signature(AutoE2E.__init__).parameters, and pass the result.

Whichever way it is done, it needs no special handling for is_pretrained: the saved config already
carries "is_pretrained": False (workflows.py:4055), and that is deliberate — the comment above
it, :4053-4054, reads "Checkpoints contain the complete backbone. Reconstruction must not
download pretrained weights before loading that state." So keeping it False, as the plugin already
does, is correct.

4. map_context is a different representation, not just a different channel count.

parser.py:140-153 fills it from generate_bev_map_tile
(Model/data_parsing/kit_scenes/map.py:239), which renders an RGB canvas — :264 is
np.full((rs, rs, 3), 255, dtype=np.uint8) — with the line types encoded as colours, and
parser.py:119 defaults it to torch.zeros(1, 3, 256, 256). The map_context the model was
trained on is the 14-channel float32 semantic raster; the contract is enforced in
Model/navigation/rasterizer.py:68-69, which raises
ValueError("map_context must have shape [14,H,W]"), and the rasters are produced at :512 and
:558. Both paths exist in the dataset — Model/data_parsing/kit_scenes/dataset.py:462
(map_for_row, the raw uint8 RGB tile) and :482 (navigation_members_for_row, the semantic
members) — and the plugin is wired to the first while the checkpoints use the second. Channel count,
value range (0-255 unnormalised) and semantics all differ, so coercing the shape would not help.
Worth flagging because the 3-channel shape is currently pinned as correct by the suite: four
assertions in Model/tests/test_alpasim_stream.py (:172, :662, :702, :713).

The semantic raster is already in hand at the point the tile is fetched: parser.py:137 calls
self.rasterizer.render(...) and uses only raster.route_mask, while the same NavigationRaster
carries map_context with the 14 channels.

5. The integrator differs from the evaluation one by about a metre at 6.4 s, so closed-loop and
open-loop numbers will not be comparable.

integrate_trajectory (Model/evaluation/metrics.py:46-49 on main) updates
v = max(0.0, v + a*dt), then theta += curvature*v*dt, then position with the updated v and
theta. _unroll_unicycle_controls (plugin.py:25-44) advances position first using the previous
step's v and theta (:36-37), updates theta at :38 and v last at :39, with no
max(0, v) clamp.

I ran both over 64 steps at 0.1 s on the same control sequences rather than arguing from the code:

controls (from 8 m/s)	mean gap	final gap	final v, eval vs plugin
straight, constant speed	0.000 m	0.000 m	8.00 / 8.00
curvature 0.05, constant speed	0.904 m	1.533 m	8.00 / 8.00
a = +1.0, curvature 0.03	1.150 m	2.593 m	14.40 / 14.40
a = -1.5, straight	0.426 m	0.025 m	0.00 / -1.60

This is not a braking problem, it is a turning problem. Braking is the mildest case in the set:
the negative velocity walks the path back and the two agree to 0.025 m at the end. A plain
constant-speed turn, positive velocity throughout, ends 1.533 m apart. So it does not affect the
occasional hard-deceleration rollout, it affects every rollout that turns, and it grows with |a|
and |curvature|. The straight row is the control at 0.000 m exactly, so the gap is the integrator
and not the measurement. Headings agree to within 1.1 degrees everywhere, so it is a positional
phase error, not a steering one. The missing clamp is still worth adding — the plugin reaches
-1.60 m/s, the ego driving backwards along its heading — but it is the smaller of the two.
Importing integrate_trajectory removes both and gives train/sim consistency by construction.

6. §6.1 is labelled "(Implemented)", but RewardManager is not reachable from the driver and
nothing truncates the episode.

RewardManager (rewards.py:116-143) is not imported or constructed anywhere in plugin.py;
AutoE2EDriver.__init__ does not take a rewards argument (plugin.py:50-57) and from_config
reads only checkpoint_path (:97-113), so the rewards: block in autoe2e.yaml:16-18
(w_gt_dev, w_offroad) is configuration nothing reads. Two of the reward's inputs also have no
source yet: gt_trajectory — PredictionInput carries past ego poses, not the recorded future —
and navigation_map, which lives in the parser (parser.py:78). And §6.1 :122 says a terminal
penalty is applied and the episode is truncated early, but GroundTruthDeviationReward.compute
returns a scalar with -terminal_penalty folded into it (rewards.py:47-52); nothing returns a
termination signal.

The class and its tests are fine as far as they go. The point is the label: this is the same thing
you were asked on 7 August about not documenting future work as implemented. Either "implemented as
a library, wiring pending" in §6.1, or wire it in this PR.

7. The camera-topology parity test compares the parser against a hardcoded copy of itself.

Model/tests/test_alpasim_stream.py:341-364 asserts that PARSER_CAMERA_NAMES (the plugin's own
seven names, config.py:16-24) equals EXPECTED_KITSCENES_TOPOLOGY (:349-357), a literal list of
the same seven, and then asserts len == 7 with "AutoE2E expects exactly 7 cameras." (:364). The
rig has seven, but the model input contract is six — those are different counts and the plugin is
using the first where the second applies. Model/data_parsing/kit_scenes/camera.py is
byte-identical on main and on this branch, so this is not a rebase artefact: its CAMERA_NAMES
omits camera_ring_front, with the reason written above it — "camera_ring_front is omitted because
it duplicates the long-range front camera's approximately 88-degree forward coverage at a lower
resolution" — and the module docstring says it "stacks the 6 camera views into the tensor the model
expects". Because BEVViewFusion derives V from the projection operator rather than from the
construction-time num_views (Model/model_components/view_fusion/bev_fusion.py:198), the extra
view degrades silently instead of raising.

This shares a root cause with item 3. num_views is saved in the checkpoint config too
(workflows.py:4044), while plugin.py:78-79 computes it as len(self._camera_ids). Rebuilding
from the saved config restores it at construction, but on its own that does not drop the seventh
view: V comes from the operator at runtime, and workflows.py:414-415 says the same —
"AutoE2E is runtime-V-dynamic. The construction value only sizes defaults". What removes the
seventh view is the camera list the runtime requests, which is a separate decision. Avoiding an
import of camera.py in CI is reasonable — it does from kitscenes.sensors import SensorDataLoader
at module level — but the hardcoded list then needs to match what that module declares, or the
parity test cannot fail.

Three corrections to the design document, all about numbers or references from #123.

§6.2.1 (:144) says g reads zero "as it does in early checkpoints", which reads as a
training-maturity transient. The measurement in #123 (comment of 20 July) is a forced-coupling
calibration: the trajectory delta is linear in alpha across two decades, 5.05e-03 m per unit of
alpha on arm B and 3.75e-03 on arm C, and it back-predicts the deltas at the trained alphas to
within about 2%. So the near-zero reading is a measurement rather than an instrument floor, and the
reason given in the issue body is structural: the reasoning head's inputs are a strict subset of the
planner's, so the reasoning latent cannot carry trajectory information the planner lacks. More
epochs do not move that; a non-redundant input would. The statement that holds is that the causal
influence is tiny and the gated term does not contribute measurably.

§6.2.2 (:148) attributes the under-progress point to "an imitation-only policy evaluated in
AlpaSim" and gives "driving 44 km/h slower than traffic" as the example. The run is the one in #123
(comment of 21 July): one scene and one rollout, reported there as a signal rather than a result,
with progress 0.11, 8.48 m travelled against a ground truth of 73.77 m, and collision_rear 1.00
with collision_at_fault 0.00. The 44 km/h figure is from a different comment and a different
experiment — it is the mean initial speed of the labelled set (12.3 m/s), not a speed deficit
relative to traffic. The mechanism the run supports is under-progress leading to being rear-ended,
which is enough for the point §6.2.2 makes.

§7.2 (:164) refers to "the non-redundancy constraint outlined in Section 5". Since a9f9afc the
document no longer states that constraint anywhere — the §6 preamble that carried it went with the
Phase 1 rewrite — so the reference points at nothing. Either restore the sentence in §6.2 or drop
the reference. Also, the sensor figure in §3 (:44) sums to 73.0 MPix rather than 72.5.