feat: resume interrupted LeRobot imports - #384
Conversation
|
👋 Hi @victorwon2001 — thanks for the contribution! To keep starter issues available You already have #380 open, so this one is being closed automatically. |
…hes the date (#385) test_preview_stats_render_timestamps_in_utc_on_a_non_utc_host asserted "-04" not in the rendered bound to catch the host's -04:00 offset. The substring also matches the day in 2026-09-04 and the month in any April date, so the test failed on the 4th of every month and through all of April regardless of the code under test. It is failing on main today. The bounds are now compared to the row timestamps and to the wall clock instead. That drops the date collision and covers more: a render that shifted the clock into the host zone while still stamping +00:00 passed both the old substring check and a rows-to-stats comparison on its own, because rows and stats go through the same projection and shift together. Reported independently by @akshatpatel64 (#383) and @victorwon2001 (#384).
|
Before you reopen this: #390 is now an open PR against #303 and is further along, so reopening here would put two of you on the same issue. #303 still has no assignee, so nothing is formally reserved, but there is no point in you both writing it. Worth knowing either way, because it applies to both attempts: stamping the import identity into The other half of this PR is gone: #389 merged the schema-3 per-episode entries for #379 about two hours ago, so the manifest work here is done. #380 is still the one open under your name and it is waiting on you. Your other reopen candidate, #381 against #320, is untouched and nobody else is on it. |
2 participants
Summary
Resume interrupted multi-episode LeRobot imports by reusing complete landing MCAPs only when their stored dataset revision, source episode, selected camera topics, converter version, and canonical pipeline version match the requested import. Bump
prepared-manifest.jsonto schema v3 with canonical pipeline identity and per-episode output keys while preserving the existing top-level fields.Fixes #303
Why
A failure late in an import previously left valid earlier MCAPs but no durable way to identify them as completed on retry, so they were converted again. Invalid, summary-less, truncated, or identity-mismatched outputs now fall back to the existing atomic conversion/publish path; fresh conversion uses the same transform configuration, so canonical output behavior is unchanged.
Validation
uv run ruff check— passeduv run ruff format --check— passeduv run ty check— passeduv run pytest tests/test_lerobot_converter.py -q— 42 passeduv run pytest -q— 1455 passed, 6 skipped, 2 failed outside the LeRobot change paths; the bounded-concurrency test is timing-sensitive, and the UTC-preview test has a date-sensitive-04assertion on 2026-09-04, with the latter also reproducing against HEADlychee --no-progress --include-fragments --exclude '^https://github\.com/Hebbian-Robotics/hflow/(issues|security/advisories/new)$' --exclude-path references/mcap-spec.md --exclude-path references/foxglove-CompressedVideo.proto .— passed with 0 errorsChecklist
uv run ruff check --fix,uv run ruff format, anduv run ty check.