Skip to content

[Bug] 4 corrupt motion files in data/motions/rpo_lab (unreadable pickle)(rpo_lab 中有 4 个损坏的动作文件) #14

Description

@mecha-d

Affected files (paths relative to robolab):

  • data/motions/rpo_lab/run_start_180_R_001__A345.pkl (112,194 bytes)
  • data/motions/rpo_lab/run_start_180_R_001__A345_M.pkl (112,194 bytes)
  • data/motions/rpo_lab/run_stop_180_R_001__A345.pkl (149,746 bytes)
  • data/motions/rpo_lab/run_stop_180_R_001__A345_M.pkl (149,746 bytes)

Environment: verified at commit efb32c6; the corruption is in the
committed git blobs themselves (see below), so it reproduces on any
OS/Python.

Expected: pickle.load() returns the motion dict (fps, root_pos,
root_rot, dof_pos, key_body_pos) as it does for the other 26 clips.

Actual: pickle.load() fails on all four with:

_pickle.UnpicklingError: invalid load key, '\x05'.

The files begin with a valid pickle protocol-4 header (\x80\x04\x95) but the
stream is corrupt partway through. Note each _M file is byte-identical in
size to its non-_M partner, which suggests a single failed export/copy step
produced both.

Reproduction:

import pickle
with open("data/motions/rpo_lab/run_start_180_R_001__A345.pkl", "rb") as f:
    pickle.load(f)  # UnpicklingError: invalid load key, '\x05'.

Verified not a download problem: the on-disk bytes match the committed git
blobs exactly (git status clean, git cat-file blob header identical to
disk), and no LFS is involved — the corruption is in the repository content
itself. All other 26 clips in rpo_lab load and parse fine.

Impact: any pipeline that iterates over rpo_lab/*.pkl (e.g. AMP dataset
loading) crashes unless these four files are skipped.

Suggested fix: re-export these clips from the source data and recommit, or
remove them from the dataset.


中文版本(点击展开)/ Chinese version

受影响的文件(路径相对于 robolab):

  • data/motions/rpo_lab/run_start_180_R_001__A345.pkl(112,194 字节)
  • data/motions/rpo_lab/run_start_180_R_001__A345_M.pkl(112,194 字节)
  • data/motions/rpo_lab/run_stop_180_R_001__A345.pkl(149,746 字节)
  • data/motions/rpo_lab/run_stop_180_R_001__A345_M.pkl(149,746 字节)

环境: 在提交 efb32c6 上验证;损坏存在于 git 提交的 blob 本身(见下文),
因此在任何操作系统/Python 版本上都可复现。

预期行为: pickle.load() 返回动作字典(fps、root_pos、root_rot、dof_pos、
key_body_pos),与其余 26 个文件一致。

实际行为: 这四个文件用 pickle.load() 读取时全部失败,报错:

_pickle.UnpicklingError: invalid load key, '\x05'.

文件开头是有效的 pickle protocol-4 头(\x80\x04\x95),但数据流在中途已损坏。
另外,每个 _M 文件与对应的非 _M 文件字节大小完全相同,推测是同一次失败的
导出/复制操作同时产生了这两组文件。

复现方法:

import pickle
with open("data/motions/rpo_lab/run_start_180_R_001__A345.pkl", "rb") as f:
    pickle.load(f)  # UnpicklingError: invalid load key, '\x05'.

已确认不是下载问题: 磁盘上的字节与 git 提交的 blob 完全一致(git status
干净,git cat-file blob 的文件头与磁盘相同),也未使用 LFS——损坏存在于仓库
内容本身。rpo_lab 中其余 26 个动作文件均可正常读取和解析。

影响: 任何遍历 rpo_lab/*.pkl 的流程(例如 AMP 数据集加载)都会崩溃,
除非跳过这四个文件。

建议修复: 从源数据重新导出这些动作并重新提交,或将它们从数据集中移除。

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions