Questions on the poses data

[ruoshiwen]

Kudos on this great work.

I am looking at the annotation pkl file. I am using the mano parameters. I print the shape of each key:

rh__pose_coeffs (1, 16, 4) torch.float32
lh__pose_coeffs (1, 16, 4) torch.float32
rh__tsl (1, 3) torch.float32
lh__tsl (1, 3) torch.float32
rh__betas (1, 10) torch.float32
lh__betas (1, 10) torch.float32

I would like to know rh_tsl and rh__pose_coeffs's wrist rot is relative to which frame? Is it world or camera? If it is in world, where is world defined? If camera, in which camera's frame?

And I would like to get the hand pose in the egocentric cam's frame, can I trust cam_extr["egocentric"] to get the transformation?

And the order of joints in rh__pose_coeffs? Based on mano convention, my guess would be wrist, index, middle, ring, pinky, thumb? I would like to confirm with you.

I look forward to your reply. Many thanks in advance.

[kelvin34501]

Thank you for your interest in OakInk2 dataset!

rh_tsl and rh__pose_coeffs's wrist rot is relative to which frame? Is it world or camera? If it is in world, where is world defined? If camera, in which camera's frame?

In OakInk2, the annotations are defined relative to the world_frame. Each annotation file includes the camera extrinsic parameters; using these extrinsics, you can transform and obtain the transformation in the camera frame.

The world frame is defined by the MoCap system’s origin point.

Can I trust cam_extr["egocentric"] to get the transformation?

This may be the best way to do it.

And the order of joints in rh__pose_coeffs? Based on mano convention, my guess would be wrist, index, middle, ring, pinky, thumb?

It does follow the MANO convention; please refer to manotorch for more details.

Hope these answers help!

[ruoshiwen]

Thank you for your detailed explainations. They are very helpful.

One more thing to confirm with you:
RGBs are synced to 30 fps, and the poses are captured at 120 Hz?

Thank you !