Unordered Pointcloud2 processing

[simutisernestas]

Bug report

Required Info:

• Operating System:
  • Ubuntu 20.04
• ROS2 Version:
  • Foxy binary
• DDS implementation:
  • Fast-RTPS

Steps to reproduce the issue

Here while decoding pointcloud to rgb it is assumed that point cloud is actually ordered. http://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/PointCloud2.html states that: "If the cloud is unordered, height is 1 and width is the length of the point cloud.". So the code breaks later if callback gets unordered point cloud message because img shape is (1, vector size, 3) and detectron2 can't make a prediction from image properly.

Expected behavior

Both case (ordered and unordered point cloud) handling should be implemented. One possible solution to deal with unordered PC2 is to project points to the image plane and make sure it has proper shape before feeding it to the detector.

Actual behavior

Node crash on detection step.

[SteveMacenski]

Isn't there a NaN/Inf component of this you brought up?

[simutisernestas]

Those aren't actually related, I've filled in that one here: #20

[soldierofhell]

@simutisernestas, but if we take arbitrary pointcloud then we have to fill holes in the projected image. There's no guarantee it will have proper sematic meaning for detector. To me in practice the source of colored pointcloud is RGBD or stereo camera

[SteveMacenski]

Not all drivers are going to make properly assembled organized point clouds - as @simutisernestas mentions