camera distortion

[dcnieho]

Hi, I find this work very interesting, as i wish to figure out exactly where my markers are on a non-coplanar ArUco board. I am looking through the code, and am wondering the following:

1. I see you use the camera_matrix in calls to cv2.solvePnP() but not distortion coefficients. Should i undistort my images before using them with the map builder?
2. I also see that your project function doesn't take distortion parameters. For the cv2.solvePnP() calls i could just add them, for this function as you wrote it yourself its more complicated. Could it be replaced with OpenCV functions like cv2.projectPoints() so that distortion coefficients can be used?

Thanks!

[dcnieho]

Ah, first paragraph of your readme :p

[dcnieho]

Super cool stuff, got it to work. Do you welcome pull requests? I changed show_map.py to use a 3D plot when plotting a 3D map, can send that your way.

[dcnieho]

Ok, and a last question. For a single marker, cv2.solvePnP() has an ambiguity (see, e.g. opencv/opencv#8813). One can get both solutions by calling ret, rvecs, tvecs, reprojectionError = cv2.solvePnPGeneric(objectPoints, imgPoints, cameraMatrix, distCoeffs, flags=cv2.SOLVEPNP_IPPE_SQUARE), whereas cv2.solvePnP() only returns the "best" one of the two. Would the built map be better if both solutions are taken into account, and consistency in poses between markers in the various images used to prune the wrong solutions returned by the PnP solver? I don't yet understand the logic and math of the method code, so am not able to help.

[markisus]

Hi @dcnieho, sorry for the late response. I'm glad you got everything to work!

Yes a pull request is welcome. The only concern is if it pulls in a massive dependency on a new third party library.

As for your second question, I'm not sure. When building maps, an image with a single marker contributes no information to the map. For live inside out tracking, you may end up with a camera frame with just a single marker (or no markers) during camera movement. In these scenarios, it could be a good idea to try both solutions and see which one is more consistent with the previous known camera pose.