Unit test and pipeline debuging
Hi David,
I am working on the unit test, but there are a few things I am not sure about. I thought we could have the discussion here, but let me know if it is not appropriate.
First thing, I am not sure I understand how the reproject_image function works.
projected_image = reproject_image(groundtruth, map_3d, color, camera_pair)Here I was trying to project an image of the color camera onto the same camera, expecting that it would be an identity operation. However the output is different from the input image, it is shifted up right. Is it normal or am I understanding it wrong ? In order to project the image from the nir_left camera onto the color, would this work ?
projected_image = reproject_image(left_image, map_3d, color, camera_pair)Second thing, what would be the best metric to measure if the generated output is close enough to the input ? Similarity measure from bob.ip.qualitymeasure looked promising, but is it enough ? What do you think ?
Thank you, Vincent
Merge request reports
Activity
- bob/ip/stereo/test/test.py 0 → 100644
36 groundtruth = groundtruth.mean(axis=0, keepdims=True, dtype=groundtruth.dtype) 37 38 # Load camera configurations 39 nir_left_stereo = load_camera_config(pkg_resources.resource_filename('bob.ip.stereo', 40 'config/idiap_face_calibration.json'), 'nir_left') 41 nir_right_stereo = load_camera_config(pkg_resources.resource_filename('bob.ip.stereo', 42 'config/idiap_face_calibration.json'), 'nir_right') 43 color = load_camera_config(pkg_resources.resource_filename('bob.ip.stereo', 44 'config/idiap_face_calibration.json'), 'color') 45 camera_pair = CameraPair(nir_left_stereo, nir_right_stereo) 46 47 # Create 3D image 48 map_3d = stereo_match(left_image, right_image, camera_pair) 49 50 # project left image on color stream 51 projected_image = reproject_image(groundtruth, map_3d, color, camera_pair) changed this line in version 3 of the diff
Hey
First thing to understand is that you have three cameras: left camera and right camera (that form the "camera_pair") plus a third camera (here color).
Each camera yields an image, and in addition has metadata encoding its geometric parameters and spatial positionning:
- intrinsic parameters: camera matrix (projection matrix) & distortion coefficients (lens distortion).
- extrinsic parameters: relative rotation and translation to some reference (left camera is reference here)
When computing the stereo map (with stereo_match) the first step is to rectify left and right images (remove lens distortion and align the images horizontally, see opencv reference for more details), this is done via the stereo_rectify camera_pair member. The second step is to perform block-matching, i.e. finding the distance in pixel between similar blocks in the two pictures, high disparity is for a close block, low disparity for a distant block (this is done via StereoSGBM_create and compute). The first step is to take this disparity, and with meta data compute the 3d coordinates of each pixels (done with reprojectImageTo3D, after some image processing).
The result is a map_3d object, an image where "pixels" are the 3d position of the pixel compared to the left camera center. If you link adjacent pixels you obtain a 3d surface.
The idea of "reprojection" is to project a third (color) image on this surface and then project back this on the reference camera (left rectified). You then get (if all goes as planned) a color picture which is perfectly, pixel-wise, aligned with the left (rectified) image (and therefore with the map_3d image).
To test that i propose to run the algoritm, save map3d and reprojected color image and save the results in the package. when the test is performed just compared the results with the one stored in the package, so if a commit changes algoritm, you see it in the test. You can just bit-wise compare or compare with a small epsilon.
Hope that's clearer?
added 1 commit
- 15b9324a - Test by comparing 3d map and projected image against previously saved
added 2 commits
added 1 commit
- 3628c7ba - [CI] debuging: use pkg_resource to access to test images for unit test
added 1 commit
- e1b19e61 - [CI] debuging: using pkg_resources everywhere it is needed
added 1 commit
- 21f72a09 - [CI] debuging: add pybind requirements to run config
added 1 commit
- 9f5f0ba8 - [CI] debuging: use c++14 to avoid clang error
added 3 commits
-
10238e8b...b3fd0f98 - 2 commits from branch
improvements - 85b5c38e - Merge branch 'improvements' into unit_tests
-
10238e8b...b3fd0f98 - 2 commits from branch
added 2 commits