Extrinsics Calibration keeps exceeding re-projection error stereo calibration fails
-
Hi, I have set up extrinsics and went on to calibrate the cameras, the tracking went well with a relatively low Re-projection error but when trying the stereo cameras >20 times in various lighting conditions and checkerboard patterns on the flattest possible condition I could make it and still fails during the extrinsics portion with a re-projection error >1.0. The lowest I seen was ~1.0. I also have the cameras securely mounted to a very rigid nylon frame with a 50mm baseline.
Here are my results for two of my most recent runs.
voxl2:~$ voxl-calibrate-camera stereo -m -s 5x6 -l 0.030 Please open voxl-portal in a web browser to view the camera calibrator overlay stream Running Calibration... Running Calibration... Calibrating Left Camera using intrinsics guess for ov7251 stereo cam Running Calibration... Running Calibration... Running Calibration... Matrix [496.1055783954243, 0, 296.1486262081339; 0, 496.2031197960783, 233.5686171509622; 0, 0, 1] Distortion [-0.2006227740330762; 0.1943412528850569; 0.001684284025097157; 0.0005588605630898206; -0.1513942386525285] distortion_model: plumb_bob Re-projection error reported by calibrateCamera: 0.190827 Calibration Succeded! Calibrating Right Camera using intrinsics guess for ov7251 stereo cam Running Calibration... Running Calibration... Running Calibration... Matrix [501.5883241355902, 0, 345.7963887419036; 0, 501.851954580047, 232.1727102043802; 0, 0, 1] Distortion [-0.2194527435830186; 0.2557207382732015; -0.00117769944366343; -0.001960436819949325; -0.2151900023556012] distortion_model: plumb_bob Re-projection error reported by calibrateCamera: 0.160419 Calibration Succeded! Calibrating Extrinsics 7 frames will be processed R [0.999782165046139, -0.01943157064343308, 0.007618170251720145; 0.01949255023822487, 0.9997778861700867, -0.008013663986189442; -0.007460760072888237, 0.008160415896419748, 0.9999388704673563] T [-0.04979996279758246; -0.0008864205194144189; 0.001177508840831284] Re-projection error reported by stereoCalibrate: 1.068609 Detected horizontal stereo pair Distance between cameras: 0.0498 Extrinsics Calibration Failed exceeded allowable max reprojection error: 0.5 Saved intrinsics to: /data/modalai/opencv_stereo_intrinsics.yml Exiting Cleanly voxl2:~$ voxl-calibrate-camera stereo -s 5x6 -l 0.030 Please open voxl-portal in a web browser to view the camera calibrator overlay stream Running Calibration... Calibrating Left Camera using intrinsics guess for ov7251 stereo cam Running Calibration... Running Calibration... Running Calibration... Running Calibration... Matrix [496.137885529379, 0, 298.2813704979028; 0, 496.1703851997294, 233.5087596886206; 0, 0, 1] Distortion [-0.1932838219499451; 0.1459382945761954; 0.001269118335604353; 0.0005504223022471147; -0.08971837286429113] distortion_model: plumb_bob Re-projection error reported by calibrateCamera: 0.353754 Calibration Succeded! Calibrating Right Camera using intrinsics guess for ov7251 stereo cam Running Calibration... Running Calibration... Running Calibration... Matrix [507.2197751990111, 0, 344.0102575009872; 0, 506.706680266534, 234.8746648064781; 0, 0, 1] Distortion [-0.2095882198052747; 0.2014659639319257; -0.0008889029412247639; -0.001171786662587318; -0.1412516485541331] distortion_model: plumb_bob Re-projection error reported by calibrateCamera: 0.372202 Calibration Succeded! Calibrating Extrinsics 7 frames will be processed R [0.9998302779823016, -0.01798544595018529, 0.00399236318570032; 0.01804211081554981, 0.999730001084031, -0.01464264900360799; -0.003727930679746103, 0.01471219448268697, 0.9998848202999937] T [-0.04225583835700653; -5.679041345745874e-05; 0.006198915607984797] Re-projection error reported by stereoCalibrate: 1.294870 Detected horizontal stereo pair Distance between cameras: 0.0423 Extrinsics Calibration Failed exceeded allowable max reprojection error: 0.5 Saved intrinsics to: /data/modalai/opencv_stereo_intrinsics.yml Exiting Cleanly My extrinsics for this custom drone are here: voxl2:~$ voxl-inspect-extrinsics #0: parent: imu_apps child: tracking T_child_wrt_parent: 0.055 0.006 0.010 RPY_parent_to_child: 0.0 45.0 90.0 R_child_to_parent: 0.000 -0.707 0.707 1.000 0.000 -0.000 -0.000 0.707 0.707 #1: parent: body child: imu_apps T_child_wrt_parent: 0.117 -0.006 0.005 RPY_parent_to_child: 0.0 0.0 0.0 R_child_to_parent: 1.000 -0.000 0.000 0.000 1.000 -0.000 0.000 0.000 1.000 #2: parent: body child: stereo_l T_child_wrt_parent: 0.176 -0.025 0.003 RPY_parent_to_child: 0.0 90.0 90.0 R_child_to_parent: 0.000 -0.000 1.000 1.000 0.000 -0.000 -0.000 1.000 0.000 #3: parent: body child: ground T_child_wrt_parent: 0.000 0.000 0.033 RPY_parent_to_child: 0.0 0.0 0.0 R_child_to_parent: 1.000 -0.000 0.000 0.000 1.000 -0.000 0.000 0.000 1.000I cannot seem to get this to work so any help is appreciated!!
-
@Gary-Holmgren there is mounting guidance here: https://docs.modalai.com/configure-extrinsics/#stereo-camera-mounting
-
@Moderator I have already followed these exact requirements in the procedure of designing and installing the sensors. Im not sure what else I could try.
-
@Gary-Holmgren is your checkerboard mounted perfectly flat on a very stiff backing? If there is any inconsistencies in the checkerboard it can cause issues
-
@Moderator When checking the sensors it appears one of the stereo cams is blurry due to the lens being scuffed, I cannot get it clean so going to have to get a new one as its probably causing the issue.
