OV9782 tracking configuration

[david.moro]

Re: Using OV9782 for tracking

Hi, I am trying to use the OV9782 camera (M0113) for VIO, and I am struggling to make it work. The VOXL2 recognizes the camera, and I can see it in the portal. However, it fails to calibrate. I am currently running voxl-suite 0.9.4. Can you confirm I need to be on 0.9.5 to make it work? Also, what are the steps to configure it? Can I use voxl-configure-cameras to set the system up?

Thanks!

[Moderator]

If you see the sensor in the portal, you have the sensor configured properly.

By fails to calibrate, are you using voxl-camera-calibration ?

[david.moro]

@Moderator Correct. Both with and without the --fisheye flag. In one case I get this:

In the other case I get:

Also, I moved to SDK 1.0.0

[A Former User]

@david-moro

Hello! Sorry to hear you're having trouble. With regards to your first screenshot: your re-projection error in the first shot of 1.028 is too high for a fluke occurrence. I would review your calibration setup: make sure you have good lighting conditions, a completely flat calibration grid, and that your lens is focused on the scene. I have calibrated these cameras a few times before and changing lighting conditions can make all the difference. With regards to your second screenshot, the OV9782 sensor is not a fisheye lens which is my best guess as to why you're getting the error you're getting.

Hope this helps, let me know if you have any further questions.

Thomas

[david.moro]

@Thomas-Patton , no worries, the fight is part of the fun!. So, I tried a dozen times outdoors with no success. In all cases, I made sure the chess board had good/uniform illumination. Would you mind pasting the results of a successful calibration on your end? I wonder if the reprojection error is due to the last coefficient being forced to zero.

Also, I copied and pasted the "failed" values by hand into a intrinsic config file and I got the VIO to work on it without trouble. We will do some flying soon. I'll keep you posted.

One last question: would the reprojection error depend on the resolution of the camera? This camera is twice the resolution of the standard tracking one.

Cheers!

[A Former User]

@david-moro

Hm, I'm sorry to hear you're still having trouble with the camera calibration. We're trying to work on a more sophisticated algorithm to calibrate in more robust conditions but for now this is all we have. Below I've pasted the .yml results for my correct calibration, you're certainly welcome to use them if you'd like but I can't make any guarantees with how well they will work.

For your question on the reprojection error, my intuition says a higher resolution could actually lead to a lower reprojection error given the larger amount of available data to form a pixel-to-world mapping. However, I think in general other factors like the clarity of checkerboard used probably outweigh this factor.

Lastly, I'll point you to our camera calibration page which you've probably already seen but has a ton of info and some videos of us running the same calibration routine in our office.

Again, sorry for the troubles here, hope some of this helps!

Thomas

opencv_stereo_grey_extrinsics.yml

%YAML:1.0
---
R: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 9.9938944667465623e-01, -7.4044245288328496e-03,
       3.4145400462161628e-02, 9.3430943413412692e-03,
       9.9833208538277962e-01, -5.6971518177061871e-02,
       -3.3666607542984874e-02, 5.7255757725029832e-02,
       9.9779173064516791e-01 ]
T: !!opencv-matrix
   rows: 3
   cols: 1
   dt: d
   data: [ -7.9822710237796926e-02, -2.4983938280042977e-04,
       1.3478076846433324e-03 ]
reprojection_error: 9.0313540855854690e-01
orientation: vertical
calibration_time: "2023-07-19 19:10:58"

opencv_stereo_grey_intrinsics.yml

%YAML:1.0
---
M1: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 5.3055673012705324e+02, 0., 6.5505404689306852e+02, 0.,
       5.3191382931166481e+02, 3.9505491684283311e+02, 0., 0., 1. ]
D1: !!opencv-matrix
   rows: 5
   cols: 1
   dt: d
   data: [ -3.7237267115525718e-02, -5.0149156174346086e-03,
       -4.0019302977205550e-04, -1.8178997714387169e-04, 0. ]
reprojection_error1: 4.4450130427731271e-01
M2: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 5.3165184840643917e+02, 0., 6.3496372217095029e+02, 0.,
       5.3296270599190177e+02, 3.9779778516864519e+02, 0., 0., 1. ]
D2: !!opencv-matrix
   rows: 5
   cols: 1
   dt: d
   data: [ -4.3110297717821523e-02, -3.0787131445127801e-03,
       5.5046174350197768e-04, 3.0979567848984051e-04, 0. ]
reprojection_error2: 4.3755421944960809e-01
width: 1280
height: 800
distortion_model: plumb_bob
calibration_time: "2023-07-19 19:10:58"

[david.moro]

@Thomas-Patton Thanks a lot! A VERY flat checkerboard did it. Reprojection error is now in the order of 0.3.

[A Former User]

@david-moro

Awesome! Glad we were able to get this resolved

Thinking about it more, this makes a lot of sense. A higher resolution camera is going to be ultra-precise in its pixel-to-world mapping meaning small inaccuracies in the grid would probably cause the model to blow up. I'll make a note to make this very clear on our documentation.

Don't hesitate to reach out if you have any other questions. Best of luck!!

Thomas