ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Extrinsics Calibration keeps exceeding re-projection error stereo calibration fails

    Support Request Format for Best Results
    2
    5
    265
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Gary HolmgrenG
      Gary Holmgren
      last edited by Gary Holmgren

      Hi, I have set up extrinsics and went on to calibrate the cameras, the tracking went well with a relatively low Re-projection error but when trying the stereo cameras >20 times in various lighting conditions and checkerboard patterns on the flattest possible condition I could make it and still fails during the extrinsics portion with a re-projection error >1.0. The lowest I seen was ~1.0. I also have the cameras securely mounted to a very rigid nylon frame with a 50mm baseline.

      Here are my results for two of my most recent runs.

      portal.png

      voxl2:~$ voxl-calibrate-camera stereo -m -s 5x6 -l 0.030 
      Please open voxl-portal in a web browser to view the camera calibrator overlay stream
      Running Calibration...
      Running Calibration...
      Calibrating Left Camera
      using intrinsics guess for ov7251 stereo cam
      Running Calibration...
      Running Calibration...
      Running Calibration...
      Matrix
      [496.1055783954243, 0, 296.1486262081339;
       0, 496.2031197960783, 233.5686171509622;
       0, 0, 1]
      Distortion
      [-0.2006227740330762;
       0.1943412528850569;
       0.001684284025097157;
       0.0005588605630898206;
       -0.1513942386525285]
      distortion_model: plumb_bob
      Re-projection error reported by calibrateCamera: 0.190827
      Calibration Succeded!
      Calibrating Right Camera
      using intrinsics guess for ov7251 stereo cam
      Running Calibration...
      Running Calibration...
      Running Calibration...
      Matrix
      [501.5883241355902, 0, 345.7963887419036;
       0, 501.851954580047, 232.1727102043802;
       0, 0, 1]
      Distortion
      [-0.2194527435830186;
       0.2557207382732015;
       -0.00117769944366343;
       -0.001960436819949325;
       -0.2151900023556012]
      distortion_model: plumb_bob
      Re-projection error reported by calibrateCamera: 0.160419
      Calibration Succeded!
      Calibrating Extrinsics
      7 frames will be processed
      R
      [0.999782165046139, -0.01943157064343308, 0.007618170251720145;
       0.01949255023822487, 0.9997778861700867, -0.008013663986189442;
       -0.007460760072888237, 0.008160415896419748, 0.9999388704673563]
      T
      [-0.04979996279758246;
       -0.0008864205194144189;
       0.001177508840831284]
      Re-projection error reported by stereoCalibrate: 1.068609
      Detected horizontal stereo pair
      Distance between cameras: 0.0498
      Extrinsics Calibration Failed
      exceeded allowable max reprojection error:  0.5
      
      Saved intrinsics to: /data/modalai/opencv_stereo_intrinsics.yml
      Exiting Cleanly
      voxl2:~$ voxl-calibrate-camera stereo -s 5x6 -l 0.030 
      Please open voxl-portal in a web browser to view the camera calibrator overlay stream
      Running Calibration...
      Calibrating Left Camera
      using intrinsics guess for ov7251 stereo cam
      Running Calibration...
      Running Calibration...
      Running Calibration...
      Running Calibration...
      Matrix
      [496.137885529379, 0, 298.2813704979028;
       0, 496.1703851997294, 233.5087596886206;
       0, 0, 1]
      Distortion
      [-0.1932838219499451;
       0.1459382945761954;
       0.001269118335604353;
       0.0005504223022471147;
       -0.08971837286429113]
      distortion_model: plumb_bob
      Re-projection error reported by calibrateCamera: 0.353754
      Calibration Succeded!
      Calibrating Right Camera
      using intrinsics guess for ov7251 stereo cam
      Running Calibration...
      Running Calibration...
      Running Calibration...
      Matrix
      [507.2197751990111, 0, 344.0102575009872;
       0, 506.706680266534, 234.8746648064781;
       0, 0, 1]
      Distortion
      [-0.2095882198052747;
       0.2014659639319257;
       -0.0008889029412247639;
       -0.001171786662587318;
       -0.1412516485541331]
      distortion_model: plumb_bob
      Re-projection error reported by calibrateCamera: 0.372202
      Calibration Succeded!
      Calibrating Extrinsics
      7 frames will be processed
      R
      [0.9998302779823016, -0.01798544595018529, 0.00399236318570032;
       0.01804211081554981, 0.999730001084031, -0.01464264900360799;
       -0.003727930679746103, 0.01471219448268697, 0.9998848202999937]
      T
      [-0.04225583835700653;
       -5.679041345745874e-05;
       0.006198915607984797]
      Re-projection error reported by stereoCalibrate: 1.294870
      Detected horizontal stereo pair
      Distance between cameras: 0.0423
      Extrinsics Calibration Failed
      exceeded allowable max reprojection error:  0.5
      
      Saved intrinsics to: /data/modalai/opencv_stereo_intrinsics.yml
      Exiting Cleanly
      
      My extrinsics for this custom drone are here:
      voxl2:~$ voxl-inspect-extrinsics
      #0:
          parent:                imu_apps
          child:                 tracking
          T_child_wrt_parent:    0.055   0.006   0.010 
          RPY_parent_to_child:   0.0    45.0    90.0  
          R_child_to_parent:     0.000  -0.707   0.707 
                                 1.000   0.000  -0.000 
                                -0.000   0.707   0.707 
      #1:
          parent:                body
          child:                 imu_apps
          T_child_wrt_parent:    0.117  -0.006   0.005 
          RPY_parent_to_child:   0.0     0.0     0.0  
          R_child_to_parent:     1.000  -0.000   0.000 
                                 0.000   1.000  -0.000 
                                 0.000   0.000   1.000 
      #2:
          parent:                body
          child:                 stereo_l
          T_child_wrt_parent:    0.176  -0.025   0.003 
          RPY_parent_to_child:   0.0    90.0    90.0  
          R_child_to_parent:     0.000  -0.000   1.000 
                                 1.000   0.000  -0.000 
                                -0.000   1.000   0.000 
      #3:
          parent:                body
          child:                 ground
          T_child_wrt_parent:    0.000   0.000   0.033 
          RPY_parent_to_child:   0.0     0.0     0.0  
          R_child_to_parent:     1.000  -0.000   0.000 
                                 0.000   1.000  -0.000 
                                 0.000   0.000   1.000 
      

      I cannot seem to get this to work so any help is appreciated!!

      ModeratorM 1 Reply Last reply Reply Quote 0
      • ModeratorM
        Moderator ModalAI Team @Gary Holmgren
        last edited by

        @Gary-Holmgren there is mounting guidance here: https://docs.modalai.com/configure-extrinsics/#stereo-camera-mounting

        Gary HolmgrenG 1 Reply Last reply Reply Quote 0
        • Gary HolmgrenG
          Gary Holmgren @Moderator
          last edited by

          @Moderator I have already followed these exact requirements in the procedure of designing and installing the sensors. Im not sure what else I could try.

          ModeratorM 1 Reply Last reply Reply Quote 0
          • ModeratorM
            Moderator ModalAI Team @Gary Holmgren
            last edited by

            @Gary-Holmgren is your checkerboard mounted perfectly flat on a very stiff backing? If there is any inconsistencies in the checkerboard it can cause issues

            Gary HolmgrenG 1 Reply Last reply Reply Quote 0
            • Gary HolmgrenG
              Gary Holmgren @Moderator
              last edited by Gary Holmgren

              @Moderator When checking the sensors it appears one of the stereo cams is blurry due to the lens being scuffed, I cannot get it clean so going to have to get a new one as its probably causing the issue.

              1 Reply Last reply Reply Quote 0
              • First post
                Last post
              Powered by NodeBB | Contributors