M0149 camera refocusing and tuning parameters

Alex Kushleyev

I will check this a bit later but can you verify that “ /usr/lib/python3.6/dist-packages/” path exists on voxl2, if not just create it “mkdir -p /usr/lib/python3.6/dist-packages/“ and then install the package again. Thanks!

Aaky

@Alex-Kushleyev Thanks Alex. This fix worked. Now I can see hires_debug over voxl-portal.

On a seperate note, I need some hardware help over AR0144 camera. Actually while we were assembling this camera over our drone, the lens holder which is black color as shown in picture below came off from one side.

M0149_angle_1920x1920 (1).jpg

In order to fix this, we poured some fevi quick onto it, this made it sturdy but now voxl isnt able to find this camera. Even voxl-camera-server -l says 1 camera detected which is TOF.
I am aware this must be some hardware problem, but can you advice me anything else if I can check?
Logs :

voxl2:~$ voxl-camera-server -d 0
=================================================================
configuration for 2 cameras:

cam #0
    name:                tracking
    sensor type:         ar0144
    isEnabled:           1
    camId:               0
    camId2:              -1
    fps:                 30
    en_rotate:           0
    en_rotate2:          0

    en_preview:          1
    pre_width:           1280
    pre_height:          800
    en_raw_preview:      1

    en_small_video:      0
    small_video_width:   -1
    small_video_height:  -1

    en_large_video:      0
    large_video_width:   -1
    large_video_height:  -1

    en_snapshot:         0
    snap_width:          -1
    snap_height:         -1

    ae_mode:             lme_msv
    standby_enabled:     0
    decimator:           1
    independent_exposure:0

cam #1
    name:                tof
    sensor type:         pmd-tof
    isEnabled:           1
    camId:               1
    camId2:              -1
    fps:                 5
    en_rotate:           0
    en_rotate2:          0

    en_preview:          1
    pre_width:           224
    pre_height:          1557
    en_raw_preview:      1

    en_small_video:      0
    small_video_width:   -1
    small_video_height:  -1

    en_large_video:      0
    large_video_width:   -1
    large_video_height:  -1

    en_snapshot:         0
    snap_width:          -1
    snap_height:         -1

    ae_mode:             off
    standby_enabled:     0
    decimator:           5
    independent_exposure:0

=================================================================
DEBUG:   Attempting to open the hal module
DEBUG:   SUCCESS: Camera module opened on attempt 0
DEBUG:   ----------- Number of cameras: 1

DEBUG:   Cam idx: 0, Cam slot: 3, Slave Address: 0x007A, Sensor Id: 0x003D
GPS server Connected 
DEBUG:   Connected to cpu-monitor
DEBUG:   ------ voxl-camera-server: Starting 2 cameras
Starting Camera: tracking (id #0)
DEBUG:   Checking Gain limits for Camera: tracking
Using gain limits min: 54 max: 8000
DEBUG:   cam ID 0 checking for fmt:   37  w: 1280 h:  800 o:   0
DEBUG:   i:  0 fmt: 34 w: 176 h: 144 o:0
DEBUG:   i:  4 fmt: 34 w: 176 h: 144 o:1
DEBUG:   i:  8 fmt: 35 w: 176 h: 144 o:0
DEBUG:   i: 12 fmt: 35 w: 176 h: 144 o:1
DEBUG:   i: 16 fmt: 33 w: 176 h: 144 o:0
DEBUG:   i: 20 fmt: 37 w: 224 h:1557 o:0
DEBUG:   i: 24 fmt: 38 w: 224 h:1557 o:0
DEBUG:   i: 28 fmt: 32 w: 224 h:1557 o:0
DEBUG:   i: 32 fmt: 36 w: 224 h:1557 o:0
ERROR:   Camera 0 failed to find supported preview config: 1280x800
WARNING: Failed to start cam tracking due to invalid resolution
WARNING: assuming cam is missing and trying to compensate
Starting Camera: tof (originally id #1) with id offset: 1
DEBUG:   Checking Gain limits for Camera: tof
Using gain limits min: 54 max: 8000
DEBUG:   cam ID 0 checking for fmt:   38  w:  224 h: 1557 o:   0
DEBUG:   i:  0 fmt: 34 w: 176 h: 144 o:0
DEBUG:   i:  4 fmt: 34 w: 176 h: 144 o:1
DEBUG:   i:  8 fmt: 35 w: 176 h: 144 o:0
DEBUG:   i: 12 fmt: 35 w: 176 h: 144 o:1
DEBUG:   i: 16 fmt: 33 w: 176 h: 144 o:0
DEBUG:   i: 20 fmt: 37 w: 224 h:1557 o:0
DEBUG:   i: 24 fmt: 38 w: 224 h:1557 o:0
DEBUG:   i: 28 fmt: 32 w: 224 h:1557 o:0
DEBUG:   i: 32 fmt: 36 w: 224 h:1557 o:0
VERBOSE: Successfully found configuration match for camera 0: 224x1557
VERBOSE: Adding preview stream for camera: 0
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
VERBOSE: Opened GBM fd
gbm_create_device(156): Info: backend name is: msm_drm
VERBOSE: Created GBM device
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Converted gralloc flags 0x20900 to GBM flags 0x1400000
VERBOSE: Dumping GBM flags
DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
DEBUG:   	Found flag GBM_BO_USAGE_HW_COMPOSER_QTI	
DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12
DEBUG:   Successfully set up pipeline for stream: PREVIEW
VERBOSE: Entered thread: cam0-request(tid: 2856)
DEBUG:   Started Camera: tof

------ voxl-camera-server: Started 1 of 2 cameras

------ voxl-camera-server: Camera server is now running
VERBOSE: Entered thread: cam0-result(tid: 2857)
VERBOSE: Found Royale module config: imagerType 2, illuminationConfig.dutyCycle: 4 temp_sensor_type: 5
VERBOSE: Found Royale module config: maxImgW 224 maxImgH 172 frameTxMode 1 camName X1.1_850nm_2W
VERBOSE: Found Royale module config: tempLimitSoft 60.000000 tempLimitHard 65.000000 autoExpoSupported yes
VERBOSE: Found Royale usecase: MODE_9_5FPS - phases: 9 fps: 5
VERBOSE: Found Royale usecase: MODE_9_5FPS - exposure group[0] = gray
VERBOSE: Found Royale usecase: MODE_9_5FPS - exposure group[1] = mod1
VERBOSE: Found Royale usecase: MODE_9_5FPS - exposure group[2] = mod2
VERBOSE: Found Royale usecase: MODE_9_5FPS - exp_limit[0] = (8 2200)
VERBOSE: Found Royale usecase: MODE_9_5FPS - exp_limit[1] = (8 2200)
VERBOSE: Found Royale usecase: MODE_9_5FPS - exp_limit[2] = (8 2200)
VERBOSE: Found Royale usecase: MODE_9_5FPS - exp_time[0] = 200
VERBOSE: Found Royale usecase: MODE_9_5FPS - exp_time[1] = 2200
VERBOSE: Found Royale usecase: MODE_9_5FPS - exp_time[2] = 2200
VERBOSE: Found Royale usecase: MODE_9_10FPS - phases: 9 fps: 10
VERBOSE: Found Royale usecase: MODE_9_10FPS - exposure group[0] = gray
VERBOSE: Found Royale usecase: MODE_9_10FPS - exposure group[1] = mod1
VERBOSE: Found Royale usecase: MODE_9_10FPS - exposure group[2] = mod2
VERBOSE: Found Royale usecase: MODE_9_10FPS - exp_limit[0] = (8 1100)
VERBOSE: Found Royale usecase: MODE_9_10FPS - exp_limit[1] = (8 1100)
VERBOSE: Found Royale usecase: MODE_9_10FPS - exp_limit[2] = (8 1100)
VERBOSE: Found Royale usecase: MODE_9_10FPS - exp_time[0] = 200
VERBOSE: Found Royale usecase: MODE_9_10FPS - exp_time[1] = 1100
VERBOSE: Found Royale usecase: MODE_9_10FPS - exp_time[2] = 1100
VERBOSE: Found Royale usecase: MODE_9_15FPS - phases: 9 fps: 15
VERBOSE: Found Royale usecase: MODE_9_15FPS - exposure group[0] = gray
VERBOSE: Found Royale usecase: MODE_9_15FPS - exposure group[1] = mod1
VERBOSE: Found Royale usecase: MODE_9_15FPS - exposure group[2] = mod2
VERBOSE: Found Royale usecase: MODE_9_15FPS - exp_limit[0] = (8 750)
VERBOSE: Found Royale usecase: MODE_9_15FPS - exp_limit[1] = (8 750)
VERBOSE: Found Royale usecase: MODE_9_15FPS - exp_limit[2] = (8 750)
VERBOSE: Found Royale usecase: MODE_9_15FPS - exp_time[0] = 200
VERBOSE: Found Royale usecase: MODE_9_15FPS - exp_time[1] = 750
VERBOSE: Found Royale usecase: MODE_9_15FPS - exp_time[2] = 750
VERBOSE: Found Royale usecase: MODE_9_20FPS - phases: 9 fps: 20
VERBOSE: Found Royale usecase: MODE_9_20FPS - exposure group[0] = gray
VERBOSE: Found Royale usecase: MODE_9_20FPS - exposure group[1] = mod1
VERBOSE: Found Royale usecase: MODE_9_20FPS - exposure group[2] = mod2
VERBOSE: Found Royale usecase: MODE_9_20FPS - exp_limit[0] = (8 560)
VERBOSE: Found Royale usecase: MODE_9_20FPS - exp_limit[1] = (8 560)
VERBOSE: Found Royale usecase: MODE_9_20FPS - exp_limit[2] = (8 560)
VERBOSE: Found Royale usecase: MODE_9_20FPS - exp_time[0] = 200
VERBOSE: Found Royale usecase: MODE_9_20FPS - exp_time[1] = 560
VERBOSE: Found Royale usecase: MODE_9_20FPS - exp_time[2] = 560
VERBOSE: Found Royale usecase: MODE_9_30FPS - phases: 9 fps: 30
VERBOSE: Found Royale usecase: MODE_9_30FPS - exposure group[0] = gray
VERBOSE: Found Royale usecase: MODE_9_30FPS - exposure group[1] = mod1
VERBOSE: Found Royale usecase: MODE_9_30FPS - exposure group[2] = mod2
VERBOSE: Found Royale usecase: MODE_9_30FPS - exp_limit[0] = (8 370)
VERBOSE: Found Royale usecase: MODE_9_30FPS - exp_limit[1] = (8 370)
VERBOSE: Found Royale usecase: MODE_9_30FPS - exp_limit[2] = (8 370)
VERBOSE: Found Royale usecase: MODE_9_30FPS - exp_time[0] = 200
VERBOSE: Found Royale usecase: MODE_9_30FPS - exp_time[1] = 370
VERBOSE: Found Royale usecase: MODE_9_30FPS - exp_time[2] = 370
VERBOSE: Found Royale usecase: MODE_5_15FPS - phases: 5 fps: 15
VERBOSE: Found Royale usecase: MODE_5_15FPS - exposure group[0] = gray
VERBOSE: Found Royale usecase: MODE_5_15FPS - exposure group[1] = mod
VERBOSE: Found Royale usecase: MODE_5_15FPS - exp_limit[0] = (8 1500)
VERBOSE: Found Royale usecase: MODE_5_15FPS - exp_limit[1] = (8 1500)
VERBOSE: Found Royale usecase: MODE_5_15FPS - exp_time[0] = 200
VERBOSE: Found Royale usecase: MODE_5_15FPS - exp_time[1] = 1500
VERBOSE: Found Royale usecase: MODE_5_30FPS - phases: 5 fps: 30
VERBOSE: Found Royale usecase: MODE_5_30FPS - exposure group[0] = gray
VERBOSE: Found Royale usecase: MODE_5_30FPS - exposure group[1] = mod
VERBOSE: Found Royale usecase: MODE_5_30FPS - exp_limit[0] = (8 750)
VERBOSE: Found Royale usecase: MODE_5_30FPS - exp_limit[1] = (8 750)
VERBOSE: Found Royale usecase: MODE_5_30FPS - exp_time[0] = 200
VERBOSE: Found Royale usecase: MODE_5_30FPS - exp_time[1] = 750
VERBOSE: Found Royale usecase: MODE_5_45FPS - phases: 5 fps: 45
VERBOSE: Found Royale usecase: MODE_5_45FPS - exposure group[0] = gray
VERBOSE: Found Royale usecase: MODE_5_45FPS - exposure group[1] = mod
VERBOSE: Found Royale usecase: MODE_5_45FPS - exp_limit[0] = (8 500)
VERBOSE: Found Royale usecase: MODE_5_45FPS - exp_limit[1] = (8 500)
VERBOSE: Found Royale usecase: MODE_5_45FPS - exp_time[0] = 200
VERBOSE: Found Royale usecase: MODE_5_45FPS - exp_time[1] = 500
VERBOSE: Found Royale usecase: MODE_5_60FPS - phases: 5 fps: 60
VERBOSE: Found Royale usecase: MODE_5_60FPS - exposure group[0] = gray
VERBOSE: Found Royale usecase: MODE_5_60FPS - exposure group[1] = mod
VERBOSE: Found Royale usecase: MODE_5_60FPS - exp_limit[0] = (8 370)
VERBOSE: Found Royale usecase: MODE_5_60FPS - exp_limit[1] = (8 370)
VERBOSE: Found Royale usecase: MODE_5_60FPS - exp_time[0] = 200
VERBOSE: Found Royale usecase: MODE_5_60FPS - exp_time[1] = 370
VERBOSE: sid: 0x00AC, addr: 0x0000, data: 0x0050
VERBOSE: sid: 0x00AC, addr: 0x0001, data: 0x004D
VERBOSE: sid: 0x00AC, addr: 0x0002, data: 0x0044
VERBOSE: sid: 0x00AC, addr: 0x0003, data: 0x0054
VERBOSE: sid: 0x00AC, addr: 0x0004, data: 0x0045
VERBOSE: sid: 0x00AC, addr: 0x0005, data: 0x0043
VERBOSE: sid: 0x00AC, addr: 0x0006, data: 0x0007
VERBOSE: sid: 0x00AC, addr: 0x0007, data: 0x0000

I dont know if we can recover this camera back it would be really helpful if someone can guide me if we can get some information about this camera also if in future this holder comes out, what should be used to fix this. My observations were due to his lens holder coming out our image was becoming completely blurry even after focusing it, also image quality reduced alot. What are better ways to fix this kind of lenses? Also better way to know if this camera has gone completely bad or can be recovered?

Alex Kushleyev

@Aaky , you should check if any components got knocked off , which are sitting on the flex cable right next to the camera lens holder. If something got knocked loose, maybe you could attempt to replace a component (I could tell you what the component is, if there is a specific one that got knocked off).

Besides this, I do not have any other suggestions, unfortunately. It may be worth just plugging in another camera into that camera slot on VOXL2 just to make sure the camera port is still works.

Alex

Aaky

@Alex-Kushleyev Thanks for the suggestion Alex.
My doubt is, Can Adhesive like Fevi quick damage the PCB or components over it? I was under impression fevi quick acts as good insulator and wouldn't cause any problems but it seems to be have gone wrong.

In future if something like this happens, what should be used for sticking the holder back?

Quick Update: Camera seems to be working but unable to focus it now. I might have to clean the surface of CMOS as well. Any suggestions in this? Current image is extremely out of focused and looks very blurry.

Alex Kushleyev

Hi @Aaky , my guess is that CA-based super glue should not damage pcb components in general (at least in short term). I dont know about this particular glue, but it should be a good insulator. It is probably not officially approved for electronics though

My worry about such glue is that its fumes (while drying) could deposit residue or damage the Image sensor surface. Super glue sometimes leaves white residue around application site.

Maybe this is what happened, you can take a look at the sensor surface..

I am curious how the lens holder came off - was a lot of force applied to it? I dont think we have had this happen before.

Aaky

@Alex-Kushleyev While assembling the sensor, it might have got some force onto it since our drone previously had OV7251 mount and this camera is significantly bigger in size.

How should I clean the sensor surface? Any particular liquid is recommended? I have lens cleaner with me does that work? Link.

Let me know if its safe to use this over sensor surface and also over lens surface.

Alex Kushleyev

@Aaky , I am not sure about that, i can check, but it should be a cleaning solution approved for cleaning camera sensors.

Aaky

@Alex-Kushleyev Thanks Alex. I managed to cleanup the lens and also is in the process of focusing the camera. Please look at below image. Is this ideal focus or can be done better? The checker board was 2.5 - 3 meters away from camera in this scenario.

Aaky

@Alex-Kushleyev One more problem which I am facing is, with above shown focus I am trying to perform camera calibration but it is always failing. On the same checker board I did calibration of new OV7251 and it worked well providing reprojection error of 0.34 at the best. With AR0144 I am always getting reprojection error of around 0.6 to 0.8 and never going below 0.5. Can this also be somehow related to focus or the repair which I did manually?

Please let me know your thoughts on both these queries.

Alex Kushleyev

@Aaky ,

I think the focus looks ok.

Keep in mind AR0144 has 2x resolution and reprojection error is reported in pixels, so 2x larger reprojection error for AR0144 roughly represents the same error as ov7251 in terms of angle that each pixel represents.

I will double check with the team what reprojection error is expected on AR0144

By the way, how did you resolve the issue of camera not being detected?

Aaky

@Alex-Kushleyev Yes I will be waiting for your input on reprojection error on AR0144.

Camera was not detecting maybe because the glue which I used was wet yesterday night. In the morning, probably after 10 hours when I checked camera was working normally. This might be only due to glue got enough time to dry off.

Also just FYI, I remember first time when I got AR0144 and I performed calibration I did get reprojection error below 0.5.

Alex Kushleyev

@Aaky , our team provided feedback that typically we get around 0.6 pixel reprojection error when calibrating AR0144, so this is consistent with your results. At least, it seems that your camera is not performing worse than expected. We do see higher variance in reprojection error when calibrated multiple times (compared to ov7251). This could be due to the convergence criteria of the calibration procedure - we did not do a thorough investigation.

Aaky

@Alex-Kushleyev Okay. Thanks for the feedback. In that case should I increase the reprojection criteria in voxl-camera-calibration package and save the calibration below 0.6 reprojection? Any other suggestions are welcome. I will perform handheld qvio tests also with this calibration. Any specific error I should be looking for in qvio? Like BAD_CAM_CAL?

Alex Kushleyev

@Aaky , yes you can adjust the threshold, since the threshold was tuned for ov7251. Reprojection error of 0.5 or 0.6 is not going to matter too much.

QVIO will complain about BAD_CAM_CAL if the calibration is REALLY bad, so it probably wont be the case here.

The ultimate metric of QVIO is drift vs distance traveled (and stability), in good feature rich environments it could get as low as few % drift over distance.

Aaky

@Alex-Kushleyev Thanks for your feedback Alex. I got re-projection error of 0.51 on AR0144 and now I have started to test QVIO in indoors. On few instances I got BAD_CAM_CAL while testing handheld. My MSV was around 70 and extrinsic is caliberated correctly as per my UAV's IMU and camera positions. Please look at below image. I didnt got any error other then BAD_CAM_CAL (ofcourse discounting IMU and vibration errors)

As a observation I didnt see odometry drift in QVIO while I roam around and came back to same origin where my odometry said same x,y and z axis to be almost 0.

Even if Quality in above image looks fine, there are good enough points for tracking still why does QVIO report BAD_CAM_CAL? I am aware QVIO is blackbox for everyone but just wanted your analysis.

Also on a very seperate note, I read somewhere on forum there is some work happening on dual camera support for QVIO? Can you provide me any early stage update on same? Even how is the software architecture for dual QVIO instance and fusion?

Alex Kushleyev

@Aaky ,

I am not sure why QVIO is reporting bad cam cal. Please double check that you entered the camera calibration parameters correctly for QVIO.

In your test image, this is actually not a great scenario for VIO because all the points are roughly in the same location and at the same distance. Try to test in a more feature rich environment and see if the bad camera calibration warning is still there.

Sometimes it is possible that something else is off and the algorithm "thinks" that it is caused by a bad camera calibration. It is trying to provide a solution within a large state space (many parameters) so it could incorrectly estimate some parameter and think the camera calibration is off.

We are not working on a dual QVIO fusion, but we are working on another flavor of VIO algorithm that supports dual cameras.

Alex

Aaky

@Alex-Kushleyev Thanks Alex. Will test this further. Any update over shading correction for corners of AR0144 camera stream as we discussed previously?

Alex Kushleyev

@Aaky ,

Sorry, there is no update just yet. it is in the queue to experiment with, but we are testing with AR0144 with the same lens shading issues and it seems the performance is good. It could potentially be better with the shading corrected, but it is not a blocking issues at this point, as we understand. Just to be realistic, I would estimate at least a few weeks before I would have something to test for this feature.

Alex

Aaky

@Alex-Kushleyev Thanks for the update. Can you provide me with camera parameters of AR0144 with which you got better results? In my tests I am getting good results in indoors but it fails in outdoors (imagine flat grass land) where in both cases camera is looking 35 degree downwards from horizon and ideally should be able to get some features. My camera reporjection error came out to be 0.51. If you can share any test bench data and parameters with which you tested QVIO with AR0144 will act as good reference point for my development.
Thank you!

Alex Kushleyev

@Aaky , camera calibration parameters are specific to each camera module, so i don't think any other calibration parameters would work for you better than what you have. Does that make sense? Or are you asking about different parameters (please specify).

I will check to see if we can provide reference data sets running QVIO with AR0144