Stereo Image sensor selection

SKA

@Alex-Kushleyev We have been advised that the ov sensor modules for dfs are approaching end of life. Which camera options do you recommend we consider instead? Thank you for your help!

Alex Kushleyev

Hello @SKA,

The replacement for OV7251 in DFS use case (Depth From Stereo) will be AR0144 which is a 1280x800 resolution global shutter sensor, compared to 640x480 OV7251. The AR0144 camera is already available and we use for VIO (tracking) cameras.

You would probably want to use the micro-coax version of AR0144 (M0166) because it will be much more flexible in terms of getting the wiring connected to VOXL2 (so M0166 + M0173 (Starling Camera Front End). Different cable lengths are available to support different base lines.

Take a look at some camera bundle options : https://docs.modalai.com/voxl2-coax-camera-bundles/

We are working on testing this configuration of dual AR0144 in DFS application and updating the DFS server to optimize performance. I expect to have an update in the next 1-2 weeks regarding the DFS performance.

Alex

SKA

Hi @Alex-Kushleyev! I am trying to get my stereo AR0144 cam setup working. I originally had AR0144 stereo+DFS working with the flex sensor cameras, but after shifting to M0188 + microcoax cameras I haven't been able to accomplish the same.

Currently I have 2 AR0144 microcoax cams connected to port J0 and J6 of the M0188 board. When I configure the cameras and run voxl-camera-server, I see two separate video outputs from stereo (1280x800) and stereo_child (1280x800) respectively. Then, when I go into voxl-camera-server.conf and adjust it to stereo mode by adding camera_second_id, I see the new stereo output (1280x1600) but video is either empty or all dark. I have all the necessary intrinsics and extrinsics files that worked previously with the flex connector cameras. Let me know if I'm missing anything, and how I can try to get this working. Thanks!

I'm using voxl-camera-server version: 2.2.6-202508142059
Here's the output when I run voxl-camera-server:

Setting MISP AWB to Auto
MISP channels enabled in defaults : 0
MISP channels enabled in config file: 0
=================================================================
configuration for 1 cameras:

cam #0
    name:                stereo
    type:                ar0144
    bayer_type:          0
    enabled:             1
    camera_id:           0
    camera_id_second:    2
    fps:                 30
    en_rotate:           0
    en_rotate2:          0

    en_preview:          1
    en_raw_preview:      1
    preview_width:       1280
    preview_height:      800

    en_misp:             1
    misp_width:          1280
    misp_height:         800

    en_small_video:      0
    small_video_width:   -1
    small_video_height:  -1

    en_large_video:      0
    large_video_width:   -1
    large_video_height:  -1

    en_snapshot:         0
    snap_width:          -1
    snap_height:         -1
    exif_focal_length:   0.000000
    exif_focal_len_35mm_format:0
    exif_fnumber:        0.000000

    ae_mode:             lme_msv
    msv_exposure_min_us: 20
    msv_exposure_max_us: 12000
    gain_min           : 54
    gain_max           : 8000
    standby_enabled:     0
    decimator:           1
    independent_exposure:0

fsync_en:            1
fsync_gpio:          109
=================================================================
thread is locked to cores: 4 5 6 7
connected to mavlink pipe
Connected to cpu-monitor
Starting Camera: stereo (id #0)
Opened GBM fd
gbm_create_device(156): Info: backend name is: msm_drm
Created GBM device
MISP Initializing for camera stereo
 Detected 1 platform(s)
 Detected 1 GPU device(s)
Estimated imu dt = 0.000978s
Got unsupported format in getUVStartFromFmt, returning nullptr
WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (2000000 bps provided). Using FPS hack. scale = 1.500000
Loading intrinsics cal file stereo_intrinsics.yml for camera stereo, ret 0
stereo lens calibration (fisheye):
        w: 1280, h:800
        fx: 563.677, fy: 562.169
        cx: 663.707, cy: 390.645
        D: -0.062035, 0.065024, -0.041547, 0.008022, 0.000000
Loading extrinsics for camera stereo, ret 0
Extrinsics:
         0.0000 1.0000 0.0000
         -0.0000 0.0000 1.0000
         1.0000 -0.0000 0.0000
Loading EIS output extrinsics for camera stereo, fetch ret 0
EIS Extrinsics:
         1.0000 -0.0000 0.0000
         0.0000 1.0000 -0.0000
         0.0000 0.0000 1.0000
MISP Initializing for camera stereo_child
Got unsupported format in getUVStartFromFmt, returning nullptr
WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (2000000 bps provided). Using FPS hack. scale = 1.500000
Loading intrinsics cal file stereo_child_intrinsics.yml for camera stereo_child, ret 0
stereo_child lens calibration (fisheye):
        w: 1280, h:800
        fx: 563.677, fy: 562.169
        cx: 663.707, cy: 390.645
        D: -0.062035, 0.065024, -0.041547, 0.008022, 0.000000
Loading extrinsics for camera stereo_child, ret 0
Extrinsics:
         0.0000 -1.0000 0.0000
         0.0000 0.0000 1.0000
         -1.0000 -0.0000 0.0000
Loading EIS output extrinsics for camera stereo_child, fetch ret 0
EIS Extrinsics:
         1.0000 -0.0000 0.0000
         0.0000 1.0000 -0.0000
         0.0000 0.0000 1.0000

------ voxl-camera-server: Started 1 of 1 cameras

------ voxl-camera-server: Camera server is now running

Alex Kushleyev

@SKA , how many cameras do you have connected?

Also, what does “voxl-camera-server -l” show and what sensormodules do you have in /usr/lib/camera?

Alex

SKA

@Alex-Kushleyev I have 3 cameras connected in total. The earlier camera server output was for just the stereo pair.

Cam ID 0 connected at port J0 - AR0144
Cam ID 1 connected at port J1 - IMX412
Cam ID 2 connected at port J6 - AR0144

I have experimented with using all of the ar0144 combo drivers, currently I have com.qti.sensormodule.ar0144_fsin_combo_0 and com.qti.sensormodule.ar0144_fsin_combo_6.

Using these drivers, the video streams appear normal when voxl-camera-server is configured to have the 2 AR0144 cameras stream separately instead of as a stereo pair.

When I run voxl-camera-server -l, it recognizes 3 cameras connected. I'll update with the exact output soon.

Alex Kushleyev

@SKA ,

If you would like to get the joined stereo image, i believe you need to disable MISP. currently you have en_misp set to true. Please set it to false and try again.

MISP is not set up to work with joining two images into one. Disabling MISP will default to the original behavior that you are expecting.

I should add a warning about this, when MISP is enabled. Thank you.

Please let me know if disabling MISP works for you.

Alex