Link VIO to stereo instead of tracking camera



  • Hello,

    I'd like your advice on how to connect VIO to the stereo camera instead of the tracking camera. Is it only possible through the extrinsic configuration file or is there a tool that allows you to do it?

    Thanks!



  • Hello,

    I changed the extrinsics.conf to add the reference between imu0 and imu1 to stereo_l, and changed the voxl-qvio-server.conf tu use the stereo camera for qvio instead of tracking camera.

    I have two problems :

    1. when I use the voxl tool to change the extrinsics configuration, it added par default the conf cor imu0 / imu1 to the tracking camera

    2. when I restart the qvio server with the new configuration file it recognise my conf for stereo (I used stereo_l) but it use the tracking intrinsics.yml file instaead of the stero intrinsics conf. And, of course I don't get localisation by stereo camera

    Below is the output of the qvio server:

    voxl-qvio-server
    loading qvio config file

    imu_name: imu1
    cam_name: stereo_l
    odr_hz: 30.000
    use_camera_height_bootstrap: 1
    camera_height_off_ground_m: 0.225
    enable_init_while_moving: 1
    cam_imu_timeshift_s: 0.002
    cam_imu_timeshift_s_uncertainty: 0.001
    T_cam_wrt_imu_uncertainty: 0.005 0.005 0.005
    R_cam_to_imu_uncertainty: 0.040 0.040 0.040
    accl_fsr_ms2: 156.000
    gyro_fsr_rad: 34.000
    accl_noise_std_dev: 0.316
    gyro_noise_std_dev: 0.010
    cam_noise_std_dev: 100.000
    min_std_pixel_noise: 0.500
    fail_high_pixel_noise_points: 1.665
    limited_imu_bw_trigger: 35.000
    gps_imu_time_alignment_s: 0.000
    T_gps_wrt_imu: -0.115 0.450 0.100
    enable_mapping: 1
    enable_gps_vel: 0

    exising instance of qvio-server found, attempting to stop it
    setting scheduler
    set FIFO priority successfully!
    loading extrinsics config file
    #0:
    parent: imu1
    child: stereo_l
    T_child_wrt_parent: 0.012 0.055 0.008
    RPY_parent_to_child: 0.0 90.0 90.0
    R_child_to_parent: 0.000 -0.000 1.000
    1.000 0.000 -0.000
    -0.000 1.000 0.000
    tbc: 0.012 0.055 0.008
    ombc: 1.209 1.209 1.209 (axis angle)
    ombc: 0.000 90.000 90.000 (RPY deg)
    loading /data/modalai/opencv_tracking_intrinsics.yml
    using camera intrinsics:

    Could you let me know what I am missing there, please?



  • Hi @marian I'm trying the same solution as my tracking camera is not working, are you able to get it working this way?
    yocto:/etc/modalai$ voxl-qvio-server
    loading qvio config file

    imu_name: imu1
    cam_name: stereo_l
    odr_hz: 30.000
    use_camera_height_bootstrap: 1
    camera_height_off_ground_m: 0.100
    enable_init_while_moving: 1
    cam_imu_timeshift_s: 0.002
    cam_imu_timeshift_s_uncertainty: 0.001
    T_cam_wrt_imu_uncertainty: 0.005 0.005 0.005
    R_cam_to_imu_uncertainty: 0.040 0.040 0.040
    accl_fsr_ms2: 156.000
    gyro_fsr_rad: 34.000
    accl_noise_std_dev: 0.316
    gyro_noise_std_dev: 0.010
    cam_noise_std_dev: 100.000
    min_std_pixel_noise: 0.500
    fail_high_pixel_noise_points: 1.665
    limited_imu_bw_trigger: 35.000
    gps_imu_time_alignment_s: 0.000
    T_gps_wrt_imu: -0.115 0.450 0.100
    enable_mapping: 1
    enable_gps_vel: 0

    exising instance of qvio-server found, attempting to stop it
    setting scheduler
    WARNING Failed to set priority, errno = 1
    This seems to be a problem with ADB, the scheduler
    should work properly when this is a background process
    ERROR failed to set scheduler
    loading extrinsics config file
    #0:
    parent: imu1
    child: stereo_l
    T_child_wrt_parent: 0.017 0.015 0.013
    RPY_parent_to_child: 0.0 0.0 90.0
    R_child_to_parent: 0.000 -1.000 0.000
    1.000 0.000 -0.000
    0.000 0.000 1.000
    tbc: 0.017 0.015 0.013
    ombc: 0.000 0.000 1.571 (axis angle)
    ombc: 0.000 0.000 90.000 (RPY deg)
    loading /data/modalai/opencv_tracking_intrinsics.yml
    using camera intrinsics:
    focal lengths: 291.001662 291.248381
    principle points: 303.812740 244.632261
    distortion: -0.007658 -0.008713 0.013827 -0.003393
    MachineVision is licensed as community user
    LNX_8074 supported? 1
    LNX_8096 supported? 1
    LNX_IA64 supported? 1
    WINDOWS supported? 0
    AR ERROR: arFileOpen(): Failed to open file: /etc/modalai/vislam/Configuration.SF.xml
    FASTCV: fcvAvailableHardware Linux
    mempool cur block size 307200, new block size 307200
    Please ignore the error about Configuration.SF.xml above. ^^^
    It's an optional file, and should be a warning not an error
    waiting for imu
    waiting for cam
    connected to imu server



  • No, it doesn't and I did't heard anything from the Modalai's team in this thread 😞

    In fact there are 2 problems:

    1. Extrinsics.conf:
    • when loading a new conf file via voxl tool, if the imu0/traching and imu1/tracking conf is not in the file, the voxl tool will automatically add a default conf of these entries at the and of your conf file
    1. QVIO server:
    • when starting it will load default the opencv_tracking conf as below and can't change it from the conf :

    loading /data/modalai/opencv_tracking_intrinsics.yml

    So, the system seems to be blocked to use only tracking camera for VIO


  • Dev Team

    Hi All,

    The MVVISLAM library is very touchy and pretty much requires a camera with a wide field of view to give good data. We're aware that the current setup is not very dynamic and are progressing on a couple different solutions:

    Firstly, we've implemented pipelines for a ov9782 sensor with color, 720p, and a field of view that's about halfway between the very-fisheye tracking camera and the current ov7251 pinhole camera. we've primarily been testing the software for this on the VOXL2 platform but the drivers are working on VOXL and we're hoping to have them up for sale in the coming months.

    We've also been updating voxl-qvio-server to be much more dynamic, the dev branch has had a bunch of updates removing some hardcoded places referencing the tracking camera with the intention of supporting the new higher-resolution camera. This code has been proved out on VOXL2 and we currently have a couple platforms flying vio with the new cameras in our lab. We haven't run through extensive testing on voxl1 yet but if you'd like a head start on this the source and build instructions for what we're testing right now is available on the dev branch of the qvio server (though note that this is populated with near daily updates from our team and will often include experimental code).



  • Alex,

    This new work looks great on the dev branch. We might give it a try to see how it performs.

    Highjacking the thread a bit -- have you looked at other VIO algorithms/implementations? Or is QC MVVISLAM about the best you guys have found on the platform, especially given what I imagine is hardware acceleration provided by the library?

    Thanks!



  • I'd also love to see an option for 2 tracking cameras, for an even wider tracking FOV.



  • @Alex-Gardner Thank you, is looking good and promising for this use case. I'll give it a try.


Log in to reply