Link VIO to stereo instead of tracking camera

[marian]

Hello,

I'd like your advice on how to connect VIO to the stereo camera instead of the tracking camera. Is it only possible through the extrinsic configuration file or is there a tool that allows you to do it?

Thanks!

[marian]

Hello,

I changed the extrinsics.conf to add the reference between imu0 and imu1 to stereo_l, and changed the voxl-qvio-server.conf tu use the stereo camera for qvio instead of tracking camera.

I have two problems :

1. when I use the voxl tool to change the extrinsics configuration, it added par default the conf cor imu0 / imu1 to the tracking camera

2. when I restart the qvio server with the new configuration file it recognise my conf for stereo (I used stereo_l) but it use the tracking intrinsics.yml file instaead of the stero intrinsics conf. And, of course I don't get localisation by stereo camera

Below is the output of the qvio server:

voxl-qvio-server
loading qvio config file

imu_name: imu1
cam_name: stereo_l
odr_hz: 30.000
use_camera_height_bootstrap: 1
camera_height_off_ground_m: 0.225
enable_init_while_moving: 1
cam_imu_timeshift_s: 0.002
cam_imu_timeshift_s_uncertainty: 0.001
T_cam_wrt_imu_uncertainty: 0.005 0.005 0.005
R_cam_to_imu_uncertainty: 0.040 0.040 0.040
accl_fsr_ms2: 156.000
gyro_fsr_rad: 34.000
accl_noise_std_dev: 0.316
gyro_noise_std_dev: 0.010
cam_noise_std_dev: 100.000
min_std_pixel_noise: 0.500
fail_high_pixel_noise_points: 1.665
limited_imu_bw_trigger: 35.000
gps_imu_time_alignment_s: 0.000
T_gps_wrt_imu: -0.115 0.450 0.100
enable_mapping: 1
enable_gps_vel: 0

exising instance of qvio-server found, attempting to stop it
setting scheduler
set FIFO priority successfully!
loading extrinsics config file
#0:
parent: imu1
child: stereo_l
T_child_wrt_parent: 0.012 0.055 0.008
RPY_parent_to_child: 0.0 90.0 90.0
R_child_to_parent: 0.000 -0.000 1.000
1.000 0.000 -0.000
-0.000 1.000 0.000
tbc: 0.012 0.055 0.008
ombc: 1.209 1.209 1.209 (axis angle)
ombc: 0.000 90.000 90.000 (RPY deg)
loading /data/modalai/opencv_tracking_intrinsics.yml
using camera intrinsics:

Could you let me know what I am missing there, please?

[Techsawi Techsawi]

Hi @marian I'm trying the same solution as my tracking camera is not working, are you able to get it working this way?
yocto:/etc/modalai$ voxl-qvio-server
loading qvio config file

imu_name: imu1
cam_name: stereo_l
odr_hz: 30.000
use_camera_height_bootstrap: 1
camera_height_off_ground_m: 0.100
enable_init_while_moving: 1
cam_imu_timeshift_s: 0.002
cam_imu_timeshift_s_uncertainty: 0.001
T_cam_wrt_imu_uncertainty: 0.005 0.005 0.005
R_cam_to_imu_uncertainty: 0.040 0.040 0.040
accl_fsr_ms2: 156.000
gyro_fsr_rad: 34.000
accl_noise_std_dev: 0.316
gyro_noise_std_dev: 0.010
cam_noise_std_dev: 100.000
min_std_pixel_noise: 0.500
fail_high_pixel_noise_points: 1.665
limited_imu_bw_trigger: 35.000
gps_imu_time_alignment_s: 0.000
T_gps_wrt_imu: -0.115 0.450 0.100
enable_mapping: 1
enable_gps_vel: 0

exising instance of qvio-server found, attempting to stop it
setting scheduler
WARNING Failed to set priority, errno = 1
This seems to be a problem with ADB, the scheduler
should work properly when this is a background process
ERROR failed to set scheduler
loading extrinsics config file
#0:
parent: imu1
child: stereo_l
T_child_wrt_parent: 0.017 0.015 0.013
RPY_parent_to_child: 0.0 0.0 90.0
R_child_to_parent: 0.000 -1.000 0.000
1.000 0.000 -0.000
0.000 0.000 1.000
tbc: 0.017 0.015 0.013
ombc: 0.000 0.000 1.571 (axis angle)
ombc: 0.000 0.000 90.000 (RPY deg)
loading /data/modalai/opencv_tracking_intrinsics.yml
using camera intrinsics:
focal lengths: 291.001662 291.248381
principle points: 303.812740 244.632261
distortion: -0.007658 -0.008713 0.013827 -0.003393
MachineVision is licensed as community user
LNX_8074 supported? 1
LNX_8096 supported? 1
LNX_IA64 supported? 1
WINDOWS supported? 0
AR ERROR: arFileOpen(): Failed to open file: /etc/modalai/vislam/Configuration.SF.xml
FASTCV: fcvAvailableHardware Linux
mempool cur block size 307200, new block size 307200
Please ignore the error about Configuration.SF.xml above. ^^^
It's an optional file, and should be a warning not an error
waiting for imu
waiting for cam
connected to imu server

[marian]

No, it doesn't and I did't heard anything from the Modalai's team in this thread

In fact there are 2 problems:

1. Extrinsics.conf:

• when loading a new conf file via voxl tool, if the imu0/traching and imu1/tracking conf is not in the file, the voxl tool will automatically add a default conf of these entries at the and of your conf file

2. QVIO server:

• when starting it will load default the opencv_tracking conf as below and can't change it from the conf :

loading /data/modalai/opencv_tracking_intrinsics.yml

So, the system seems to be blocked to use only tracking camera for VIO

[A Former User]

Hi All,

The MVVISLAM library is very touchy and pretty much requires a camera with a wide field of view to give good data. We're aware that the current setup is not very dynamic and are progressing on a couple different solutions:

Firstly, we've implemented pipelines for a ov9782 sensor with color, 720p, and a field of view that's about halfway between the very-fisheye tracking camera and the current ov7251 pinhole camera. we've primarily been testing the software for this on the VOXL2 platform but the drivers are working on VOXL and we're hoping to have them up for sale in the coming months.

We've also been updating voxl-qvio-server to be much more dynamic, the dev branch has had a bunch of updates removing some hardcoded places referencing the tracking camera with the intention of supporting the new higher-resolution camera. This code has been proved out on VOXL2 and we currently have a couple platforms flying vio with the new cameras in our lab. We haven't run through extensive testing on voxl1 yet but if you'd like a head start on this the source and build instructions for what we're testing right now is available on the dev branch of the qvio server (though note that this is populated with near daily updates from our team and will often include experimental code).

[Steve Turner]

Alex,

This new work looks great on the dev branch. We might give it a try to see how it performs.

Highjacking the thread a bit -- have you looked at other VIO algorithms/implementations? Or is QC MVVISLAM about the best you guys have found on the platform, especially given what I imagine is hardware acceleration provided by the library?

Thanks!

[Andy Jarosz]

I'd also love to see an option for 2 tracking cameras, for an even wider tracking FOV.

[marian]

@Alex-Gardner Thank you, is looking good and promising for this use case. I'll give it a try.