VIO Indoor Navigation
-
Hello @modaltb ,
We are trying to run MAVROS on voxl m500, but we're facing issues with the VIO. The VIO is enabled in the voxl-vision-px4 configuration file. However, when checking the x and y values in the odometry mavlink telemetry packets they're found to be 0s.
The position mode is continuously rejected and it seems like the VIO data is not being transmitted. We did not change the flight core's PX4-EKF2 settings yet we tried to use the helper file here: https://gitlab.com/voxl-public/flight-core-px4/px4-parameters/-/blob/master/helpers/ekf2_indoor_vio.paramsand we still did not manage to get it to work. Could you please help?
-
Hi @Maha-Riyad ,
One way to debug is using voxl-vision-px4 via command line, with the 0.6.8 production release, there's a
-ooption that prints out odometry data straight out of voxl-vision-px4 which is useful.If you run
voxl-vision-px4 -ofrom command line, how are things looking there? -
I forgot, James has a great video here too: https://www.youtube.com/watch?v=0j7CjmDvluM&feature=emb_logo
-
Hi @modaltb,
I'm getting the exact same problem with the same parameter settings. I tried running voxl-vision-px4 -o and I was seeing no messages when attempting to switch to position control. I'm also not seeing any Odometry section in MAVLink Inspector.
I'm seeing no errors when running voxl-vision-px4 as a service or in command line.
-
Hi @Lynn-Koudsi ,
Hmmm, this one is confusing me a bit, if you're available maybe we could setup a screen share on Monday to troubleshoot? I'm be available 8:15-10:00, 12-2 and 4-5 PST, or Tuesday anytime after 8:15 AM PST.
Thanks!!
-
@modaltb 12-2 works!
Also just for more context, I had done a clean image installation at V3.1.0 and had removed my data folder. I'm thinking the issue is that calibration files are not there. I tried running voxl-cam-ros but it's not finding left and right yaml files. I tried just pushing the sample calibration files from https://gitlab.com/voxl-public/utilities/voxl-camera-calibration but it won't find ost.yaml.
So I guess my question is, is there a default data folder available online? Or can you provide the default factory data folder?Thanks!
-
Just sent out an invite for tomorrow, if I can get the SN of the m500 I can get you the calibration files we have in our production database in advance and that might get things going!
-
@modaltb Thank you!
Serial# is M20000000TJ
-
Hi @modaltb
I'm getting the same problem as described by Lynn. I had also done a clean image installation at V3.1.0 and had removed my data folder.
yocto:~# voxl-vision-px4 -o loading our own config file ================================================================= Parameters as loaded from config file: qgc_ip: 192.168.43.84 en_localhost_mavlink_udp 1 en_secondary_qgc: 0 secondary_qgc_ip: 192.168.1.214 qgc_udp_port_number: 14550 localhost_udp_port_number: 14551 en_vio: 1 en_voa: 0 en_send_vio_to_qgc: 0 en_send_voa_to_qgc: 0 en_adsb: 1 adsb_uart_bus: 7 adsb_uart_baudrate: 57600 px4_uart_bus: 5 px4_uart_baudrate: 921600 offboard_mode: off follow_tag_id: 0 en_apriltag_fixed_frame: 0 fixed_frame_filter_len: 5 en_transform_mavlink_pos_setpoints_from_fixed_frame:0 ================================================================= loading extrinsics and apriltag external config files #0: tag id: 0 name: default_name location: fixed size_m: 0.400 T_tag_wrt_fixed: 0.0 0.0 0.0 R_tag_to_fixed: 0.0 -1.0 0.0 1.0 0.0 0.0 0.0 0.0 1.0 #1: tag id: 1 name: default_name location: fixed size_m: 0.400 T_tag_wrt_fixed: 0.0 0.0 0.0 R_tag_to_fixed: 0.0 -1.0 0.0 1.0 0.0 0.0 0.0 0.0 1.0 loaded in 2 fixed apriltags #0: parent: imu1 child: imu0 T_child_wrt_parent: -0.048 0.037 0.002 RPY_parent_to_child: 0.0 0.0 0.0 R_child_to_parent: 1.000 -0.000 0.000 0.000 1.000 -0.000 0.000 0.000 1.000 #1: parent: imu0 child: tracking T_child_wrt_parent: 0.065 -0.014 0.013 RPY_parent_to_child: 0.0 45.0 90.0 R_child_to_parent: 0.000 -0.707 0.707 1.000 0.000 -0.000 -0.000 0.707 0.707 #2: parent: imu1 child: tracking T_child_wrt_parent: 0.017 0.015 0.013 RPY_parent_to_child: 0.0 45.0 90.0 R_child_to_parent: 0.000 -0.707 0.707 1.000 0.000 -0.000 -0.000 0.707 0.707 #3: parent: body child: imu0 T_child_wrt_parent: 0.020 0.014 -0.008 RPY_parent_to_child: 0.0 0.0 0.0 R_child_to_parent: 1.000 -0.000 0.000 0.000 1.000 -0.000 0.000 0.000 1.000 #4: parent: body child: imu1 T_child_wrt_parent: 0.068 -0.015 -0.008 RPY_parent_to_child: 0.0 0.0 0.0 R_child_to_parent: 1.000 -0.000 0.000 0.000 1.000 -0.000 0.000 0.000 1.000 #5: parent: body child: stereo_l T_child_wrt_parent: 0.100 -0.040 0.000 RPY_parent_to_child: 0.0 90.0 90.0 R_child_to_parent: 0.000 -0.000 1.000 1.000 0.000 -0.000 -0.000 1.000 0.000 #6: parent: body child: tof T_child_wrt_parent: 0.100 0.000 0.000 RPY_parent_to_child: 0.0 90.0 90.0 R_child_to_parent: 0.000 -0.000 1.000 1.000 0.000 -0.000 -0.000 1.000 0.000 #7: parent: body child: ground T_child_wrt_parent: 0.000 0.000 0.100 RPY_parent_to_child: 0.0 0.0 0.0 R_child_to_parent: 1.000 -0.000 0.000 0.000 1.000 -0.000 0.000 0.000 1.000 exising instance of voxl-vision-px4 found, attempting to stop it starting geometry module starting uart mavlink Successfully opened mavparser starting udp mavlink Detected PX4 Mavlink SYSID 1 Adding manual QGC IP address to udp connection list: 192.168.43.84 Added new UDP connection to 192.168.43.84 starting px4 shell starting fixed pose input starting vio manager starting apriltag manager starting adsb manager ADSB thread starting Init complete, entering main loop PX4 Connected over UART with sysid 1But when I run 'voxl-test-vision-lib -o' I can see the VIO works.
... [6619.210687] SNAV INFO: IMU sensor finished initializing. [6619.211066] SNAV INFO: IMU sensor settings retrieved. [6619.211122] SNAV INFO: Creating IMU thread. [6619.211332] SNAV INFO: Creating VISLAM processing thread. [6619.211463] SNAV INFO: ImuManager thread has tid = 3558, nice = -15 [6619.211483] SNAV INFO: Creating VISLAM publishing thread. [6619.211768] SNAV INFO: VISLAM publishing thread has tid = 3560, nice = -5 VIO camera frame: points: 0 X: 0.000 Y: 0.000 Z: 0.000 R: 0.000 P: -0.000 Y: 0.000 [6619.211958] SNAV INFO: VISLAM processing thread has tid = 3559, nice = -5 VIO camera frame: points: 0 X: 0.000 Y: 0.000 Z: 0.000 R: 0.000 P: -0.000 Y: 0.000 VIO camera frame: points: 0 X: 0.000 Y: 0.000 Z: 0.000 R: 0.000 P: -0.000 Y: 0.000 VIO camera frame: points: 0 X: 0.000 Y: 0.000 Z: 0.000 R: 0.000 P: -0.000 Y: 0.000 VIO camera frame: points: 0 X: 0.000 Y: 0.000 Z: 0.000 R: 0.000 P: -0.000 Y: 0.000 VIO camera frame: points: 0 X: 0.000 Y: 0.000 Z: 0.000 R: 0.000 P: -0.000 Y: 0.000 VIO camera frame: points: 39 X: 0.000 Y: 0.000 Z: 0.000 R: 0.000 P: -0.000 Y: 0.000 VIO camera frame: points: 39 X: 0.000 Y: 0.000 Z: 0.000 R: 0.000 P: -0.000 Y: 0.000 VIO camera frame: points: 39 X: 0.000 Y: 0.000 Z: 0.000 R: 0.000 P: -0.000 Y: 0.000 VIO camera frame: points: 39 X: -0.000 Y: 0.000 Z: -0.000 R: -0.000 P: -0.000 Y: -0.000 VIO camera frame: points: 39 X: -0.000 Y: 0.000 Z: 0.000 R: -0.000 P: 0.000 Y: -0.000 VIO camera frame: points: 39 X: -0.000 Y: 0.001 Z: 0.000 R: -0.001 P: -0.000 Y: -0.000 VIO camera frame: points: 39 X: -0.001 Y: -0.002 Z: -0.005 R: 0.001 P: -0.001 Y: -0.000 VIO camera frame: points: 39 X: 0.000 Y: -0.000 Z: -0.008 R: 0.000 P: -0.001 Y: -0.001 VIO camera frame: points: 39 X: -0.000 Y: -0.001 Z: -0.010 R: 0.001 P: -0.002 Y: -0.001 VIO camera frame: points: 39 X: -0.000 Y: -0.000 Z: -0.014 R: 0.002 P: -0.003 Y: -0.001 VIO camera frame: points: 39 X: 0.002 Y: 0.002 Z: -0.019 R: 0.001 P: -0.003 Y: -0.002 VIO camera frame: points: 39 X: 0.003 Y: 0.002 Z: -0.022 R: 0.001 P: -0.004 Y: -0.002 VIO camera frame: points: 39 X: 0.004 Y: 0.002 Z: -0.023 R: 0.001 P: -0.004 Y: -0.001 VIO camera frame: points: 39 X: 0.004 Y: 0.001 Z: -0.020 R: 0.001 P: -0.004 Y: -0.001 VIO camera frame: points: 39 X: 0.002 Y: 0.001 Z: -0.015 R: 0.001 P: -0.004 Y: -0.001In QGroundControl, I can not see the ODOMETRY section in the MAVLink inspector.
My serial # M20000005GT
-
@Liping My issue was resolved by installing voxl-qvio. You can reference the diagram here to learn what services are needed for the different interactions.
-
@Lynn-Koudsi Thanks Lynn. But in my case, the issue still exists after installing the voxl-qvio.
What are your installed voxl package versions?
I installed some latest stable version packages.
yocto:~# opkg list-installed | grep "voxl" libvoxl_io - 0.5.4 voxl-cam-manager - 0.2.2 voxl-docker-support - 1.1.1 voxl-hal3-tof-cam-ros - 0.0.2 voxl-modem - 0.10.0 voxl-mpa-tools - 0.1.6 voxl-nodes - 0.0.8 voxl-qvio-server - 0.2.1 voxl-rtsp - 1.0.2 voxl-time-sync - 0.0.1 voxl-utils - 0.5.2 voxl-vision-px4 - 0.8.1 voxl-vpn - 0.0.2 voxl_imu - 0.0.4 -
@Liping Is voxl-qvio actually running? You can check by calling
systemctl status voxl-qvio-serverit might be that you need calibration files for the tracking camera -
@Lynn-Koudsi It shows waiting for imu and camera. Could you help to point out where is the calibration file located? Is it this calibration process (https://docs.modalai.com/calibrate-cameras/) that you refer to?
yocto:~# systemctl status voxl-qvio-server ā voxl-qvio-server.service - voxl-qvio-server Loaded: loaded (/usr/bin/voxl-qvio-server; disabled; vendor preset: enabled) Active: active (running) since Mon 2021-03-08 15:13:41 UTC; 7min ago Main PID: 3199 (voxl-qvio-serve) CGroup: /system.slice/voxl-qvio-server.service āā3199 /usr/bin/voxl-qvio-server Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: LNX_8096 supported? 1 Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: LNX_IA64 supported? 1 Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: WINDOWS supported? 0 Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: AR ERROR: arFileOpen(): Failed to open file: //vislam/Configuration.SF.xml Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: FASTCV: fcvAvailableHardware Linux Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: mempool cur block size 307200, new block size 307200 Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: Please ignore the error about Configuration.SF.xml above. ^^^ Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: It's an optional file, and should be a warning not an error Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: waiting for imu Mar 08 15:13:42 apq8096 voxl-qvio-server[3199]: waiting for camBefore, I was using the following two command to set the configuration file:
voxl-configure-vision-px4 voxl-configure-modalai-vl -
@Liping that's a curious problem... try checking the status of the imu and camera servers:
systemctl status voxl-camera-server
systemctl status voxl-imu-serverIf you check the status of voxl-qvio again, does it say it failed? or it waits forever?
You should have the following calibration files:
etc/snav/calibration.stereo.xml home/root/.ros/camera_info/left.yaml home/root/.ros/camera_info/right.yaml etc/snav/calibration.downward.xml home/root/.ros/camera_info/downward.yamlBut I'm pretty sure you're going to need
/data/modalai/opencv_stereo_intrinsics.yml /data/modalai/opencv_stereo_extrinsics.ymlI got those by running the calibration process you referenced.
-
@Liping I just realized that you don't have
voxl-camera-serverinstalled. You will need that. I also found that having bothvoxl-camera-serverand voxl-cam-managerhas given me some issues so I would recommend uninstallingvoxl-cam-manager -
-
@Lynn-Koudsi Thanks a lot! The issue has been solved. The QGroundControl shows meaningful ODOMETRY data now. As you mentioned, the issue comes from the voxl-camera-server and voxl-imu-server. voxl-qvio can not get data from both the camera and the imu.
Besides, I noticed that the configuration of the camera and imu does not work as expected. It may be caused by the conflict of the old (imp_app, voxl-cam-manager)and new (voxl-camera-server, voxl-imu-server) camera and imu ipk.
To make it simple, I flashed the image with the System Image 3.2.0 (https://developer.modalai.com/asset/eula-download/64) without installing the voxl-suite, since I want to use the new Modal Pipe Architecture (https://gitlab.com/voxl-public/modal-pipe-architecture).
I manually installed the following voxl packages:
yocto:/# opkg list-installed | grep "libmodal" libmodal_json - 0.3.4 libmodal_pipe - 1.7.6 yocto:/# opkg list-installed | grep "voxl" libvoxl_io - 0.5.4 voxl-camera-server - 0.5.5 voxl-dfs-server - 0.0.7 voxl-docker-support - 1.1.1 voxl-imu-server - 0.7.8 voxl-mpa-tools - 0.1.6 voxl-qvio-server - 0.2.1 voxl-utils - 0.5.9 voxl-vision-px4 - 0.8.1Then I run the configuration commands for the qvio, voxl-vision-px4 and camera (voxl-imu-server has done some default configuration during the ipk installation):
yocto:/# voxl-configure-qvio -f rm: cannot remove '/etc/modalai/voxl-qvio-server.conf': No such file or directory loading qvio config file Creating new config file: /etc/modalai/voxl-qvio-server.conf The config file was modified during parsing, saving the changes to disk enabling voxl-qvio-server systemd service Created symlink from /etc/systemd/system/multi-user.target.wants/voxl-qvio-server.service to /etc/systemd/system/voxl-qvio-server.service. starting voxl-qvio-server systemd service Done configuring voxl-qvio-server yocto:/# voxl-configure-vision-px4 -f rm: cannot remove '/etc/modalai/voxl-vision-px4.conf': No such file or directory Created new empty json file: /etc/modalai/voxl-vision-px4.conf The JSON config file data was modified during parsing, saving the changes to disk enabling voxl-vision-px4 systemd service Created symlink from /etc/systemd/system/multi-user.target.wants/voxl-vision-px4.service to /etc/systemd/system/voxl-vision-px4.service. starting voxl-vision-px4 systemd service DONE configuring voxl-vision-px4 yocto:/# voxl-configure-cameras ...Regarding the configuration of the camera, i was following the guidance in https://gitlab.com/voxl-public/modal-pipe-architecture/voxl-camera-server to use the command
voxl-configure-camerasprovided byvoxl-utilspackage.I rebooted the VOXL and checked related services (camera, imu, qvio, voxl-vision-px4) using
systemctl statusBut I noticed that the
voxl-camera-serverwas not automaticly started on boot.
So I enabled it with:yocto:/# systemctl enable voxl-camera-server@modaltb Please correct me if any step was not right or there is a better way.
In addition, I also followed the https://forum.modalai.com/topic/70/warning-px4-disconnected-error-message-from-voxl-vision-px4 to solve the WARNING PX4 DISCONNECTED error message from voxl-vision-px4
-
WOW! Thanks @Lynn-Koudsi for the help, when we setup the forum this is the kind of interaction we were hoping for, just awesome!!
Sorry for the setup and 'wonkyness' right now, we've been somewhat caught with getting pre-release software out there without all the setup/docs lined up.... so we're doing a little catchup it seems.
@Liping there until we have our docs updated, if this is working, I'd say this is a good approach!
Thank you both for the details, we'll relay this through the team.
