QVIO Flight violently unstable
-
Hello, I am trying to debug my QVIO flight which is very unstable. When hovering the vehicle does not stay still instead it pitches back and forth aggressively. I can fly in manual mode and altitude mode without any issues at all, but when hovering in altitude mode and switching to position it immediately starts cyclically shaking back and forth and gets worse the longer I let it go.
Its paired with the AR0144 tracking cam and has been calibrated. This is setup to also use a GPS/MAG but indoors use VIO. there is also a VL53L1X mounted for altitude sensing below 4m. Is there something that anyone would recommend checking to see what may be happening here. I followed along with the EKF2 param helpers for VIO in px4 as well.
This is my extrinsics:
"extrinsics": [{ "parent": "imu_apps", "child": "tracking", "T_child_wrt_parent": [0.0237, 0.0024, -0.0073], "RPY_parent_to_child": [0, 90, 90] }, { "parent": "body", "child": "imu_apps", "T_child_wrt_parent": [0.1138, -0.006, 0.040], "RPY_parent_to_child": [0, 0, 0] }, { "parent": "body", "child": "ground", "T_child_wrt_parent": [0, 0, 0.0537], "RPY_parent_to_child": [0, 0, 0] }] }and qvio-server config:
{ "odr_hz": 30, "use_camera_height_bootstrap": true, "camera_height_off_ground_m": 0.085000000894069672, "enable_init_while_moving": false, "cam_imu_timeshift_s": 0, "cam_imu_timeshift_s_uncertainty": 9.99999974737875e-05, "T_cam_wrt_imu_uncertainty": [0.00050000002374872565, 0.00050000002374872565, 0.00050000002374872565], "R_cam_to_imu_uncertainty": [0.00499999988824129, 0.00499999988824129, 0.00499999988824129], "accl_fsr_ms2": 156, "gyro_fsr_rad": 34, "accl_noise_std_dev": 0.31600001454353333, "gyro_noise_std_dev": 0.00999999977648258, "cam_noise_std_dev": 100, "min_std_pixel_noise": 0.5, "fail_high_pixel_noise_points": 1.6650999784469604, "limited_imu_bw_trigger": 25, "gps_imu_time_alignment_s": 0, "T_gps_wrt_imu": [0, 0, 0], "enable_mapping": true, "enable_gps_vel": false, "en_auto_reset": true, "auto_reset_max_velocity": 10, "auto_reset_max_v_cov_instant": 0.10000000149011612, "auto_reset_max_v_cov": 0.00999999977648258, "auto_reset_max_v_cov_timeout_s": 0.5, "auto_reset_min_features": 3, "auto_reset_min_feature_timeout_s": 1, "en_standby_mode": true, "standby_skip_frames": 1, "mask_file_path": "" }vio_cams config:
{ "cams": [{ "enable": true, "name": "tracking", "pipe_for_preview": "tracking", "pipe_for_tracking": "tracking_misp_norm", "is_occluded_on_ground": false, "imu": "imu_apps", "cal_file": "opencv_tracking_intrinsics.yml" },{ "enable": false, "name": "tracking_down", "pipe_for_preview": "tracking_down", "pipe_for_tracking": "tracking_down_misp_norm", "is_occluded_on_ground": true, "imu": "imu_apps", "cal_file": "opencv_tracking_down_intrinsics.yml" }] }And heres my qvio overlay:
Would looking into opvenins be a solution, and can it work well with just a single tracking camera ?
-
Hello, I am trying to debug my QVIO flight which is very unstable. When hovering the vehicle does not stay still instead it pitches back and forth aggressively. I can fly in manual mode and altitude mode without any issues at all, but when hovering in altitude mode and switching to position it immediately starts cyclically shaking back and forth and gets worse the longer I let it go.
Its paired with the AR0144 tracking cam and has been calibrated. This is setup to also use a GPS/MAG but indoors use VIO. there is also a VL53L1X mounted for altitude sensing below 4m. Is there something that anyone would recommend checking to see what may be happening here. I followed along with the EKF2 param helpers for VIO in px4 as well.
This is my extrinsics:
"extrinsics": [{ "parent": "imu_apps", "child": "tracking", "T_child_wrt_parent": [0.0237, 0.0024, -0.0073], "RPY_parent_to_child": [0, 90, 90] }, { "parent": "body", "child": "imu_apps", "T_child_wrt_parent": [0.1138, -0.006, 0.040], "RPY_parent_to_child": [0, 0, 0] }, { "parent": "body", "child": "ground", "T_child_wrt_parent": [0, 0, 0.0537], "RPY_parent_to_child": [0, 0, 0] }] }and qvio-server config:
{ "odr_hz": 30, "use_camera_height_bootstrap": true, "camera_height_off_ground_m": 0.085000000894069672, "enable_init_while_moving": false, "cam_imu_timeshift_s": 0, "cam_imu_timeshift_s_uncertainty": 9.99999974737875e-05, "T_cam_wrt_imu_uncertainty": [0.00050000002374872565, 0.00050000002374872565, 0.00050000002374872565], "R_cam_to_imu_uncertainty": [0.00499999988824129, 0.00499999988824129, 0.00499999988824129], "accl_fsr_ms2": 156, "gyro_fsr_rad": 34, "accl_noise_std_dev": 0.31600001454353333, "gyro_noise_std_dev": 0.00999999977648258, "cam_noise_std_dev": 100, "min_std_pixel_noise": 0.5, "fail_high_pixel_noise_points": 1.6650999784469604, "limited_imu_bw_trigger": 25, "gps_imu_time_alignment_s": 0, "T_gps_wrt_imu": [0, 0, 0], "enable_mapping": true, "enable_gps_vel": false, "en_auto_reset": true, "auto_reset_max_velocity": 10, "auto_reset_max_v_cov_instant": 0.10000000149011612, "auto_reset_max_v_cov": 0.00999999977648258, "auto_reset_max_v_cov_timeout_s": 0.5, "auto_reset_min_features": 3, "auto_reset_min_feature_timeout_s": 1, "en_standby_mode": true, "standby_skip_frames": 1, "mask_file_path": "" }vio_cams config:
{ "cams": [{ "enable": true, "name": "tracking", "pipe_for_preview": "tracking", "pipe_for_tracking": "tracking_misp_norm", "is_occluded_on_ground": false, "imu": "imu_apps", "cal_file": "opencv_tracking_intrinsics.yml" },{ "enable": false, "name": "tracking_down", "pipe_for_preview": "tracking_down", "pipe_for_tracking": "tracking_down_misp_norm", "is_occluded_on_ground": true, "imu": "imu_apps", "cal_file": "opencv_tracking_down_intrinsics.yml" }] }And heres my qvio overlay:
Would looking into opvenins be a solution, and can it work well with just a single tracking camera ?
VOXL Version -------------------------------------------------------------------------------- system-image: 1.8.02-M0054-14.1a-perf kernel: #1 SMP PREEMPT Mon Nov 11 22:08:01 UTC 2024 4.19.125 -------------------------------------------------------------------------------- hw platform: M0054 mach.var: 1.0.0 -------------------------------------------------------------------------------- voxl-suite: 1.4.0 -------------------------------------------------------------------------------- Packages: Repo: http://voxl-packages.modalai.com/ ./dists/qrb5165/sdk-1.4/binary-arm64/ Last Updated: 2025-02-11 17:57:14 List: kernel-module-voxl-fsync-mod-4.19.125 1.0-r0 kernel-module-voxl-gpio-mod-4.19.125 1.0-r0 kernel-module-voxl-platform-mod-4.19.125 1.0-r0 libfc-sensor 1.0.7 libmodal-cv 0.5.16 libmodal-exposure 0.1.3 libmodal-journal 0.2.2 libmodal-json 0.4.3 libmodal-pipe 2.10.4 libqrb5165-io 0.4.7 libvoxl-cci-direct 0.2.3 libvoxl-cutils 0.1.1 modalai-slpi 1.1.19 mv-voxl 0.1-r0 qrb5165-bind 0.1-r0 qrb5165-dfs-server 0.2.0 qrb5165-imu-server 1.1.0 qrb5165-rangefinder-server 0.1.4 qrb5165-slpi-test-sig 01-r0 qrb5165-system-tweaks 0.3.2 qrb5165-tflite 2.8.0-2 voxl-bind-spektrum 0.1.1 voxl-camera-calibration 0.5.9 voxl-camera-server 2.0.8 voxl-ceres-solver 2:1.14.0-10 voxl-configurator 0.9.4 voxl-cpu-monitor 0.5.1 voxl-docker-support 1.3.1 voxl-elrs 0.3.4 voxl-esc 1.4.8 voxl-feature-tracker 0.5.2 voxl-flow-server 0.3.6 voxl-fsync-mod 1.0-r0 voxl-gphoto2-server 0.0.10 voxl-gpio-mod 1.0-r0 voxl-io-server 0.0.4 voxl-jpeg-turbo 2.1.3-5 voxl-lepton-server 1.3.3 voxl-lepton-tracker 0.0.4 voxl-libgphoto2 0.0.4 voxl-libuvc 1.0.7 voxl-logger 0.4.9 voxl-mavcam-manager 0.5.7 voxl-mavlink 0.1.1 voxl-mavlink-server 1.4.4 voxl-modem 1.1.3 voxl-mongoose 7.7.0-1 voxl-mpa-to-ros 0.3.9 voxl-mpa-tools 1.3.4 voxl-open-vins 0.4.14 voxl-open-vins-server 0.2.83 voxl-opencv 4.5.5-3 voxl-osd 0.0.2 voxl-platform-mod 1.0-r0 voxl-portal 0.7.3-202501201257 voxl-px4 1.14.0-2.0.99 voxl-px4-drover voxl-px4-drover-v1 voxl-px4-imu-server 0.1.2 voxl-px4-params 0.5.8 voxl-qvio-server 1.1.1 voxl-remote-id 0.0.9 voxl-reset-slpi 0.0.1 voxl-state-estimator 0.0.3 voxl-streamer 0.7.5 voxl-suite 1.4.0 voxl-tag-detector 0.0.4 voxl-tflite-server 0.3.7 voxl-utils 1.4.4 voxl-uvc-server 0.1.7 voxl-vision-hub 1.8.17 voxl-vtx 1.1.3 voxl2-io 0.0.3 voxl2-system-image 1.8.02-r0 voxl2-wlan 1.0-r0 -
VOXL Version -------------------------------------------------------------------------------- system-image: 1.8.02-M0054-14.1a-perf kernel: #1 SMP PREEMPT Mon Nov 11 22:08:01 UTC 2024 4.19.125 -------------------------------------------------------------------------------- hw platform: M0054 mach.var: 1.0.0 -------------------------------------------------------------------------------- voxl-suite: 1.4.0 -------------------------------------------------------------------------------- Packages: Repo: http://voxl-packages.modalai.com/ ./dists/qrb5165/sdk-1.4/binary-arm64/ Last Updated: 2025-02-11 17:57:14 List: kernel-module-voxl-fsync-mod-4.19.125 1.0-r0 kernel-module-voxl-gpio-mod-4.19.125 1.0-r0 kernel-module-voxl-platform-mod-4.19.125 1.0-r0 libfc-sensor 1.0.7 libmodal-cv 0.5.16 libmodal-exposure 0.1.3 libmodal-journal 0.2.2 libmodal-json 0.4.3 libmodal-pipe 2.10.4 libqrb5165-io 0.4.7 libvoxl-cci-direct 0.2.3 libvoxl-cutils 0.1.1 modalai-slpi 1.1.19 mv-voxl 0.1-r0 qrb5165-bind 0.1-r0 qrb5165-dfs-server 0.2.0 qrb5165-imu-server 1.1.0 qrb5165-rangefinder-server 0.1.4 qrb5165-slpi-test-sig 01-r0 qrb5165-system-tweaks 0.3.2 qrb5165-tflite 2.8.0-2 voxl-bind-spektrum 0.1.1 voxl-camera-calibration 0.5.9 voxl-camera-server 2.0.8 voxl-ceres-solver 2:1.14.0-10 voxl-configurator 0.9.4 voxl-cpu-monitor 0.5.1 voxl-docker-support 1.3.1 voxl-elrs 0.3.4 voxl-esc 1.4.8 voxl-feature-tracker 0.5.2 voxl-flow-server 0.3.6 voxl-fsync-mod 1.0-r0 voxl-gphoto2-server 0.0.10 voxl-gpio-mod 1.0-r0 voxl-io-server 0.0.4 voxl-jpeg-turbo 2.1.3-5 voxl-lepton-server 1.3.3 voxl-lepton-tracker 0.0.4 voxl-libgphoto2 0.0.4 voxl-libuvc 1.0.7 voxl-logger 0.4.9 voxl-mavcam-manager 0.5.7 voxl-mavlink 0.1.1 voxl-mavlink-server 1.4.4 voxl-modem 1.1.3 voxl-mongoose 7.7.0-1 voxl-mpa-to-ros 0.3.9 voxl-mpa-tools 1.3.4 voxl-open-vins 0.4.14 voxl-open-vins-server 0.2.83 voxl-opencv 4.5.5-3 voxl-osd 0.0.2 voxl-platform-mod 1.0-r0 voxl-portal 0.7.3-202501201257 voxl-px4 1.14.0-2.0.99 voxl-px4-drover voxl-px4-drover-v1 voxl-px4-imu-server 0.1.2 voxl-px4-params 0.5.8 voxl-qvio-server 1.1.1 voxl-remote-id 0.0.9 voxl-reset-slpi 0.0.1 voxl-state-estimator 0.0.3 voxl-streamer 0.7.5 voxl-suite 1.4.0 voxl-tag-detector 0.0.4 voxl-tflite-server 0.3.7 voxl-utils 1.4.4 voxl-uvc-server 0.1.7 voxl-vision-hub 1.8.17 voxl-vtx 1.1.3 voxl2-io 0.0.3 voxl2-system-image 1.8.02-r0 voxl2-wlan 1.0-r0@Gary-Holmgren Updated this but still having errors
-
@Gary-Holmgren Updated this but still having errors
@Gary-Holmgren Tried re-calibrating the tracking cam and IMU, as well as ensuring everything was rigid. Still no luck.
-
@Gary-Holmgren Tried re-calibrating the tracking cam and IMU, as well as ensuring everything was rigid. Still no luck.
Not sure why these would be outputting different roll values? could it be the cause of the forward and back oscillation that happens when I put it into position mode? The vehicle was completely still and I ran these commands within 10 seconds of each other.
I also see that voxl-inspect-pose and inspect-vio are matching but qvio is not
also, is this something that ha to be set in PX4 relative to IMU_APPS??
-
Not sure why these would be outputting different roll values? could it be the cause of the forward and back oscillation that happens when I put it into position mode? The vehicle was completely still and I ran these commands within 10 seconds of each other.
I also see that voxl-inspect-pose and inspect-vio are matching but qvio is not
also, is this something that ha to be set in PX4 relative to IMU_APPS??
@Gary-Holmgren @Alex-Kushleyev Are you able to check this out I'm not sure what could be causing this. I had qvio working perfectly fine on this stack before.
-
@Gary-Holmgren @Alex-Kushleyev Are you able to check this out I'm not sure what could be causing this. I had qvio working perfectly fine on this stack before.
@Gary-Holmgren calibration results with AR0144
-
@Gary-Holmgren calibration results with AR0144
@Gary-Holmgren , sorry for the delay.
You mentioned that VIO was working fine on this stack - what has changed since it was working fine?
I suggest performing a hand-held test any time you change your VIO configuration. With propellers off, just pick up the drone and move it around after starting QVIO. Even moving around +/- 0.5 meters or so will tell you if there is something clearly wrong with QVIO (which usually happens to be the incorrect extrinsics params).
When you perform the hand-held test, start QVIO process, open QVIO debug console in
voxl-portalthen slowly pick up the vehicle while looking at the features in theqvio-overlay. If it appears that qvio feature are not being tracked (the features appear and disapper), then most likely the extrinsics for the cam<->apps_proc_imu are incorrect. If extrinsics are correct, QVIO should track features very well and it should be evident as the feature markers will stay locked on to the same physical features as you move the vehicle around.Another good test (assuming the first test passes) is to do a slightly longer test where you pick up the vehicle, carry it around the room and put it back into the same spot (position and orientation) where you started. If QVIO is working well, the expected drift should be a few % of the total traveled distance and orientation drift should also be very small.
Alex
-
@Gary-Holmgren , sorry for the delay.
You mentioned that VIO was working fine on this stack - what has changed since it was working fine?
I suggest performing a hand-held test any time you change your VIO configuration. With propellers off, just pick up the drone and move it around after starting QVIO. Even moving around +/- 0.5 meters or so will tell you if there is something clearly wrong with QVIO (which usually happens to be the incorrect extrinsics params).
When you perform the hand-held test, start QVIO process, open QVIO debug console in
voxl-portalthen slowly pick up the vehicle while looking at the features in theqvio-overlay. If it appears that qvio feature are not being tracked (the features appear and disapper), then most likely the extrinsics for the cam<->apps_proc_imu are incorrect. If extrinsics are correct, QVIO should track features very well and it should be evident as the feature markers will stay locked on to the same physical features as you move the vehicle around.Another good test (assuming the first test passes) is to do a slightly longer test where you pick up the vehicle, carry it around the room and put it back into the same spot (position and orientation) where you started. If QVIO is working well, the expected drift should be a few % of the total traveled distance and orientation drift should also be very small.
Alex
@Alex-Kushleyev I have narrowed one of the main causes of instability down to my quads own frame (prop protectors) within the QVIO tacking FOV. Since I am using the AR0144 cam I modified the mask file in GIMP to include only 0 and 255 for the same aspect ratio of 1280x800 at full and 1.4 res to try to apply the mask in the qvio config file but no matter what I try it seems the mask is not affecting QVIO at all. I tired a fully blacked out mask as a test and it didn't change anything. I also ensured the header matching the format found in this post.
Have you gotten a mask example to work with the AR0144?
-
@Alex-Kushleyev I have narrowed one of the main causes of instability down to my quads own frame (prop protectors) within the QVIO tacking FOV. Since I am using the AR0144 cam I modified the mask file in GIMP to include only 0 and 255 for the same aspect ratio of 1280x800 at full and 1.4 res to try to apply the mask in the qvio config file but no matter what I try it seems the mask is not affecting QVIO at all. I tired a fully blacked out mask as a test and it didn't change anything. I also ensured the header matching the format found in this post.
Have you gotten a mask example to work with the AR0144?
@Gary-Holmgren , we have not tried the mask with 1280x800 resolution.
Can you please try using the mask of size 160x120 and make it half white / half black for sanity testing. It is possible that the mask size has to be of these specific dimensions (and it would get resized to match the aspect ratio of the actual image, not sure).
Also, you should enable the debug prints in QVIO, as mentioned here https://forum.modalai.com/topic/3459/masking-qvio/6 and there is another potential "gotcha", the mask path is relative to working directory where qvio server starts.. but after you enable the qvio server debug prints, you will see the path from which the mask file is being loaded, as you can see from screen shots from the other post.
Unfortunately, the underlying QVIO source code is not available, so I cant check the exact behavior of the masking feature.
Alex
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login