# Vision Hub Issue Source: https://forum.modalai.com/topic/3590/vision-hub-issue Category: General Questions (https://forum.modalai.com/category/2/general-questions) Posted: 2024-06-28 07:43:43 UTC by Kris Replies: 5 · Views: 1816 ## Kris · 2024-06-28 07:43:43 UTC Hello, I am having a problem with my VOA and VIO unstable flights. I am on the Seeker Voxl1 with a fresh stable release of the voxl_SDK_1.2.0. This is the error message when I check my voxl-vision-hub; ``` voxl:~$ voxl-vision-hub loading our own config file ================================================================= Parameters as loaded from config file: config_file_version: 1 MAVROS / MAVSDK en_localhost_mavlink_udp 1 localhost_udp_port_number: 14551 VIO en_vio: 1 vio_pipe: qvio secondary_vio_pipe: ov en_reset_vio_if_initialized_inverted: 1 vio_warmup_s: 3.000000 send_odom_while_failed: 1 APQ8096-ONLY FEATURES en_set_clock_from_gps: 1 en_force_onboard_mav1_mode: 1 en_reset_px4_on_error: 1 MISC FEATURES horizon_cal_tolerance: 0.500000 OFFBOARD MODE offboard_mode: trajectory follow_tag_id: 0 figure_eight_move_home: 1 robot_radius: 0.300000 collision_sampling_dt: 0.100000 max_lookahead_distance: 1.000000 FIXED FRAME RELOCALIZATION en_tag_fixed_frame: 0 fixed_frame_filter_len: 5 en_transform_mavlink_pos_setpoints_from_fixed_frame:0 COLLISION PREVENTION (VOA) en_voa: 1 voa_upper_bound_m: -0.150000 voa_lower_bound_m: 0.150000 voa_memory_s: 1.000000 voa_max_pc_per_fusion: 100 voa_pie_min_dist_m: 0.250000 voa_pie_max_dist_m: 20.000000 voa_pie_under_trim_m: 1.000000 voa_pie_threshold: 3 voa_pie_slices: 36 voa_pie_bin_depth_m: 0.150000 voa_send_rate_hz: 20.000000 voa_input #0 enabled: 1 type: point_cloud input_pipe: dfs_point_cloud frame: stereo_l max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 voa_input #1 enabled: 1 type: point_cloud input_pipe: stereo_front_pc frame: stereo_front_l max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 voa_input #2 enabled: 1 type: point_cloud input_pipe: stereo_rear_pc frame: stereo_rear_l max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 voa_input #3 enabled: 1 type: tof input_pipe: tof frame: tof max_depth: 6.000000 min_depth: 0.150000 cell_size: 0.080000 threshold: 3 x_fov_deg: 106.500000 y_fov_deg: 85.099998 conf_cutoff: 125 voa_input #4 enabled: 1 type: rangefinder input_pipe: rangefinders frame: body max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 ================================================================= loading extrinsics config file loading horizon cal file existing instance of voxl-vision-hub found, attempting to stop it starting geometry module starting autopilot monitor starting mavlink IO Waiting to connect to voxl-mavlink-server Connected to voxl-mavlink-server starting mavlink for ros starting fixed pose input starting vio manager Connected to VIO pipe: qvio Geometry module updating to use imu: imu1 for VIO new T imu wrt body: 0.0465 0.0332 -0.0130 new R imu to body: 0.0000 -0.0000 1.0000 1.0000 0.0000 -0.0000 -0.0000 1.0000 0.0000 done updating transforms to use imu: imu1 Detected Autopilot Mavlink SYSID 1 starting tag manager starting voa manager Connected to VOA input pipe: dfs_point_cloud Connected to VOA input pipe: tof starting horizon cal module starting offboard trajectory Trajectory Monitor connected to voa pointcloud Init complete ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server WARNING in VOA manager, no attitude data from px4 requesting autopilot_version Detected autopilot version: 1.11.3 ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ^C received SIGINT Ctrl-C Starting shutdown sequence stopping offboard trajectory exiting offboard trajectory thread stopping horizon_cal stopping voa manager exiting VOA thread stopping tag manager stopping vio manager stopping fixed pose in module Stopping mavlink for ros module exiting mavlink for ros localhost udp listener thread udp mavlink for ros stopped Stopping mavlink io module waiting for mavlink timer thread to join Stopping autopilot monitor closing remaining client pipes closing remaining server pipes Removing PID file exiting ``` Here is what is currently running; ``` voxl:~$ voxl-inspect-services Scanning services... Service Name | Enabled | Running | CPU Usage ------------------------------------------------------------------- docker-autorun | Disabled | Not Running | docker-daemon | Disabled | Not Running | modallink-relink | Disabled | Not Running | voxl-camera-server | Enabled | Running | 6.5% voxl-cpu-monitor | Enabled | Running | 0.0% voxl-dfs-server | Enabled | Running | 13.0% voxl-imu-server | Enabled | Running | 0.0% voxl-lepton-server | Disabled | Not Running | voxl-mavcam-manager | Enabled | Running | 0.0% voxl-mavlink-server | Enabled | Running | 4.3% voxl-modem | Disabled | Not Running | voxl-portal | Enabled | Running | 0.0% voxl-qvio-server | Enabled | Running | 6.5% voxl-rangefinder-server | Disabled | Not Running | voxl-remote-id | Disabled | Not Running | voxl-streamer | Disabled | Not Running | voxl-tag-detector | Disabled | Not Running | voxl-tflite-server | Disabled | Not Running | voxl-time-sync | Disabled | Not Running | voxl-uvc-server | Disabled | Not Running | voxl-vision-hub | Enabled | Running | 2.1% voxl-wait-for-fs | Enabled | Completed | ``` ``` voxl:~$ voxl-vision-hub -s loading our own config file ================================================================= Parameters as loaded from config file: config_file_version: 1 MAVROS / MAVSDK en_localhost_mavlink_udp 1 localhost_udp_port_number: 14551 VIO en_vio: 1 vio_pipe: qvio secondary_vio_pipe: ov en_reset_vio_if_initialized_inverted: 1 vio_warmup_s: 3.000000 send_odom_while_failed: 1 APQ8096-ONLY FEATURES en_set_clock_from_gps: 1 en_force_onboard_mav1_mode: 1 en_reset_px4_on_error: 1 MISC FEATURES horizon_cal_tolerance: 0.500000 OFFBOARD MODE offboard_mode: trajectory follow_tag_id: 0 figure_eight_move_home: 1 robot_radius: 0.300000 collision_sampling_dt: 0.100000 max_lookahead_distance: 1.000000 FIXED FRAME RELOCALIZATION en_tag_fixed_frame: 0 fixed_frame_filter_len: 5 en_transform_mavlink_pos_setpoints_from_fixed_frame:0 COLLISION PREVENTION (VOA) en_voa: 1 voa_upper_bound_m: -0.150000 voa_lower_bound_m: 0.150000 voa_memory_s: 1.000000 voa_max_pc_per_fusion: 100 voa_pie_min_dist_m: 0.250000 voa_pie_max_dist_m: 20.000000 voa_pie_under_trim_m: 1.000000 voa_pie_threshold: 3 voa_pie_slices: 36 voa_pie_bin_depth_m: 0.150000 voa_send_rate_hz: 20.000000 voa_input #0 enabled: 1 type: point_cloud input_pipe: dfs_point_cloud frame: stereo_l max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 voa_input #1 enabled: 1 type: point_cloud input_pipe: stereo_front_pc frame: stereo_front_l max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 voa_input #2 enabled: 1 type: point_cloud input_pipe: stereo_rear_pc frame: stereo_rear_l max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 voa_input #3 enabled: 1 type: tof input_pipe: tof frame: tof max_depth: 6.000000 min_depth: 0.150000 cell_size: 0.080000 threshold: 3 x_fov_deg: 106.500000 y_fov_deg: 85.099998 conf_cutoff: 125 voa_input #4 enabled: 1 type: rangefinder input_pipe: rangefinders frame: body max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 ================================================================= loading extrinsics config file loading horizon cal file existing instance of voxl-vision-hub found, attempting to stop it starting geometry module starting autopilot monitor starting mavlink IO Waiting to connect to voxl-mavlink-server Connected to voxl-mavlink-server starting mavlink for ros starting fixed pose input starting vio manager Detected Autopilot Mavlink SYSID 1 Connected to VIO pipe: qvio Geometry module updating to use imu: imu1 for VIO new T imu wrt body: 0.0465 0.0332 -0.0130 new R imu to body: 0.0000 -0.0000 1.0000 1.0000 0.0000 -0.0000 -0.0000 1.0000 0.0000 done updating transforms to use imu: imu1 starting tag manager starting voa manager Connected to VOA input pipe: dfs_point_cloud Connected to VOA input pipe: tof starting horizon cal module starting offboard trajectory Init complete Trajectory Monitor connected to voa pointcloud slice 0 dist none back slice 1 dist none slice 2 dist none slice 3 dist none slice 4 dist none slice 5 dist none slice 6 dist none slice 7 dist none slice 8 dist none slice 9 dist none left slice 10 dist none slice 11 dist none slice 12 dist none slice 13 dist none slice 14 dist none slice 15 dist none slice 16 dist none slice 17 dist none slice 18 dist none forward slice 19 dist none slice 20 dist none slice 21 dist none slice 22 dist none slice 23 dist none slice 24 dist none slice 25 dist none slice 26 dist none slice 27 dist none right slice 28 dist none slice 29 dist none slice 30 dist none slice 31 dist none slice 32 dist none slice 33 dist none slice 34 dist none slice 35 dist none WARNING in VOA manager, no attitude data from px4 slice 0 dist none back slice 1 dist none slice 2 dist none slice 3 dist none slice 4 dist none slice 5 dist none slice 6 dist none slice 7 dist none slice 8 dist none slice 9 dist none left slice 10 dist none slice 11 dist none slice 12 dist none slice 13 dist none slice 14 dist none slice 15 dist none slice 16 dist none slice 17 dist none slice 18 dist none forward slice 19 dist none slice 20 dist 1.91m slice 21 dist none slice 22 dist none slice 23 dist none slice 24 dist none slice 25 dist none slice 26 dist none slice 27 dist none right slice 28 dist none slice 29 dist none slice 30 dist none slice 31 dist none slice 32 dist none slice 33 dist none slice 34 dist none slice 35 dist none slice 0 dist none back slice 1 dist none slice 2 dist none slice 3 dist none slice 4 dist none slice 5 dist none slice 6 dist none slice 7 dist none slice 8 dist none slice 9 dist none left slice 10 dist none slice 11 dist none slice 12 dist none slice 13 dist none slice 14 dist none slice 15 dist none slice 16 dist none slice 17 dist none slice 18 dist none forward slice 19 dist none slice 20 dist 1.91m slice 21 dist none slice 22 dist none slice 23 dist none slice 24 dist none slice 25 dist none slice 26 dist none slice 27 dist none right slice 28 dist none slice 29 dist none slice 30 dist none slice 31 dist none slice 32 dist none slice 33 dist none slice 34 dist none slice 35 dist none ERROR validating TOF2 data received through pipe: read partial packet read 693516 bytes, but it should be a multiple of 693516 make sure you are using a recent voxl-camera-server that publishes a tof2_data_t type it's likely you are using an old voxl-camera-server ``` ``` voxl:~$ voxl-version -------------------------------------------------------------------------------- system-image: 4.0.0 kernel: #1 SMP PREEMPT Thu Oct 13 17:57:05 UTC 2022 3.18.71-perf -------------------------------------------------------------------------------- hw platform: VOXL mach.var: 1.0 -------------------------------------------------------------------------------- voxl-suite: 1.2.0 -------------------------------------------------------------------------------- Packages: Repo: http://voxl-packages.modalai.com/dists/apq8096/sdk-1.2/binary-arm64/ Last Updated: 2024-06-28 07:54:15 List: apq8096-dfs-server 0.3.1 apq8096-imu-server 1.1.0 apq8096-libpng 1.6.38-1 apq8096-rangefinder-server 0.1.3 apq8096-system-tweaks 0.2.3 apq8096-tflite 2.8.3-1 libapq8096-io 0.6.1 libmodal-cv 0.4.0 libmodal-exposure 0.1.0 libmodal-journal 0.2.2 libmodal-json 0.4.3 libmodal-pipe 2.10.0 libvoxl-cci-direct 0.2.1 libvoxl-cutils 0.1.1 voxl-camera-calibration 0.5.4 voxl-camera-server 1.9.2 voxl-configurator 0.5.6 voxl-cpu-monitor 0.4.8 voxl-docker-support 1.3.1 voxl-gphoto2-server 0.0.10 voxl-jpeg-turbo 2.1.3-5 voxl-lepton-server 1.2.0 voxl-libgphoto2 0.0.4 voxl-libuvc 1.0.7 voxl-logger 0.4.0 voxl-mavcam-manager 0.5.3 voxl-mavlink 0.1.1 voxl-mavlink-server 1.4.0 voxl-modem 1.0.9 voxl-mongoose 7.7.0-1 voxl-mpa-to-ros 0.3.7 voxl-mpa-tools 1.1.5 voxl-opencv 4.5.5-2 voxl-portal 0.6.5 voxl-qvio-server 1.0.2 voxl-remote-id 0.0.9 voxl-streamer 0.7.4 voxl-suite 1.2.0 voxl-system-image 4.0-r0 voxl-tag-detector 0.0.4 voxl-tflite-server 0.3.2 voxl-utils 1.3.9 voxl-uvc-server 0.1.6 voxl-vision-hub 1.7.4 voxl-vpn 0.0.6 -------------------------------------------------------------------------------- ``` ``` voxl:/$ voxl-inspect-qvio waiting for server dt(ms) | T_imu_wrt_vio (m) |Roll Pitch Yaw (deg)|features|quality| state|| 30.0 | 0.00 0.00 0.00| -0.0 -0.1 0.1| 23 | 100% | OKAY |C received SIGINT Ctrl-C closing and exiting ``` voxl-inspect-cam qvio ``` | Pipe Name | bytes | wide | hgt |exp(ms)| gain | frame id |latency(ms)| fpt ERROR: invalid metadata, magic number=88429860, expected 1448040524 most likely client fell behind and pipe overflowed ``` voxl-inspect-cam stereo ``` | Pipe Name | bytes | wide | hgt |exp(ms)| gain | frame id |latency(ms)| fpt | stereo | 614400 | 640 | 480 | 5.07 | 125 | 8787 | 28.0 | 30 ``` voxl-inspect-cam tracking ``` | Pipe Name | bytes | wide | hgt |exp(ms)| gain | frame id |latency(ms)| t | tracking | 307200 | 640 | 480 | 5.01 | 100 | 10838 | 12.3 | 3C ``` voxl-inspect-cam tof ``` | Pipe Name | bytes | wide | hgt |exp(ms)| gain | frame id |latency(ms)| t | tof | 0 | 0 | 0 | 0.00 | 0 | 0 | 25.4 | 8 ERROR: invalid metadata, magic number=0, expected 1448040524 most likely client fell behind and pipe overflowed ``` Everything in Voxl Portal seems to be working as normal though. ## Reply by Kris · 2024-07-01 09:10:17 UTC I have also reverted the SDK to voxl_platform_0.9.5 and everything was working fine. It does not seem like a hardware issue. So I am wondering why the two newer SDK's are giving me issues with VIO and VOA. ## Reply by Moderator (ModalAI staff) · 2024-07-01 14:54:59 UTC (in reply to Kris) @Kris we would need a lot more information to provide an educated guess. Have you gone through the Troubleshooting VIO document? https://docs.modalai.com/flying-with-vio/#troubleshooting-vio Have you looked at the qvio_overlay? "unstable flight" could mean so many things. We recommend you break it down into smaller problems. 1. how well does it fly in manual? Do the PX4 logs look good? 2. how well does it fly with VIO? 3. Does qvio_overlay indicate any issues? 4. enable VOA, do things change? 5. do the VOA diagnostic tools show anything? ## Reply by Kris · 2024-07-01 23:50:08 UTC (in reply to Moderator) @Moderator Thank you for your reply, Yes, I have gone through the troubleshooting guides for VIO. Yes, I have looked at the qvio_overlay. It was actually one of the first things I did. Everything looked normal but then suddenly the point percentage would drop from 100% to 0% when the seeker was on the bench. 1. The seeker performs as usualy in manual flight modes, otherthan the usual low stick throttle when trying to ascend. 2. It will fly ok in VIO, the main problem is with VOA. I cannot even disable the VOA. After disabling VOA in voxl-vision-hub.conf, nothing changes in PX4. QGC says that VOA is still on and the Seeker does not move in any direction. When VOA is enabled, Obstacle distance remains at 2001,2001,2001,2001.... and does not change when I place my hand in front of the sensors on the Seeker. Then I will try to fly the Seeker with VOA enabled at 0.6 to 1 meter stopping distance, VOA does not work. 3. qvio_overlay does not indicate any issues. 4. Enabling and disabling VOA changes things but not in a good way as I mentioned above in #2 response. 5. I posted the diagnostics of all these issues above in my first post. This only happens with voxl_SDK_1.1.0, 1.1.1, 1.1.2, and 1.2.0 releases. There are no problems with the voxl_platform_0.9.5 release. ## Reply by Kris · 2024-07-03 01:57:18 UTC (in reply to Kris) ``` voxl:/$ voxl-list-pipes -t camera_image_metadata_t dfs_disparity dfs_disparity_scaled qvio_overlay stereo tof_conf tof_depth tof_ir tracking cpu_stats_t cpu_monitor ext_vio_data_t qvio_extended imu_data_t imu0 imu1 mavlink_message_t mavlink_ap_heartbeat mavlink_attitude mavlink_from_gcs mavlink_gps_raw_int mavlink_local_position_ned mavlink_onboard mavlink_sys_status mavlink_to_gcs point_cloud_metadata_t dfs_point_cloud tof_pc voa_pc_out pose_vel_6dof_t vvhub_body_wrt_fixed vvhub_body_wrt_local string gcs_ip_list text horizon_cal_io tof_data_t tof vio_data_t qvio ``` ## Reply by James Strawson (ModalAI staff) · 2024-07-13 00:46:36 UTC (in reply to Kris) @Kris I found and fixed a bug in voxl-camera-server that seems to be the culprit. I flashed my seeker with SDK1.2.0 and sideloaded the following voxl-camera-server with the bugfix to re-enable ingestion of TOF data into voxl-vision-hub http://voxl-packages.modalai.com/dists/apq8096/dev/binary-arm64/voxl-camera-server_1.9.7_202407111658.ipk Note VOA simply does not work in the default PX4 acceleration-based position mode so MPC_POS_MODE needs to be set to 0 (velocity control) as done by the voa_enable px4 param helper file here: https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-px4-params/-/blob/dev/params/v1.14/voa_helpers/voa_enable_indoor.params?ref_type=heads