@Alex-Kushleyev We have been advised that the ov sensor modules for dfs are approaching end of life. Which camera options do you recommend we consider instead? Thank you for your help!
S
Latest posts made by SKA
-
Stereo Image sensor selectionposted in Video and Image Sensors
-
RE: OV with External Autopilotposted in GPS-denied Navigation (VIO)
@tom Thanks! I referred to this page in order to set up what I have right now.
My voxl 2 mini is connected to the external autopilot using the one exposed UART. It seems to be connected fine, since I'm receiving the voxl heartbeat on the autopilot end, and I'm receiving autopilot mavlink messages on the voxl side.I wonder if there is anything I might be missing on the voxl-vision-hub side, or elsewhere.
Thanks for your help! -
OV with External Autopilotposted in GPS-denied Navigation (VIO)
Hi! I'm trying to use
voxl-vision-hubto send voxl-open-vins position data to an external autopilot (ardupilot). I start by making sure the following services are running:- voxl-imu-server
- voxl-open-vins-server
- voxl-camera-server
- voxl-vision-hub
- voxl-mavlink-server
I can confirm that the voxl and external autopilot are connected because I see this on Mission Planner MAVLink Inspector:
and the debug output I see after running
voxl-vision-hub -dis:voxl2-mini:~$ voxl-vision-hub -d Enabling recv debugging for localhost UDP port from ROS loading our own config file ================================================================= Parameters as loaded from config file: config_file_version: 1 MAVROS / MAVSDK en_localhost_mavlink_udp 1 localhost_udp_port_number: 14551 VIO en_vio: 1 vio_pipe: ov secondary_vio_pipe: qvio en_reset_vio_if_initialized_inverted: 0 vio_warmup_s: 0.000000 send_odom_while_failed: 1 MISC FEATURES horizon_cal_tolerance: 0.500000 en_hitl: 0 OFFBOARD MODE offboard_mode: off follow_tag_id: 0 figure_eight_move_home: 1 wps_move_home: 1 wps_timeout: 0.000000 wps_damp: 1.000000 robot_radius: 0.300000 collision_sampling_dt: 0.100000 max_lookahead_distance: 1.000000 backtrack_seconds : 60 backtrack_rc_chan : 8 backtrack_rc_thresh : 1500 FIXED FRAME RELOCALIZATION en_tag_fixed_frame: 0 fixed_frame_filter_len: 5 en_transform_mavlink_pos_setpoints_from_fixed_frame:0 VOXL FLIGHT CONTROLLER (VFC) vfc_rate 100.000000 vfc_rc_chan_min 980 vfc_rc_chan_max 2020 vfc_thrust_ch 3 vfc_roll_ch 1 vfc_pitch_ch 2 vfc_yaw_ch 4 vfc_submode_ch 6 vfc_alt_mode_rc_min 0 vfc_alt_mode_rc_max 0 vfc_flow_mode_rc_min 0 vfc_flow_mode_rc_max 0 vfc_hybrid_flow_mode_rc_min 0 vfc_hybrid_flow_mode_rc_max 0 vfc_position_mode_rc_min 0 vfc_position_mode_rc_max 2100 vfc_traj_mode_rc_min 0 vfc_traj_mode_rc_max 0 vfc_yaw_deadband 30 vfc_vxy_deadband 50 vfc_vz_deadband 150 vfc_min_thrust 0.000000 vfc_max_thrust 0.800000 vfc_tilt_max 0.436000 vfc_yaw_rate_max 3.000000 vfc_thrust_hover 0.500000 vfc_vz_max 1.000000 vfc_kp_z 5.290000 vfc_kd_z 5.980000 vfc_vxy_max 3.000000 vfc_kp_xy 0.640000 vfc_kd_xy 2.560000 vfc_kp_z_vio 5.290000 vfc_kd_z_vio 5.980000 vfc_kp_xy_vio 3.240000 vfc_kd_xy_vio 3.960000 vfc_w_filt_xy_vio 10.000000 vfc_vel_ff_factor_vio 0.900000 vfc_xy_acc_limit_vio 2.500000 vfc_max_z_delta 3.000000 vfc_att_transition_time 0.500000 vfc_stick_move_threshold 30.000000 vfc_flow_transition_time 1.000000 vfc_q_min 10 vfc_points_min 7 vfc_en_submode_announcement 1 vfc_disable_fallback 0 COLLISION PREVENTION (VOA) en_voa: 0 voa_upper_bound_m: -0.150000 voa_lower_bound_m: 0.150000 voa_memory_s: 1.000000 voa_max_pc_per_fusion: 100 voa_pie_min_dist_m: 0.250000 voa_pie_max_dist_m: 20.000000 voa_pie_under_trim_m: 1.000000 voa_pie_threshold: 3 voa_pie_slices: 36 voa_pie_bin_depth_m: 0.150000 voa_send_rate_hz: 20.000000 voa_input #0 enabled: 0 type: point_cloud input_pipe: dfs_point_cloud frame: stereo_l max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 voa_input #1 enabled: 0 type: point_cloud input_pipe: stereo_front_pc frame: stereo_front_l max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 voa_input #2 enabled: 0 type: point_cloud input_pipe: stereo_rear_pc frame: stereo_rear_l max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 voa_input #3 enabled: 0 type: tof input_pipe: tof frame: tof max_depth: 6.000000 min_depth: 0.150000 cell_size: 0.080000 threshold: 3 x_fov_deg: 106.500000 y_fov_deg: 85.099998 conf_cutoff: 125 voa_input #4 enabled: 0 type: rangefinder input_pipe: rangefinders frame: body max_depth: 8.000000 min_depth: 0.300000 cell_size: 0.080000 threshold: 4 x_fov_deg: 68.000000 y_fov_deg: 56.000000 conf_cutoff: 0 ================================================================= loading extrinsics config file loading horizon cal file starting geometry module starting autopilot monitor starting mavlink IO Waiting to connect to voxl-mavlink-server Connected to voxl-mavlink-server starting mavlink for ros starting fixed pose input starting vio manager Connected to VIO pipe: ov Geometry module updating to use imu: imu_apps for VIO new T imu wrt body: 0.0407 -0.0060 -0.0188 new R imu to body: 1.0000 -0.0000 0.0000 0.0000 1.0000 -0.0000 0.0000 0.0000 1.0000 done updating transforms to use imu: imu_apps starting tag manager starting horizon cal module starting imu manager starting state manager Init complete from AP msgid: 0 sysid: 1 compid: 1 Detected Autopilot Mavlink SYSID 1 requesting autopilot_version from AP msgid: 0 sysid: 1 compid: 1 requesting autopilot_version from AP msgid: 77 sysid: 1 compid: 1 from AP msgid: 77 sysid: 1 compid: 1 from AP msgid: 148 sysid: 1 compid: 1 Detected autopilot version: 4.5.5 from AP msgid: 0 sysid: 1 compid: 1 from AP msgid: 0 sysid: 1 compid: 1 from AP msgid: 22 sysid: 1 compid: 1It seems like other than the heartbeat, no data is being sent to the external autopilot. Thanks in advance for any guidance that can help me sort this out!
-
RE: OpenVINS Reinitializing Issueposted in GPS-denied Navigation (VIO)
Looks like modalcv feature tracking is only set up for the ar0144 camera and not the ov7251 camera. Could you please extend feature tracking capability to other cameras? Is there any way I can do this on my end?
-
RE: OpenVINS Reinitializing Issueposted in GPS-denied Navigation (VIO)
@Cliff-Wong Hi, thanks for your response! Here is the ouput of
voxl-version:voxl2:~$ voxl-version -------------------------------------------------------------------------------- system-image: 1.8.02-M0054-14.1a-perf kernel: #1 SMP PREEMPT Mon Nov 11 22:08:01 UTC 2024 4.19.125 -------------------------------------------------------------------------------- hw platform: M0054 mach.var: 1.0.0 -------------------------------------------------------------------------------- voxl-suite: 1.4.1-202502092201 -------------------------------------------------------------------------------- Packages: Repo: http://voxl-packages.modalai.com/ ./dists/qrb5165/sdk-1.4/binary-arm64/ Last Updated: 2025-02-10 20:13:48 List: kernel-module-voxl-fsync-mod-4.19.125 1.0-r0 kernel-module-voxl-gpio-mod-4.19.125 1.0-r0 kernel-module-voxl-platform-mod-4.19.125 1.0-r0 libfc-sensor 1.0.7 libmodal-cv 0.5.16 libmodal-exposure 0.1.3 libmodal-journal 0.2.2 libmodal-json 0.4.4 libmodal-pipe 2.10.6 libqrb5165-io 0.4.9 libvoxl-cci-direct 0.2.5 libvoxl-cutils 0.1.1 modalai-slpi 1.1.19 mv-voxl 0.1-r0 qrb5165-bind 0.1-r0 qrb5165-dfs-server 0.2.0 qrb5165-imu-server 1.1.2 qrb5165-rangefinder-server 0.1.4 qrb5165-slpi-test-sig 01-r0 qrb5165-system-tweaks 0.3.4 qrb5165-tflite 2.8.0-2 voxl-bind-spektrum 0.1.1 voxl-camera-calibration 0.5.9 voxl-camera-server 2.1.0 voxl-ceres-solver 2:1.14.0-10 voxl-configurator 0.9.7 voxl-cpu-monitor 0.5.3 voxl-cross-template 0.0.1 voxl-docker-support 1.3.1 voxl-eigen3 3.4.0 voxl-elrs 0.4.1 voxl-esc 1.5.1 voxl-feature-tracker 0.5.2 voxl-flow-server 0.3.6 voxl-fsync-mod 1.0-r0 voxl-gphoto2-server 0.0.10 voxl-gpio-mod 1.0-r0 voxl-io-server 0.0.4 voxl-jpeg-turbo 2.1.3-6 voxl-lepton-server 1.3.3 voxl-lepton-tracker 0.0.4 voxl-libgphoto2 0.0.4 voxl-libuvc 1.0.7 voxl-logger 0.4.9 voxl-mavcam-manager 0.5.7 voxl-mavlink 0.1.2 voxl-mavlink-server 1.4.4 voxl-modem 1.1.5 voxl-mongoose 7.7.0-1 voxl-mpa-to-ros 0.3.9 voxl-mpa-tools 1.3.7 voxl-open-vins 0.4.16 voxl-open-vins-server 0.3.0 voxl-opencv 4.5.5-2 voxl-osd 0.0.5 voxl-platform-mod 1.0-r0 voxl-portal 0.7.5 voxl-px4 1.14.0-2.0.94 voxl-px4-imu-server 0.1.2 voxl-px4-params 0.6.2 voxl-qvio-server 1.1.1 voxl-remote-id 0.0.9 voxl-reset-slpi 0.0.1 voxl-state-estimator 0.0.4 voxl-streamer 0.7.5 voxl-suite 1.4.1-202502092201 voxl-tag-detector 0.0.4 voxl-tflite-server 0.3.9 voxl-utils 1.4.4 voxl-uvc-server 0.1.7 voxl-vision-hub 1.8.17 voxl-vtx 1.1.6 voxl2-io 0.0.3 voxl2-system-image 1.8.02-r0 voxl2-wlan 1.0-r0 --------------------------------------------------------------------------------I've upgraded to the latest SDK. I'm not using an aircraft, I'm working with the voxl 2 board only, with a tracking fisheye ov7251 camera. The intrinsics and extrinsics are set up correctly, I was able to confirm this because qvio works well.
I've traced back the error message
ERROR: Width/Height mismatch, optic flow w/h should be [640 400] , instead got [320 240]
tovoxl-feature-tracker/src/main.cppline 650:
cvp_tracker_vec[ch].init(ch, width, height);This leads back to
mcv_cvp_feature_tracker_initfromlibmodal_cvwhich is closed source.How can I run the ovins internal-mode tracker? Let me know if you have any further ideas to help me get ovins working Thanks!
-
RE: OpenVINS Reinitializing Issueposted in GPS-denied Navigation (VIO)
@Cliff-Wong Thanks for your response! I have
en_ext_feature_tracker: true. I tried runningvoxl-feature-trackerand it ended in a seg fault (included below).I also noticed that after running
voxl-open-vins-server, when I runvoxl-inspect-services,voxl-open-vins-serverremains Disabled and Not Running even though the ov_overlay video stream is visible and updating, and thevoxl-open-vins-serverhas not been quit.voxl2:~$ voxl-feature-tracker ================================================================= num_features: 30 en_blur: 0 en_normalization: 0 en_stereo_sync_ts: 0 pixel_filter_size: 2.000 pixel_search_radius: 10.000 octave_search_radius: 3.000 ================================================================= vio_cam config: cam 0: enable: 1 name: tracking pipe_for_preview: tracking pipe_for_tracking: tracking is_occluded_on_ground: 0 imu: imu_apps is_extrinsic_present: 1 extrinsic cam wrt imu: parent: imu_apps child: tracking T_cam_wrt_imu: -0.017 -0.016 0.015 RPY_imu_to_cam: 0.0 0.0 0.0 R_cam_to_imu: 1.000 -0.000 0.000 0.000 1.000 -0.000 0.000 0.000 1.000 cal file: opencv_tracking_intrinsics.yml is_cal_present: 1 cam cal intrinsics: width: 640 height: 480 fx: 277.865 fy: 278.515 cx: 310.027 cy: 230.207 n_coeffs: 4 is_fisheye: 1 D: -0.0091 0.0038 0.0082 -0.0059 ================================================================= thread is locked to cores: 0 1 2 3 cvp_set_pixel_search_radius: 10.000000 cvp_octave_search_radius: 3.000000 cvp_en_id_sort: true opening pipe: tracking Setup completed! Waiting for images... Connected to cam 0 tracking ERROR: Width/Height mismatch, optic flow w/h should be [640 400] , instead got [320 240] ERROR getting cvp feature tracker handle ========================================= created output pipes: Feature Pipe: tracked_feats Overlay Pipe: feature_tracker_overlay ========================================= starting publisher thread Segmentation fault: Fault thread: voxl-feature-tr(tid: 4606) Fault address: (nil) Address not mapped. Segmentation faultThanks for your help, let me know what you think!
-
OpenVINS Reinitializing Issueposted in GPS-denied Navigation (VIO)
OpenVINS starts up as expected, then starts reinitializing repeatedly. The intrinsics and extrinsics are set up and working as expected with qvio.
Running voxl-open-vins-server -d, I get the following:
====> Using EXTERNAL feature tracker <==== >>>>>>>>>> Current reference counts: trackDB: 1 trackFEATS: 1 propagator: 2 updaterMSCKF: 1 updaterSLAM: 1 active_tracks_initializer: 1 initializer: 1 thread is locked to cores: 4 5 6 7 create_server_pipes connecting client pipes imu pipe name: imu_apps Number of Cameras active: 1 Camera merge --- > ch: 1 to cam id: 0 Opening camera pipe: tracking tmp_camera_pipe_names.push_back(): tracking VFT connected client "voxl-portal38440978" connected to overlay [INFO] world : 1.000000 1.000000 (2) FAILED TO FETCH IMG RINGBUF at time 0 -2 -2 -2 -2 -2 [THERMAL] Resume processing done FAILED TO FETCH IMG RINGBUF at time 0 -2 -2 -2 -2 -2 [ERROR] In init too long, timeout, retry RESET 5012286699 [ERROR] Triggered RESET type: 4 [INFO] restarting managers STATELESS Set camera intrinsics and extrinsicsHow can I get past the reinitializing issue and run OpenVINS?