qvio_overlay "ERR CAM_MISSING CAM_BAD_RES STALLED" on Starling 2
-
Hello,
I recently received a Startling 2 drone and have been trying to get the Mapper functionality to work.I tried SDK 1.3.3 and then tried 1.3.4-202409192201.
TOF seems to be working fine, as is the tracking_front camera however QVIO is not. See the attached screenshot and logs.
Any advice on how to resolve this would be appreciated.
voxl2:/$ voxl-inspect-qvio waiting for server dt(ms) | T_imu_wrt_vio (m) |Roll Pitch Yaw (deg)|features|quality| state| error_codes | 31.1 | 0.00 0.00 0.00| 0.0 -0.0 0.0| 0 | -1% | FAIL | CAM_MISSING CAM_BAD_RES STALLED ^C received SIGINT Ctrl-C closing and exiting voxl2:/$ voxl-inspect-services Scanning services... Service Name | Enabled | Running | CPU Usage ------------------------------------------------------------------- docker-autorun | Disabled | Not Running | modallink-relink | Disabled | Not Running | voxl-camera-server | Enabled | Running | 93.9% voxl-cpu-monitor | Enabled | Running | 0.3% voxl-dfs-server | Disabled | Not Running | voxl-feature-tracker | Disabled | Not Running | voxl-flow-server | Disabled | Not Running | voxl-imu-server | Enabled | Running | 3.6% voxl-lepton-server | Disabled | Not Running | voxl-lepton-tracker | Disabled | Not Running | voxl-logger | Disabled | Not Running | voxl-mavcam-manager | Enabled | Running | 0.0% voxl-mavlink-server | Enabled | Running | 3.1% voxl-modem | Disabled | Not Running | voxl-neopixel-manager | Disabled | Not Running | voxl-open-vins-server | Disabled | Not Running | voxl-osd | Disabled | Not Running | voxl-portal | Enabled | Running | 24.1% voxl-px4-imu-server | Disabled | Not Running | voxl-px4 | Enabled | Running | 21.6% voxl-qvio-server | Enabled | Running | 5.0% voxl-rangefinder-server | Enabled | Running | 0.5% voxl-remote-id | Disabled | Not Running | voxl-softap | Disabled | Not Running | voxl-state-estimator | Disabled | Not Running | voxl-static-ip | Disabled | Not Running | voxl-streamer | Enabled | Running | 0.0% voxl-tag-detector | Disabled | Not Running | voxl-tflite-server | Disabled | Not Running | voxl-time-sync | Disabled | Not Running | voxl-uvc-server | Disabled | Not Running | voxl-vision-hub | Enabled | Running | 4.1% voxl-vrx | Disabled | Not Running | voxl-vtx | Disabled | Not Running | voxl-wait-for-fs | Enabled | Completed | voxl2:/$ cat /etc/modalai/voxl-camera-server.conf /** * voxl-camera-server Configuration File * * Each camera has configurations for up to 4 HAL3 streams: * - `preview` stream for raw unprocessed images from CV cameras * - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming * - `large_video` 4k (ish) h264/h265 for onboard video recording to disk * - `snapshot` ISP-processed JPG snapshots that get saved to disk * * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled * * This file is generated from default values by voxl-configure-cameras. * Do not expect arbitrary resolutions to work, the ISP and video compression * pipelines only support very specific resolutions. * * The default video compression mode is cqp or Constant Quantization Parameter * * * */ { "version": 0.1, "fsync_en": true, "fsync_gpio": 109, "cameras": [{ "type": "ar0144", "name": "tracking_front", "enabled": true, "camera_id": 0, "fps": 30, "en_rotate": true, "en_preview": true, "preview_width": 1280, "preview_height": 800, "en_raw_preview": true, "ae_mode": "lme_msv", "ae_desired_msv": 60, "exposure_min_us": 20, "exposure_max_us": 33000, "gain_min": 54, "gain_max": 8000, "exposure_soft_min_us": 5000, "ae_filter_alpha": 0.600000023841858, "ae_ignore_fraction": 0.20000000298023224, "ae_slope": 0.05000000074505806, "ae_exposure_period": 1, "ae_gain_period": 1 }, { "type": "imx412", "name": "hires", "enabled": true, "camera_id": 1, "fps": 30, "en_preview": false, "preview_width": 640, "preview_height": 480, "en_raw_preview": false, "en_small_video": true, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h265", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "en_large_video": true, "large_video_width": 4056, "large_video_height": 3040, "large_venc_mode": "h265", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 38, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 30, "en_snapshot": true, "en_snapshot_width": 4056, "en_snapshot_height": 3040, "exif_focal_length": 3.0999999046325684, "exif_focal_length_in_35mm_format": 17, "exif_fnumber": 1.2400000095367432, "ae_mode": "isp" }, { "type": "pmd-tof-liow2", "name": "tof", "enabled": true, "camera_id": 2, "fps": 10, "en_rotate": true, "ae_mode": "off", "exposure_max_us": 6000, "standby_enabled": false, "decimator": 5 }, { "type": "ar0144", "name": "tracking_down", "enabled": true, "camera_id": 3, "fps": 30, "en_rotate": false, "en_preview": true, "preview_width": 1280, "preview_height": 800, "en_raw_preview": true, "ae_mode": "lme_msv", "ae_desired_msv": 60, "exposure_min_us": 20, "exposure_max_us": 33000, "gain_min": 54, "gain_max": 8000, "exposure_soft_min_us": 5000, "ae_filter_alpha": 0.600000023841858, "ae_ignore_fraction": 0.20000000298023224, "ae_slope": 0.05000000074505806, "ae_exposure_period": 1, "ae_gain_period": 1 }] } voxl2:/$ cat /etc/modalai/voxl-qvio-server.conf /** * This file contains configuration that's specific to voxl-qvio-server. * * voxl-qvio-server also uses parameters from the following config files: * /etc/modalai/extrinsics.conf * /data/modalai/opencv_tracking_intrinsics.yml * * parameter descriptions: * * mask_file_path: A file path for qvio mask. * 1/4 resolution image (w.r.t. VGA), 160 x 120, * PGM format, the part of the camera view for which * pixels are set to 255 is blocked from feature * detection useful, e.g., to avoid detecting & tracking * points on landing gear reaching into camera view. * * imu_name: VOXL uses imu1 by defualt since it's the most reliable. On * VOXL-FLIGHT you can optionally try imu0 which is a newer icm42688 * voxl-configure-mpa will decide which IMU to use for you. * * cam_name: camera to use, defaults to tracking0 * * odr_hz: Output data date is independent from the camera frame rate so you can * choose the desired output data rate. Note that voxl-imu-server defaults to * 500hz imu sampling but new data is received by voxl-qvio-server at 100hz by * default so requesting qvio data faster requires updating voxl-imu-server. * * use_camera_height_bootstrap: When enabled, the feature estimator will assume * the system starts up with the camera pointed partially at the ground and use * this to make an intial guess of the feature's depth. This should be enabled * for drones that have known-height landing gear. * * camera_height_off_ground_m: camera distance above ground (meters) for the * above bootstrap feature. * * enable_init_while_moving: optionally allow the algorithm to initialize or * reinitialize while moving. Use this if you want to be able to reinitialize * during flight. Based on camera motion, not IMU motion. Recommended to leave * this off unless absolutely desired. * * cam_imu_timeshift_s: Misalignment between camera and imu timestamp in * seconds. * * cam_imu_timeshift_s_uncertainty: uncertainty in camera imu timestamp * misalignment * * T_cam_wrt_imu_uncertainty[3] & R_cam_to_imu_uncertainty[3]: uncertainty in * camera-imu translation * * accl_fsr_ms2 & gyro_fsr_rad: Full scale range used to detect clipping. By * default this is set to a little under the real 16G and 2000DPS FSR so * clipping is detected reliably * * accl_noise_std_dev & gyro_noise_std_dev: standard deviation of accl and gyro * noise * * cam_noise_std_dev: Standard dev of camera noise per pixel. * * min_std_pixel_noise: Minimum of standard deviation of feature measurement * noise in pixels. * * fail_high_pixel_noise_points: Scales measurement noise and compares against * search area (is search area large enough to reliably compute measurement * noise covariance matrix). * * limited_imu_bw_trigger: To prevent tracking failure during/right after (hard) * landing: If sum of 3 consecutive accelerometer samples in any dimension * divided by 4.3 exceed this threshold, IMU measurement noise is increased (and * resets become more likely); if platform vibrates heavily during flight, this * may trigger mid- flight; if poseQuality in mvVISLAMPose drops to * MV_TRACKING_STATE_LOW_QUALITY during flight, improve mechanical dampening * (and/or increase threshold). QC default is 35, we prefer 25. * * gps_imu_time_alignment_s: Misalignment between GPS and IMU time in seconds * * T_gps_wrt_imu: location of gps with respect to the IMU in meters * * enable_mapping: rudimentary lightweight mapping of feature points, leave this * on. * * enable_gps_vel: allow gps velocity to be added to VIO, requires ModalAI * custom PX4 firmware to expose this data. This is an Alpha feature, don't * enable this unless instructed to. * * * * en_auto_reset: on by default. If any one of the following thresholds are met * voxl-qvio-server will assume an otherwise undetected fault has occured in * the inderlying algorithm and will hard reset the QVIO library. * * auto_reset_max_velocity: meters per second, if velocity faster than this is * detected then an auto-reset is triggered * * auto_reset_max_v_cov_instant: if the covariance of the velocity estimate ever * exceeds this for a since sample then an auto-reset is triggered * * auto_reset_max_v_cov: if th covariance of the velocity estimate ever exceeds * this for more than auto_reset_max_v_cov_timeout_s then an auto-reset is triggered * * auto_reset_max_v_cov_timeout_s: time period for auto_reset_max_v_cov criteria * * auto_reset_min_features: if fewer than this number of features are tracked for * more than auto_reset_min_feature_timeout_s then an auto-reset is triggered. * it is okay for feature tracking to be lost for short periods of time. * * * * en_standby_mode: When enabled, qvio will skip camera frames while voxl-cpu-monitor * reports that the CPU is in standby mode. * * standby_skip_frames: number of frames to skip before processing one. default is 1, meaning skip every other frame. 30 -> 15hz */ { "imu_name": "imu_apps", "cam_name": "tracking_front", "lens_cal_path": "/data/modalai/opencv_tracking_front_intrinsics.yml", "cam_extrinsics_name": "tracking_front", "use_second_cam_if_stereo": false, "odr_hz": 30, "use_camera_height_bootstrap": true, "camera_height_off_ground_m": 0.085000000894069672, "enable_init_while_moving": false, "cam_imu_timeshift_s": -0.0020000000949949026, "cam_imu_timeshift_s_uncertainty": 0.0010000000474974513, "T_cam_wrt_imu_uncertainty": [0.00050000002374872565, 0.00050000002374872565, 0.00050000002374872565], "R_cam_to_imu_uncertainty": [0.00499999988824129, 0.00499999988824129, 0.00499999988824129], "accl_fsr_ms2": 156, "gyro_fsr_rad": 34, "accl_noise_std_dev": 0.31600001454353333, "gyro_noise_std_dev": 0.00999999977648258, "cam_noise_std_dev": 100, "min_std_pixel_noise": 0.5, "fail_high_pixel_noise_points": 1.6650999784469604, "limited_imu_bw_trigger": 25, "gps_imu_time_alignment_s": 0, "T_gps_wrt_imu": [0, 0, 0], "enable_mapping": true, "enable_gps_vel": false, "en_auto_reset": true, "auto_reset_max_velocity": 10, "auto_reset_max_v_cov_instant": 0.10000000149011612, "auto_reset_max_v_cov": 0.00999999977648258, "auto_reset_max_v_cov_timeout_s": 0.5, "auto_reset_min_features": 3, "auto_reset_min_feature_timeout_s": 1, "en_standby_mode": true, "standby_skip_frames": 1, "mask_file_path": "" } voxl2:/$ -
Hello,
I recently received a Startling 2 drone and have been trying to get the Mapper functionality to work.I tried SDK 1.3.3 and then tried 1.3.4-202409192201.
TOF seems to be working fine, as is the tracking_front camera however QVIO is not. See the attached screenshot and logs.
Any advice on how to resolve this would be appreciated.
voxl2:/$ voxl-inspect-qvio waiting for server dt(ms) | T_imu_wrt_vio (m) |Roll Pitch Yaw (deg)|features|quality| state| error_codes | 31.1 | 0.00 0.00 0.00| 0.0 -0.0 0.0| 0 | -1% | FAIL | CAM_MISSING CAM_BAD_RES STALLED ^C received SIGINT Ctrl-C closing and exiting voxl2:/$ voxl-inspect-services Scanning services... Service Name | Enabled | Running | CPU Usage ------------------------------------------------------------------- docker-autorun | Disabled | Not Running | modallink-relink | Disabled | Not Running | voxl-camera-server | Enabled | Running | 93.9% voxl-cpu-monitor | Enabled | Running | 0.3% voxl-dfs-server | Disabled | Not Running | voxl-feature-tracker | Disabled | Not Running | voxl-flow-server | Disabled | Not Running | voxl-imu-server | Enabled | Running | 3.6% voxl-lepton-server | Disabled | Not Running | voxl-lepton-tracker | Disabled | Not Running | voxl-logger | Disabled | Not Running | voxl-mavcam-manager | Enabled | Running | 0.0% voxl-mavlink-server | Enabled | Running | 3.1% voxl-modem | Disabled | Not Running | voxl-neopixel-manager | Disabled | Not Running | voxl-open-vins-server | Disabled | Not Running | voxl-osd | Disabled | Not Running | voxl-portal | Enabled | Running | 24.1% voxl-px4-imu-server | Disabled | Not Running | voxl-px4 | Enabled | Running | 21.6% voxl-qvio-server | Enabled | Running | 5.0% voxl-rangefinder-server | Enabled | Running | 0.5% voxl-remote-id | Disabled | Not Running | voxl-softap | Disabled | Not Running | voxl-state-estimator | Disabled | Not Running | voxl-static-ip | Disabled | Not Running | voxl-streamer | Enabled | Running | 0.0% voxl-tag-detector | Disabled | Not Running | voxl-tflite-server | Disabled | Not Running | voxl-time-sync | Disabled | Not Running | voxl-uvc-server | Disabled | Not Running | voxl-vision-hub | Enabled | Running | 4.1% voxl-vrx | Disabled | Not Running | voxl-vtx | Disabled | Not Running | voxl-wait-for-fs | Enabled | Completed | voxl2:/$ cat /etc/modalai/voxl-camera-server.conf /** * voxl-camera-server Configuration File * * Each camera has configurations for up to 4 HAL3 streams: * - `preview` stream for raw unprocessed images from CV cameras * - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming * - `large_video` 4k (ish) h264/h265 for onboard video recording to disk * - `snapshot` ISP-processed JPG snapshots that get saved to disk * * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled * * This file is generated from default values by voxl-configure-cameras. * Do not expect arbitrary resolutions to work, the ISP and video compression * pipelines only support very specific resolutions. * * The default video compression mode is cqp or Constant Quantization Parameter * * * */ { "version": 0.1, "fsync_en": true, "fsync_gpio": 109, "cameras": [{ "type": "ar0144", "name": "tracking_front", "enabled": true, "camera_id": 0, "fps": 30, "en_rotate": true, "en_preview": true, "preview_width": 1280, "preview_height": 800, "en_raw_preview": true, "ae_mode": "lme_msv", "ae_desired_msv": 60, "exposure_min_us": 20, "exposure_max_us": 33000, "gain_min": 54, "gain_max": 8000, "exposure_soft_min_us": 5000, "ae_filter_alpha": 0.600000023841858, "ae_ignore_fraction": 0.20000000298023224, "ae_slope": 0.05000000074505806, "ae_exposure_period": 1, "ae_gain_period": 1 }, { "type": "imx412", "name": "hires", "enabled": true, "camera_id": 1, "fps": 30, "en_preview": false, "preview_width": 640, "preview_height": 480, "en_raw_preview": false, "en_small_video": true, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h265", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "en_large_video": true, "large_video_width": 4056, "large_video_height": 3040, "large_venc_mode": "h265", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 38, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 30, "en_snapshot": true, "en_snapshot_width": 4056, "en_snapshot_height": 3040, "exif_focal_length": 3.0999999046325684, "exif_focal_length_in_35mm_format": 17, "exif_fnumber": 1.2400000095367432, "ae_mode": "isp" }, { "type": "pmd-tof-liow2", "name": "tof", "enabled": true, "camera_id": 2, "fps": 10, "en_rotate": true, "ae_mode": "off", "exposure_max_us": 6000, "standby_enabled": false, "decimator": 5 }, { "type": "ar0144", "name": "tracking_down", "enabled": true, "camera_id": 3, "fps": 30, "en_rotate": false, "en_preview": true, "preview_width": 1280, "preview_height": 800, "en_raw_preview": true, "ae_mode": "lme_msv", "ae_desired_msv": 60, "exposure_min_us": 20, "exposure_max_us": 33000, "gain_min": 54, "gain_max": 8000, "exposure_soft_min_us": 5000, "ae_filter_alpha": 0.600000023841858, "ae_ignore_fraction": 0.20000000298023224, "ae_slope": 0.05000000074505806, "ae_exposure_period": 1, "ae_gain_period": 1 }] } voxl2:/$ cat /etc/modalai/voxl-qvio-server.conf /** * This file contains configuration that's specific to voxl-qvio-server. * * voxl-qvio-server also uses parameters from the following config files: * /etc/modalai/extrinsics.conf * /data/modalai/opencv_tracking_intrinsics.yml * * parameter descriptions: * * mask_file_path: A file path for qvio mask. * 1/4 resolution image (w.r.t. VGA), 160 x 120, * PGM format, the part of the camera view for which * pixels are set to 255 is blocked from feature * detection useful, e.g., to avoid detecting & tracking * points on landing gear reaching into camera view. * * imu_name: VOXL uses imu1 by defualt since it's the most reliable. On * VOXL-FLIGHT you can optionally try imu0 which is a newer icm42688 * voxl-configure-mpa will decide which IMU to use for you. * * cam_name: camera to use, defaults to tracking0 * * odr_hz: Output data date is independent from the camera frame rate so you can * choose the desired output data rate. Note that voxl-imu-server defaults to * 500hz imu sampling but new data is received by voxl-qvio-server at 100hz by * default so requesting qvio data faster requires updating voxl-imu-server. * * use_camera_height_bootstrap: When enabled, the feature estimator will assume * the system starts up with the camera pointed partially at the ground and use * this to make an intial guess of the feature's depth. This should be enabled * for drones that have known-height landing gear. * * camera_height_off_ground_m: camera distance above ground (meters) for the * above bootstrap feature. * * enable_init_while_moving: optionally allow the algorithm to initialize or * reinitialize while moving. Use this if you want to be able to reinitialize * during flight. Based on camera motion, not IMU motion. Recommended to leave * this off unless absolutely desired. * * cam_imu_timeshift_s: Misalignment between camera and imu timestamp in * seconds. * * cam_imu_timeshift_s_uncertainty: uncertainty in camera imu timestamp * misalignment * * T_cam_wrt_imu_uncertainty[3] & R_cam_to_imu_uncertainty[3]: uncertainty in * camera-imu translation * * accl_fsr_ms2 & gyro_fsr_rad: Full scale range used to detect clipping. By * default this is set to a little under the real 16G and 2000DPS FSR so * clipping is detected reliably * * accl_noise_std_dev & gyro_noise_std_dev: standard deviation of accl and gyro * noise * * cam_noise_std_dev: Standard dev of camera noise per pixel. * * min_std_pixel_noise: Minimum of standard deviation of feature measurement * noise in pixels. * * fail_high_pixel_noise_points: Scales measurement noise and compares against * search area (is search area large enough to reliably compute measurement * noise covariance matrix). * * limited_imu_bw_trigger: To prevent tracking failure during/right after (hard) * landing: If sum of 3 consecutive accelerometer samples in any dimension * divided by 4.3 exceed this threshold, IMU measurement noise is increased (and * resets become more likely); if platform vibrates heavily during flight, this * may trigger mid- flight; if poseQuality in mvVISLAMPose drops to * MV_TRACKING_STATE_LOW_QUALITY during flight, improve mechanical dampening * (and/or increase threshold). QC default is 35, we prefer 25. * * gps_imu_time_alignment_s: Misalignment between GPS and IMU time in seconds * * T_gps_wrt_imu: location of gps with respect to the IMU in meters * * enable_mapping: rudimentary lightweight mapping of feature points, leave this * on. * * enable_gps_vel: allow gps velocity to be added to VIO, requires ModalAI * custom PX4 firmware to expose this data. This is an Alpha feature, don't * enable this unless instructed to. * * * * en_auto_reset: on by default. If any one of the following thresholds are met * voxl-qvio-server will assume an otherwise undetected fault has occured in * the inderlying algorithm and will hard reset the QVIO library. * * auto_reset_max_velocity: meters per second, if velocity faster than this is * detected then an auto-reset is triggered * * auto_reset_max_v_cov_instant: if the covariance of the velocity estimate ever * exceeds this for a since sample then an auto-reset is triggered * * auto_reset_max_v_cov: if th covariance of the velocity estimate ever exceeds * this for more than auto_reset_max_v_cov_timeout_s then an auto-reset is triggered * * auto_reset_max_v_cov_timeout_s: time period for auto_reset_max_v_cov criteria * * auto_reset_min_features: if fewer than this number of features are tracked for * more than auto_reset_min_feature_timeout_s then an auto-reset is triggered. * it is okay for feature tracking to be lost for short periods of time. * * * * en_standby_mode: When enabled, qvio will skip camera frames while voxl-cpu-monitor * reports that the CPU is in standby mode. * * standby_skip_frames: number of frames to skip before processing one. default is 1, meaning skip every other frame. 30 -> 15hz */ { "imu_name": "imu_apps", "cam_name": "tracking_front", "lens_cal_path": "/data/modalai/opencv_tracking_front_intrinsics.yml", "cam_extrinsics_name": "tracking_front", "use_second_cam_if_stereo": false, "odr_hz": 30, "use_camera_height_bootstrap": true, "camera_height_off_ground_m": 0.085000000894069672, "enable_init_while_moving": false, "cam_imu_timeshift_s": -0.0020000000949949026, "cam_imu_timeshift_s_uncertainty": 0.0010000000474974513, "T_cam_wrt_imu_uncertainty": [0.00050000002374872565, 0.00050000002374872565, 0.00050000002374872565], "R_cam_to_imu_uncertainty": [0.00499999988824129, 0.00499999988824129, 0.00499999988824129], "accl_fsr_ms2": 156, "gyro_fsr_rad": 34, "accl_noise_std_dev": 0.31600001454353333, "gyro_noise_std_dev": 0.00999999977648258, "cam_noise_std_dev": 100, "min_std_pixel_noise": 0.5, "fail_high_pixel_noise_points": 1.6650999784469604, "limited_imu_bw_trigger": 25, "gps_imu_time_alignment_s": 0, "T_gps_wrt_imu": [0, 0, 0], "enable_mapping": true, "enable_gps_vel": false, "en_auto_reset": true, "auto_reset_max_velocity": 10, "auto_reset_max_v_cov_instant": 0.10000000149011612, "auto_reset_max_v_cov": 0.00999999977648258, "auto_reset_max_v_cov_timeout_s": 0.5, "auto_reset_min_features": 3, "auto_reset_min_feature_timeout_s": 1, "en_standby_mode": true, "standby_skip_frames": 1, "mask_file_path": "" } voxl2:/$This post is deleted! -
Hello,
I recently received a Startling 2 drone and have been trying to get the Mapper functionality to work.I tried SDK 1.3.3 and then tried 1.3.4-202409192201.
TOF seems to be working fine, as is the tracking_front camera however QVIO is not. See the attached screenshot and logs.
Any advice on how to resolve this would be appreciated.
voxl2:/$ voxl-inspect-qvio waiting for server dt(ms) | T_imu_wrt_vio (m) |Roll Pitch Yaw (deg)|features|quality| state| error_codes | 31.1 | 0.00 0.00 0.00| 0.0 -0.0 0.0| 0 | -1% | FAIL | CAM_MISSING CAM_BAD_RES STALLED ^C received SIGINT Ctrl-C closing and exiting voxl2:/$ voxl-inspect-services Scanning services... Service Name | Enabled | Running | CPU Usage ------------------------------------------------------------------- docker-autorun | Disabled | Not Running | modallink-relink | Disabled | Not Running | voxl-camera-server | Enabled | Running | 93.9% voxl-cpu-monitor | Enabled | Running | 0.3% voxl-dfs-server | Disabled | Not Running | voxl-feature-tracker | Disabled | Not Running | voxl-flow-server | Disabled | Not Running | voxl-imu-server | Enabled | Running | 3.6% voxl-lepton-server | Disabled | Not Running | voxl-lepton-tracker | Disabled | Not Running | voxl-logger | Disabled | Not Running | voxl-mavcam-manager | Enabled | Running | 0.0% voxl-mavlink-server | Enabled | Running | 3.1% voxl-modem | Disabled | Not Running | voxl-neopixel-manager | Disabled | Not Running | voxl-open-vins-server | Disabled | Not Running | voxl-osd | Disabled | Not Running | voxl-portal | Enabled | Running | 24.1% voxl-px4-imu-server | Disabled | Not Running | voxl-px4 | Enabled | Running | 21.6% voxl-qvio-server | Enabled | Running | 5.0% voxl-rangefinder-server | Enabled | Running | 0.5% voxl-remote-id | Disabled | Not Running | voxl-softap | Disabled | Not Running | voxl-state-estimator | Disabled | Not Running | voxl-static-ip | Disabled | Not Running | voxl-streamer | Enabled | Running | 0.0% voxl-tag-detector | Disabled | Not Running | voxl-tflite-server | Disabled | Not Running | voxl-time-sync | Disabled | Not Running | voxl-uvc-server | Disabled | Not Running | voxl-vision-hub | Enabled | Running | 4.1% voxl-vrx | Disabled | Not Running | voxl-vtx | Disabled | Not Running | voxl-wait-for-fs | Enabled | Completed | voxl2:/$ cat /etc/modalai/voxl-camera-server.conf /** * voxl-camera-server Configuration File * * Each camera has configurations for up to 4 HAL3 streams: * - `preview` stream for raw unprocessed images from CV cameras * - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming * - `large_video` 4k (ish) h264/h265 for onboard video recording to disk * - `snapshot` ISP-processed JPG snapshots that get saved to disk * * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled * * This file is generated from default values by voxl-configure-cameras. * Do not expect arbitrary resolutions to work, the ISP and video compression * pipelines only support very specific resolutions. * * The default video compression mode is cqp or Constant Quantization Parameter * * * */ { "version": 0.1, "fsync_en": true, "fsync_gpio": 109, "cameras": [{ "type": "ar0144", "name": "tracking_front", "enabled": true, "camera_id": 0, "fps": 30, "en_rotate": true, "en_preview": true, "preview_width": 1280, "preview_height": 800, "en_raw_preview": true, "ae_mode": "lme_msv", "ae_desired_msv": 60, "exposure_min_us": 20, "exposure_max_us": 33000, "gain_min": 54, "gain_max": 8000, "exposure_soft_min_us": 5000, "ae_filter_alpha": 0.600000023841858, "ae_ignore_fraction": 0.20000000298023224, "ae_slope": 0.05000000074505806, "ae_exposure_period": 1, "ae_gain_period": 1 }, { "type": "imx412", "name": "hires", "enabled": true, "camera_id": 1, "fps": 30, "en_preview": false, "preview_width": 640, "preview_height": 480, "en_raw_preview": false, "en_small_video": true, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h265", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "en_large_video": true, "large_video_width": 4056, "large_video_height": 3040, "large_venc_mode": "h265", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 38, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 30, "en_snapshot": true, "en_snapshot_width": 4056, "en_snapshot_height": 3040, "exif_focal_length": 3.0999999046325684, "exif_focal_length_in_35mm_format": 17, "exif_fnumber": 1.2400000095367432, "ae_mode": "isp" }, { "type": "pmd-tof-liow2", "name": "tof", "enabled": true, "camera_id": 2, "fps": 10, "en_rotate": true, "ae_mode": "off", "exposure_max_us": 6000, "standby_enabled": false, "decimator": 5 }, { "type": "ar0144", "name": "tracking_down", "enabled": true, "camera_id": 3, "fps": 30, "en_rotate": false, "en_preview": true, "preview_width": 1280, "preview_height": 800, "en_raw_preview": true, "ae_mode": "lme_msv", "ae_desired_msv": 60, "exposure_min_us": 20, "exposure_max_us": 33000, "gain_min": 54, "gain_max": 8000, "exposure_soft_min_us": 5000, "ae_filter_alpha": 0.600000023841858, "ae_ignore_fraction": 0.20000000298023224, "ae_slope": 0.05000000074505806, "ae_exposure_period": 1, "ae_gain_period": 1 }] } voxl2:/$ cat /etc/modalai/voxl-qvio-server.conf /** * This file contains configuration that's specific to voxl-qvio-server. * * voxl-qvio-server also uses parameters from the following config files: * /etc/modalai/extrinsics.conf * /data/modalai/opencv_tracking_intrinsics.yml * * parameter descriptions: * * mask_file_path: A file path for qvio mask. * 1/4 resolution image (w.r.t. VGA), 160 x 120, * PGM format, the part of the camera view for which * pixels are set to 255 is blocked from feature * detection useful, e.g., to avoid detecting & tracking * points on landing gear reaching into camera view. * * imu_name: VOXL uses imu1 by defualt since it's the most reliable. On * VOXL-FLIGHT you can optionally try imu0 which is a newer icm42688 * voxl-configure-mpa will decide which IMU to use for you. * * cam_name: camera to use, defaults to tracking0 * * odr_hz: Output data date is independent from the camera frame rate so you can * choose the desired output data rate. Note that voxl-imu-server defaults to * 500hz imu sampling but new data is received by voxl-qvio-server at 100hz by * default so requesting qvio data faster requires updating voxl-imu-server. * * use_camera_height_bootstrap: When enabled, the feature estimator will assume * the system starts up with the camera pointed partially at the ground and use * this to make an intial guess of the feature's depth. This should be enabled * for drones that have known-height landing gear. * * camera_height_off_ground_m: camera distance above ground (meters) for the * above bootstrap feature. * * enable_init_while_moving: optionally allow the algorithm to initialize or * reinitialize while moving. Use this if you want to be able to reinitialize * during flight. Based on camera motion, not IMU motion. Recommended to leave * this off unless absolutely desired. * * cam_imu_timeshift_s: Misalignment between camera and imu timestamp in * seconds. * * cam_imu_timeshift_s_uncertainty: uncertainty in camera imu timestamp * misalignment * * T_cam_wrt_imu_uncertainty[3] & R_cam_to_imu_uncertainty[3]: uncertainty in * camera-imu translation * * accl_fsr_ms2 & gyro_fsr_rad: Full scale range used to detect clipping. By * default this is set to a little under the real 16G and 2000DPS FSR so * clipping is detected reliably * * accl_noise_std_dev & gyro_noise_std_dev: standard deviation of accl and gyro * noise * * cam_noise_std_dev: Standard dev of camera noise per pixel. * * min_std_pixel_noise: Minimum of standard deviation of feature measurement * noise in pixels. * * fail_high_pixel_noise_points: Scales measurement noise and compares against * search area (is search area large enough to reliably compute measurement * noise covariance matrix). * * limited_imu_bw_trigger: To prevent tracking failure during/right after (hard) * landing: If sum of 3 consecutive accelerometer samples in any dimension * divided by 4.3 exceed this threshold, IMU measurement noise is increased (and * resets become more likely); if platform vibrates heavily during flight, this * may trigger mid- flight; if poseQuality in mvVISLAMPose drops to * MV_TRACKING_STATE_LOW_QUALITY during flight, improve mechanical dampening * (and/or increase threshold). QC default is 35, we prefer 25. * * gps_imu_time_alignment_s: Misalignment between GPS and IMU time in seconds * * T_gps_wrt_imu: location of gps with respect to the IMU in meters * * enable_mapping: rudimentary lightweight mapping of feature points, leave this * on. * * enable_gps_vel: allow gps velocity to be added to VIO, requires ModalAI * custom PX4 firmware to expose this data. This is an Alpha feature, don't * enable this unless instructed to. * * * * en_auto_reset: on by default. If any one of the following thresholds are met * voxl-qvio-server will assume an otherwise undetected fault has occured in * the inderlying algorithm and will hard reset the QVIO library. * * auto_reset_max_velocity: meters per second, if velocity faster than this is * detected then an auto-reset is triggered * * auto_reset_max_v_cov_instant: if the covariance of the velocity estimate ever * exceeds this for a since sample then an auto-reset is triggered * * auto_reset_max_v_cov: if th covariance of the velocity estimate ever exceeds * this for more than auto_reset_max_v_cov_timeout_s then an auto-reset is triggered * * auto_reset_max_v_cov_timeout_s: time period for auto_reset_max_v_cov criteria * * auto_reset_min_features: if fewer than this number of features are tracked for * more than auto_reset_min_feature_timeout_s then an auto-reset is triggered. * it is okay for feature tracking to be lost for short periods of time. * * * * en_standby_mode: When enabled, qvio will skip camera frames while voxl-cpu-monitor * reports that the CPU is in standby mode. * * standby_skip_frames: number of frames to skip before processing one. default is 1, meaning skip every other frame. 30 -> 15hz */ { "imu_name": "imu_apps", "cam_name": "tracking_front", "lens_cal_path": "/data/modalai/opencv_tracking_front_intrinsics.yml", "cam_extrinsics_name": "tracking_front", "use_second_cam_if_stereo": false, "odr_hz": 30, "use_camera_height_bootstrap": true, "camera_height_off_ground_m": 0.085000000894069672, "enable_init_while_moving": false, "cam_imu_timeshift_s": -0.0020000000949949026, "cam_imu_timeshift_s_uncertainty": 0.0010000000474974513, "T_cam_wrt_imu_uncertainty": [0.00050000002374872565, 0.00050000002374872565, 0.00050000002374872565], "R_cam_to_imu_uncertainty": [0.00499999988824129, 0.00499999988824129, 0.00499999988824129], "accl_fsr_ms2": 156, "gyro_fsr_rad": 34, "accl_noise_std_dev": 0.31600001454353333, "gyro_noise_std_dev": 0.00999999977648258, "cam_noise_std_dev": 100, "min_std_pixel_noise": 0.5, "fail_high_pixel_noise_points": 1.6650999784469604, "limited_imu_bw_trigger": 25, "gps_imu_time_alignment_s": 0, "T_gps_wrt_imu": [0, 0, 0], "enable_mapping": true, "enable_gps_vel": false, "en_auto_reset": true, "auto_reset_max_velocity": 10, "auto_reset_max_v_cov_instant": 0.10000000149011612, "auto_reset_max_v_cov": 0.00999999977648258, "auto_reset_max_v_cov_timeout_s": 0.5, "auto_reset_min_features": 3, "auto_reset_min_feature_timeout_s": 1, "en_standby_mode": true, "standby_skip_frames": 1, "mask_file_path": "" } voxl2:/$@nuallaino you can see how that error is generated here: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/blob/master/server/main.cpp?ref_type=heads#L683
-
@nuallaino you can see how that error is generated here: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/blob/master/server/main.cpp?ref_type=heads#L683
@Moderator Yes, I did see that. However I expected this to work out of the box. I haven't changed anything.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login