qvio_overlay "ERR CAM_MISSING CAM_BAD_RES STALLED" on Starling 2
-
Hello,
I recently received a Startling 2 drone and have been trying to get the Mapper functionality to work.I tried SDK 1.3.3 and then tried 1.3.4-202409192201.
TOF seems to be working fine, as is the tracking_front camera however QVIO is not. See the attached screenshot and logs.
Any advice on how to resolve this would be appreciated.
voxl2:/$ voxl-inspect-qvio waiting for server dt(ms) | T_imu_wrt_vio (m) |Roll Pitch Yaw (deg)|features|quality| state| error_codes | 31.1 | 0.00 0.00 0.00| 0.0 -0.0 0.0| 0 | -1% | FAIL | CAM_MISSING CAM_BAD_RES STALLED ^C received SIGINT Ctrl-C closing and exiting voxl2:/$ voxl-inspect-services Scanning services... Service Name | Enabled | Running | CPU Usage ------------------------------------------------------------------- docker-autorun | Disabled | Not Running | modallink-relink | Disabled | Not Running | voxl-camera-server | Enabled | Running | 93.9% voxl-cpu-monitor | Enabled | Running | 0.3% voxl-dfs-server | Disabled | Not Running | voxl-feature-tracker | Disabled | Not Running | voxl-flow-server | Disabled | Not Running | voxl-imu-server | Enabled | Running | 3.6% voxl-lepton-server | Disabled | Not Running | voxl-lepton-tracker | Disabled | Not Running | voxl-logger | Disabled | Not Running | voxl-mavcam-manager | Enabled | Running | 0.0% voxl-mavlink-server | Enabled | Running | 3.1% voxl-modem | Disabled | Not Running | voxl-neopixel-manager | Disabled | Not Running | voxl-open-vins-server | Disabled | Not Running | voxl-osd | Disabled | Not Running | voxl-portal | Enabled | Running | 24.1% voxl-px4-imu-server | Disabled | Not Running | voxl-px4 | Enabled | Running | 21.6% voxl-qvio-server | Enabled | Running | 5.0% voxl-rangefinder-server | Enabled | Running | 0.5% voxl-remote-id | Disabled | Not Running | voxl-softap | Disabled | Not Running | voxl-state-estimator | Disabled | Not Running | voxl-static-ip | Disabled | Not Running | voxl-streamer | Enabled | Running | 0.0% voxl-tag-detector | Disabled | Not Running | voxl-tflite-server | Disabled | Not Running | voxl-time-sync | Disabled | Not Running | voxl-uvc-server | Disabled | Not Running | voxl-vision-hub | Enabled | Running | 4.1% voxl-vrx | Disabled | Not Running | voxl-vtx | Disabled | Not Running | voxl-wait-for-fs | Enabled | Completed | voxl2:/$ cat /etc/modalai/voxl-camera-server.conf /** * voxl-camera-server Configuration File * * Each camera has configurations for up to 4 HAL3 streams: * - `preview` stream for raw unprocessed images from CV cameras * - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming * - `large_video` 4k (ish) h264/h265 for onboard video recording to disk * - `snapshot` ISP-processed JPG snapshots that get saved to disk * * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled * * This file is generated from default values by voxl-configure-cameras. * Do not expect arbitrary resolutions to work, the ISP and video compression * pipelines only support very specific resolutions. * * The default video compression mode is cqp or Constant Quantization Parameter * * * */ { "version": 0.1, "fsync_en": true, "fsync_gpio": 109, "cameras": [{ "type": "ar0144", "name": "tracking_front", "enabled": true, "camera_id": 0, "fps": 30, "en_rotate": true, "en_preview": true, "preview_width": 1280, "preview_height": 800, "en_raw_preview": true, "ae_mode": "lme_msv", "ae_desired_msv": 60, "exposure_min_us": 20, "exposure_max_us": 33000, "gain_min": 54, "gain_max": 8000, "exposure_soft_min_us": 5000, "ae_filter_alpha": 0.600000023841858, "ae_ignore_fraction": 0.20000000298023224, "ae_slope": 0.05000000074505806, "ae_exposure_period": 1, "ae_gain_period": 1 }, { "type": "imx412", "name": "hires", "enabled": true, "camera_id": 1, "fps": 30, "en_preview": false, "preview_width": 640, "preview_height": 480, "en_raw_preview": false, "en_small_video": true, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h265", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "en_large_video": true, "large_video_width": 4056, "large_video_height": 3040, "large_venc_mode": "h265", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 38, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 30, "en_snapshot": true, "en_snapshot_width": 4056, "en_snapshot_height": 3040, "exif_focal_length": 3.0999999046325684, "exif_focal_length_in_35mm_format": 17, "exif_fnumber": 1.2400000095367432, "ae_mode": "isp" }, { "type": "pmd-tof-liow2", "name": "tof", "enabled": true, "camera_id": 2, "fps": 10, "en_rotate": true, "ae_mode": "off", "exposure_max_us": 6000, "standby_enabled": false, "decimator": 5 }, { "type": "ar0144", "name": "tracking_down", "enabled": true, "camera_id": 3, "fps": 30, "en_rotate": false, "en_preview": true, "preview_width": 1280, "preview_height": 800, "en_raw_preview": true, "ae_mode": "lme_msv", "ae_desired_msv": 60, "exposure_min_us": 20, "exposure_max_us": 33000, "gain_min": 54, "gain_max": 8000, "exposure_soft_min_us": 5000, "ae_filter_alpha": 0.600000023841858, "ae_ignore_fraction": 0.20000000298023224, "ae_slope": 0.05000000074505806, "ae_exposure_period": 1, "ae_gain_period": 1 }] } voxl2:/$ cat /etc/modalai/voxl-qvio-server.conf /** * This file contains configuration that's specific to voxl-qvio-server. * * voxl-qvio-server also uses parameters from the following config files: * /etc/modalai/extrinsics.conf * /data/modalai/opencv_tracking_intrinsics.yml * * parameter descriptions: * * mask_file_path: A file path for qvio mask. * 1/4 resolution image (w.r.t. VGA), 160 x 120, * PGM format, the part of the camera view for which * pixels are set to 255 is blocked from feature * detection useful, e.g., to avoid detecting & tracking * points on landing gear reaching into camera view. * * imu_name: VOXL uses imu1 by defualt since it's the most reliable. On * VOXL-FLIGHT you can optionally try imu0 which is a newer icm42688 * voxl-configure-mpa will decide which IMU to use for you. * * cam_name: camera to use, defaults to tracking0 * * odr_hz: Output data date is independent from the camera frame rate so you can * choose the desired output data rate. Note that voxl-imu-server defaults to * 500hz imu sampling but new data is received by voxl-qvio-server at 100hz by * default so requesting qvio data faster requires updating voxl-imu-server. * * use_camera_height_bootstrap: When enabled, the feature estimator will assume * the system starts up with the camera pointed partially at the ground and use * this to make an intial guess of the feature's depth. This should be enabled * for drones that have known-height landing gear. * * camera_height_off_ground_m: camera distance above ground (meters) for the * above bootstrap feature. * * enable_init_while_moving: optionally allow the algorithm to initialize or * reinitialize while moving. Use this if you want to be able to reinitialize * during flight. Based on camera motion, not IMU motion. Recommended to leave * this off unless absolutely desired. * * cam_imu_timeshift_s: Misalignment between camera and imu timestamp in * seconds. * * cam_imu_timeshift_s_uncertainty: uncertainty in camera imu timestamp * misalignment * * T_cam_wrt_imu_uncertainty[3] & R_cam_to_imu_uncertainty[3]: uncertainty in * camera-imu translation * * accl_fsr_ms2 & gyro_fsr_rad: Full scale range used to detect clipping. By * default this is set to a little under the real 16G and 2000DPS FSR so * clipping is detected reliably * * accl_noise_std_dev & gyro_noise_std_dev: standard deviation of accl and gyro * noise * * cam_noise_std_dev: Standard dev of camera noise per pixel. * * min_std_pixel_noise: Minimum of standard deviation of feature measurement * noise in pixels. * * fail_high_pixel_noise_points: Scales measurement noise and compares against * search area (is search area large enough to reliably compute measurement * noise covariance matrix). * * limited_imu_bw_trigger: To prevent tracking failure during/right after (hard) * landing: If sum of 3 consecutive accelerometer samples in any dimension * divided by 4.3 exceed this threshold, IMU measurement noise is increased (and * resets become more likely); if platform vibrates heavily during flight, this * may trigger mid- flight; if poseQuality in mvVISLAMPose drops to * MV_TRACKING_STATE_LOW_QUALITY during flight, improve mechanical dampening * (and/or increase threshold). QC default is 35, we prefer 25. * * gps_imu_time_alignment_s: Misalignment between GPS and IMU time in seconds * * T_gps_wrt_imu: location of gps with respect to the IMU in meters * * enable_mapping: rudimentary lightweight mapping of feature points, leave this * on. * * enable_gps_vel: allow gps velocity to be added to VIO, requires ModalAI * custom PX4 firmware to expose this data. This is an Alpha feature, don't * enable this unless instructed to. * * * * en_auto_reset: on by default. If any one of the following thresholds are met * voxl-qvio-server will assume an otherwise undetected fault has occured in * the inderlying algorithm and will hard reset the QVIO library. * * auto_reset_max_velocity: meters per second, if velocity faster than this is * detected then an auto-reset is triggered * * auto_reset_max_v_cov_instant: if the covariance of the velocity estimate ever * exceeds this for a since sample then an auto-reset is triggered * * auto_reset_max_v_cov: if th covariance of the velocity estimate ever exceeds * this for more than auto_reset_max_v_cov_timeout_s then an auto-reset is triggered * * auto_reset_max_v_cov_timeout_s: time period for auto_reset_max_v_cov criteria * * auto_reset_min_features: if fewer than this number of features are tracked for * more than auto_reset_min_feature_timeout_s then an auto-reset is triggered. * it is okay for feature tracking to be lost for short periods of time. * * * * en_standby_mode: When enabled, qvio will skip camera frames while voxl-cpu-monitor * reports that the CPU is in standby mode. * * standby_skip_frames: number of frames to skip before processing one. default is 1, meaning skip every other frame. 30 -> 15hz */ { "imu_name": "imu_apps", "cam_name": "tracking_front", "lens_cal_path": "/data/modalai/opencv_tracking_front_intrinsics.yml", "cam_extrinsics_name": "tracking_front", "use_second_cam_if_stereo": false, "odr_hz": 30, "use_camera_height_bootstrap": true, "camera_height_off_ground_m": 0.085000000894069672, "enable_init_while_moving": false, "cam_imu_timeshift_s": -0.0020000000949949026, "cam_imu_timeshift_s_uncertainty": 0.0010000000474974513, "T_cam_wrt_imu_uncertainty": [0.00050000002374872565, 0.00050000002374872565, 0.00050000002374872565], "R_cam_to_imu_uncertainty": [0.00499999988824129, 0.00499999988824129, 0.00499999988824129], "accl_fsr_ms2": 156, "gyro_fsr_rad": 34, "accl_noise_std_dev": 0.31600001454353333, "gyro_noise_std_dev": 0.00999999977648258, "cam_noise_std_dev": 100, "min_std_pixel_noise": 0.5, "fail_high_pixel_noise_points": 1.6650999784469604, "limited_imu_bw_trigger": 25, "gps_imu_time_alignment_s": 0, "T_gps_wrt_imu": [0, 0, 0], "enable_mapping": true, "enable_gps_vel": false, "en_auto_reset": true, "auto_reset_max_velocity": 10, "auto_reset_max_v_cov_instant": 0.10000000149011612, "auto_reset_max_v_cov": 0.00999999977648258, "auto_reset_max_v_cov_timeout_s": 0.5, "auto_reset_min_features": 3, "auto_reset_min_feature_timeout_s": 1, "en_standby_mode": true, "standby_skip_frames": 1, "mask_file_path": "" } voxl2:/$ -
This post is deleted! -
@nuallaino you can see how that error is generated here: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/blob/master/server/main.cpp?ref_type=heads#L683
-
@Moderator Yes, I did see that. However I expected this to work out of the box. I haven't changed anything.
