ModalAI Forum

@cch try to update voxl-mpa-to-ros, they should have fixed it in https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-to-ros/-/commit/921db500db1f7109c3f2dde870a2a31ed0c38f2c

Hi, I'm currently working on an application that requires depth information. After configuring voxl-dfs-server I'm able to obtain a good disparity. I compute the depth information as so: depth = focal length*baseline/disparity. The result that I obtain seems wrong by a scaling factor. Focal length and baseline are taken from the calibration file obtained from voxl-calibrate-camera stereo.

Can you share some information on how to perform this operation?

dfs config file:

"dfs_pair_1_input_pipe":	"stereo_rear",
	"dfs_pair_1_skip_n_frames":	1,
	"dfs_pair_1_blur_size":	9,
	"dfs_pair_1_post_median_size":	15,
	"dfs_pair_1_intrinsics_file":	"/data/modalai/opencv_stereo_rear_intrinsics.yml",
	"dfs_pair_1_extrinsics_file":	"/data/modalai/opencv_stereo_rear_extrinsics.yml",
	"dfs_pair_1_en_cvp_rectification":	1,
	"dfs_pair_1_min_disparity":	4,
	"dfs_pair_1_max_disparity":	64,
	"dfs_pair_1_min_threshold":	10,
	"dfs_pair_1_cost_threshold":	80,

Hi @Zachary-Lowell-0, sorry for the late reply.
I've run the test that you asked with the new code that you wrote (v0.3.7). The results were pretty random, we go from a lower bound of 3 detections sent through voxl-mpa-to-ros over 10 detections present in tflite, to an upper bound of 8 over 11.
This happens also for a small number of detections i.e. 2/4 as lower bound to 3/3 as an upper bound.

Are your tests aligned with mine? I'm still using a custom yolo v5 for 2 class detections.

Hi @Zachary-Lowell-0, I've had the same thought, so I lowered the frame rate of the hires camera to 10 FPS. This proved to solve some lag issues but the publishing issue remains.

I've performed another test: since I thought that the detections were too many, I've edited voxl-tflite-server to exclude one of the 2 classes (the most frequent one) from the detection pipe, publishing only the less frequent one (roughly 3-5 detections per frame). To do this, I've simply put an if to filter the detections by looking at the class name at line 243 (https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server/-/blob/master/src/main.cpp?ref_type=heads). Unfortunately, the issue still persists.

I will run the tests that you asked and I'll let you know asap. Thanks for your support.

Thanks for your answer, I'm using a custom-trained yolov5 model with 2 classes

Hi everyone,
I'm working with tflite to do object detection on VOXL2, sdk 0.9.5. The output is then sent to ros by voxl-mpa-to-ros that publishes tflite_data topic. I'm facing the following issue: only a part of the ai_detection messages within the same frame are sent. I've verified this by running rostopic echo /tflite_data and comparing it with the tflite image in which I print the frame ID. This way I can see that NOT all the detections present in a single frame in voxl-portal are actually sent via voxl-mpa-to-ros.

Why am I not able to get all the detections present in one frame from voxl-mpa-to-ros? Has this ever happened to anyone? How can I fix this?

Hi, I've flashed my sentinel with the new sdk 1.0.0. Errors similar to
Image Flashing Error occurred but I was able to successfully flash the image by following the steps reported in that post.

Unfortunately voxl-configure-mpa fails with this output:

executing: voxl-configure-cameras 3  
Camera Configuration: 3
calling camera-server-config-helper tracking:ov7251:2 hires:imx214:3 stereo:ov7251:0:1
successfully wrote this camera config to disk:
=================================================================
configuration for 3 cameras:

cam #0
    name:                tracking
    sensor type:         ov7251
    isEnabled:           1
    camId:               2
    camId2:              -1
    fps:                 30

    en_preview:          1
    pre_width:           640
    pre_height:          480
    pre_format:          raw8

    en_small_video:      0
    small_video_width:   -1
    small_video_height:  -1

    en_large_video:      0
    large_video_width:   -1
    large_video_height:  -1

    en_snapshot:         0
    snap_width:          -1
    snap_height:         -1

    ae_mode:             lme_msv
    standby_enabled:     0
    decimator:           1
    independent_exposure:0

cam #1
    name:                hires
    sensor type:         imx214
    isEnabled:           1
    camId:               3
    camId2:              -1
    fps:                 30

    en_preview:          0
    pre_width:           640
    pre_height:          480
    pre_format:          nv21

    en_small_video:      1
    small_video_width:   1024
    small_video_height:  768

    en_large_video:      1
    large_video_width:   4096
    large_video_height:  2160

    en_snapshot:         1
    snap_width:          4160
    snap_height:         3120

    ae_mode:             isp
    standby_enabled:     0
    decimator:           1
    independent_exposure:0

cam #2
    name:                stereo
    sensor type:         ov7251
    isEnabled:           1
    camId:               0
    camId2:              1
    fps:                 30

    en_preview:          1
    pre_width:           640
    pre_height:          480
    pre_format:          raw8

    en_small_video:      0
    small_video_width:   -1
    small_video_height:  -1

    en_large_video:      0
    large_video_width:   -1
    large_video_height:  -1

    en_snapshot:         0
    snap_width:          -1
    snap_height:         -1

    ae_mode:             lme_msv
    standby_enabled:     0
    decimator:           1
    independent_exposure:0

=================================================================
camera_server_config_helper is done
copying required camera drivers in place
/usr/share/modalai/chi-cdk/ov7251/com.qti.sensormodule.ov7251_2.bin
/usr/share/modalai/chi-cdk/ov7251/com.qti.sensormodule.ov7251_combo_0.bin
cp: cannot stat '/usr/share/modalai/chi-cdk/ov7251/com.qti.sensormodule.ov7251_combo_0.bin': No such file or directory
FAILED TO EXECUTE: voxl-configure-cameras 3 

How can I solve this?

system-image: 1.6.2-M0054-14.1a-perf
kernel: #1 SMP PREEMPT Fri May 19 22:19:33 UTC 2023 4.19.125
hw version: M0054
voxl-suite: 1.0.0

@Maxim-Temp hi, I'm facing the same issue, did you manage to solve this?

Hi, I'm working with the stereo cameras. The stereo images and the disparity image are shown on voxl-portal. After launching voxl-mpa-to-ros and subscribing to /stereo_disparity/left with rostopic echo, voxl-mpa-to-ros prompt this error:

Stereo interface received non-stereo frame, exiting stereo

After that, the topics related to stereo interface (/stereo_disparity/left,/stereo_disparity/right) disappear. Same happens with /stereo_disparity_scaled/right

I also tried to visualize the disparity on rviz. In that case the mpa-to-ros crashes with the following:

[voxl_mpa_to_ros_node-2] process has died [pid 5846, exit code -11, cmd /opt/ros/melodic/lib/voxl_mpa_to_ros/voxl_mpa_to_ros_node __name:=voxl_mpa_to_ros_node __log:=/home/root/.ros/log/2bfc2770-2715-11ee-9d49-00c0cab15bca/voxl_mpa_to_ros_node-2.log].
log file: /home/root/.ros/log/2bfc2770-2715-11ee-9d49-00c0cab15bca/voxl_mpa_to_ros_node-2*.log

Any clue on what is going on?

system-image: 1.5.5-M0054-14.1a-perf
kernel: #1 SMP PREEMPT Fri Mar 24 00:48:55 UTC 2023 4.19.125
hw version: M0054
voxl-suite: 0.9.5

@vivek-rk hi, I've found your same problem, did you manage to solve it?

Hi,

I'm trying to setup the tag-detector module to perform relocalization with fixed apriltags. I've enabled the service through the command: voxl-configure-tag-detector and setup the files according to the guide, even though some files and parameters names are different.

The command voxl-vision-hub -m detects properly the distances between camera and tag.
Also by running voxl-vision-hub -p it seems that the coordinates that the scripts returns are correct.
However, when I tried to start the drone offset from the origin and I framed the tag nothing happened. I am expecting that the odometry of the drone shifts to the fixed frame coordinate.

How can I solve the issue?

Here below all the information that I think could be relevant:

voxl-version output:

system-image: 1.4.1-M0054-14.1a-perf
kernel: #1 SMP PREEMPT Thu Oct 13 17:48:16 UTC 2022 4.19.125

hw version: M0054

voxl-suite:

Packages:
Repo: http://voxl-packages.modalai.com/ qrb5165 staging
Last Updated: 2023-05-31 14:22:56
List:
libmodal-cv 0.3.1-202305281402
libmodal-exposure 0.0.8-202305241844
libmodal-journal 0.2.2-202211021731
libmodal-json 0.4.3-202209281920
libmodal-pipe 2.8.4-202305292215
libqrb5165-io 0.2.0-202212220203
libvoxl-cci-direct 0.1.5-202210121755
libvoxl-cutils 0.1.1-202210072352
mv-voxl 0.1-r0
qrb5165-bind 0.1-r0
qrb5165-dfs-server 0.1.0-202210072209
qrb5165-imu-server 0.6.0-202303302223
qrb5165-slpi-test-sig 01-r0
qrb5165-system-tweaks 0.1.7-202304211703
qrb5165-tflite 2.8.0-2-202210071637
voxl-camera-server 1.5.4
voxl-ceres-solver 1.14.0-9-202208221901
voxl-configurator 0.2.3-202305261144
voxl-cpu-monitor 0.4.5-202305271751
voxl-eigen3 3.4.0-202210121638
voxl-esc 1.1.1
voxl-jpeg-turbo 2.1.3-4-202305271611
voxl-lepton-server 1.1.0
voxl-libuvc 1.0.7
voxl-mapper 0.1.9-202304271751
voxl-mavcam-manager 0.3.0
voxl-mavlink-server 1.0.5
voxl-modem 1.0.4-202305251139
voxl-mongoose 7.7.0-1-202210072105
voxl-mpa-to-ros 0.3.6-202304141005
voxl-mpa-tools 1.0.1-202305021817
voxl-nlopt 2.5.0-4-202210072358
voxl-opencv 4.5.5-1-202305271610
voxl-portal 0.5.8-202305271939
voxl-qvio-server 0.9.2-202305041224
voxl-slpi-uart-bridge 1.0.1-202304261509
voxl-streamer 0.6.1-202305271548
voxl-tag-detector 0.0.4-202206302112
voxl-tflite-server 0.3.1-202209141826
voxl-utils 1.3.0-202304261309
voxl-uvc-server 0.1.4
voxl-vision-hub 1.6.5
voxl-voxblox 1.1.3-202209261707
voxl2-system-image 1.4.1-r0
voxl2-wlan 1.0-r0

/etc/modalai/voxl-vision-hub.conf:

{
"config_file_version": 1,
"en_localhost_mavlink_udp": false,
"localhost_udp_port_number": 14551,
"en_vio": true,
"vio_pipe": "qvio",
"secondary_vio_pipe": "mvio",
"en_reset_vio_if_initialized_inverted": true,
"vio_warmup_s": 3,
"send_odom_while_failed": false,
"horizon_cal_tolerance": 0.5,
"offboard_mode": "figure_eight",
"follow_tag_id": 0,
"figure_eight_move_home": true,
"robot_radius": 0.300000011920929,
"collision_sampling_dt": 0.1,
"max_lookahead_distance": 1,
"en_tag_fixed_frame": true,
"fixed_frame_filter_len": 5,
"en_transform_mavlink_pos_setpoints_from_fixed_frame": true,
"en_voa": true,
"voa_upper_bound_m": -0.15000000596046448,
"voa_lower_bound_m": 0.15000000596046448,
"voa_voa_memory_s": 1,
"voa_max_pc_per_fusion": 100,
"voa_pie_max_dist_m": 20,
"voa_pie_min_dist_m": 0.25,
"voa_pie_under_trim_m": 1,
"voa_pie_threshold": 3,
"voa_send_rate_hz": 20,
"voa_pie_slices": 36,
"voa_pie_bin_depth_m": 0.15000000596046448,
"voa_inputs": [{
"enabled": true,
"type": "point_cloud",
"input_pipe": "dfs_point_cloud",
"frame": "stereo_l",
"max_depth": 8,
"min_depth": 0.300000011920929,
"cell_size": 0.079999998211860657,
"threshold": 4,
"x_fov_deg": 68,
"y_fov_deg": 56,
"conf_cutoff": 0
}, {
"enabled": true,
"type": "point_cloud",
"input_pipe": "stereo_front_pc",
"frame": "stereo_front_l",
"max_depth": 8,
"min_depth": 0.300000011920929,
"cell_size": 0.079999998211860657,
"threshold": 4,
"x_fov_deg": 68,
"y_fov_deg": 56,
"conf_cutoff": 0
}, {
"enabled": true,
"type": "point_cloud",
"input_pipe": "stereo_rear_pc",
"frame": "stereo_rear_l",
"max_depth": 8,
"min_depth": 0.300000011920929,
"cell_size": 0.079999998211860657,
"threshold": 4,
"x_fov_deg": 68,
"y_fov_deg": 56,
"conf_cutoff": 0
}, {
"enabled": true,
"type": "tof",
"input_pipe": "tof",
"frame": "tof",
"max_depth": 6,
"min_depth": 0.15000000596046448,
"cell_size": 0.079999998211860657,
"threshold": 3,
"x_fov_deg": 106.5,
"y_fov_deg": 85.0999984741211,
"conf_cutoff": 125
}, {
"enabled": true,
"type": "rangefinder",
"input_pipe": "rangefinders",
"frame": "body",
"max_depth": 8,
"min_depth": 0.300000011920929,
"cell_size": 0.079999998211860657,
"threshold": 4,
"x_fov_deg": 68,
"y_fov_deg": 56,
"conf_cutoff": 0
}]
}

/etc/modalai/voxl-tag-detector.conf :

{
"detector_0": {
"enable": true,
"input_pipe": "tracking",
"en_fast_mode": true,
"n_threads": 1,
"en_undistortion": true,
"undistort_scale": 0.600000023841858,
"overlay_name": "tracking_tag_overlay",
"lens_cal_file": "/data/modalai/opencv_tracking_intrinsics.yml",
"skip_n_frames": 5
},
"detector_1": {
"enable": false,
"input_pipe": "stereo",
"en_fast_mode": true,
"n_threads": 1,
"en_undistortion": true,
"undistort_scale": 0.899999976158142,
"overlay_name": "stereo_tag_overlay",
"lens_cal_file": "/data/modalai/opencv_stereo_intrinsics.yml",
"skip_n_frames": 5
},
"detector_2": {
"enable": false,
"input_pipe": "extra",
"en_fast_mode": true,
"n_threads": 1,
"en_undistortion": false,
"undistort_scale": 1,
"overlay_name": "extra_tag_overlay",
"lens_cal_file": "/data/modalai/opencv_extra_intrinsics.yml",
"skip_n_frames": 5
}
}

/etc/modalai/tag_locations.conf:
{
"locations": [{
"id": 0,
"name": "prova0",
"loc_type": "fixed",
"size_m": 0.147,
"T_tag_wrt_fixed": [10, 20, 0],
"R_tag_to_fixed": [[0, -1, 0], [1, 0, 0], [0, 0, 1]]
},
{
"id": 1,
"name": "prova1",
"loc_type": "fixed",
"size_m": 0.147,
"T_tag_wrt_fixed": [5, 1, 0],
"R_tag_to_fixed": [[0, -1, 0], [1, 0, 0], [0, 0, 1]]
}],
"default_size_m": 0.40000000596046448
}

voxl-inspect-services:

Service Name | Enabled | Running | CPU Usage

modallink-relink | Disabled | Not Running |
voxl-camera-server | Enabled | Running | 0.0
voxl-cpu-monitor | Enabled | Running | 0.0
voxl-dfs-server | Disabled | Not Running |
voxl-imu-server | Enabled | Running | 0.0
voxl-lepton-server | Disabled | Not Running |
voxl-mapper | Disabled | Not Running |
voxl-mavcam-manager | Enabled | Running | 0.0
voxl-mavlink-server | Enabled | Running | 0.0
voxl-modem | Disabled | Not Running |
voxl-portal | Enabled | Running | 0.0
voxl-qvio-server | Enabled | Running | 0.0
voxl-softap | Disabled | Not Running |
voxl-static-ip | Disabled | Not Running |
voxl-streamer | Enabled | Running | 0.0
voxl-tag-detector | Enabled | Running | 0.0
voxl-tflite-server | Disabled | Not Running |
voxl-time-sync | Disabled | Not Running |
voxl-uvc-server | Disabled | Not Running |
voxl-vision-hub | Enabled | Running | 0.0
voxl-wait-for-fs | Enabled | Completed |

The same happens when running the script on another board with another version reported here below. Other parameters are the same.

voxl-version

system-image: 1.4.1-M0054-14.1a-perf
kernel: #1 SMP PREEMPT Thu Oct 13 17:48:16 UTC 2022 4.19.125

hw version: M0054

voxl-suite: 0.9.5

Packages:
Repo: http://voxl-packages.modalai.com/ ./dists/qrb5165/sdk-0.9/binary-arm64/
Last Updated: 2023-05-26 15:27:09
List:
libmodal-cv 0.2.3
libmodal-exposure 0.0.7
libmodal-journal 0.2.1
libmodal-json 0.4.3
libmodal-pipe 2.8.2
libqrb5165-io 0.1.0
libvoxl-cci-direct 0.1.5
libvoxl-cutils 0.1.1
mv-voxl 0.1-r0
qrb5165-bind 0.1-r0
qrb5165-dfs-server 0.1.0
qrb5165-imu-server 0.5.0
qrb5165-slpi-test-sig 01-r0
qrb5165-system-tweaks 0.1.5
qrb5165-tflite 2.8.0-2
voxl-bind 0.0.1
voxl-camera-calibration 0.2.3
voxl-camera-server 1.3.5
voxl-cpu-monitor 0.3.0
voxl-docker-support 1.2.4
voxl-gphoto2-server 0.0.10
voxl-jpeg-turbo 2.1.3-4
voxl-libgphoto2 0.0.4
voxl-libuvc 1.0.7
voxl-logger 0.3.4
voxl-mavlink 0.1.0
voxl-mavlink-server 0.3.0
voxl-modem 0.16.1
voxl-mongoose 7.7.0-1
voxl-mpa-to-ros 0.3.6
voxl-mpa-tools 0.7.6
voxl-opencv 4.5.5-1
voxl-portal 0.5.0
voxl-px4 1.12.31
voxl-px4-imu-server 0.1.2
voxl-qvio-server 0.8.2
voxl-remote-id 0.0.5
voxl-streamer 0.4.1
voxl-suite 0.9.5
voxl-tag-detector 0.0.4
voxl-tflite-server 0.3.1
voxl-utils 1.2.2
voxl-uvc-server 0.1.3
voxl-vision-px4 1.4.0
voxl2-system-image 1.4.1-r0
voxl2-wlan 1.0-r0