Posts made by aashry

aashry

aashry

Hello!

I have some question regarding the onboard state estimation running onboard, and specifically for the interface between voxl-vision-px4 and voxl-qvio

As I understand, the voxl-qvio-server utilizes the Qualcomm algorithm, processing input from the tracking camera and IMU data to output QVIO data. This data is then consumed by voxl-vision-px4. My confusion arises at this stage: does voxl-vision-px4 pass this VIO data as a state estimate directly to PX4, bypassing PX4's internal EKF2 processing? In other words, what exactly is running on the PX4 in this setup?

I'm aware that the EKF2 algorithm is typically used on PX4 for state estimation by fusing data from various sensors. How does this interact with the VIO data from the qvio server? Is the EKF algorithm on PX4 still active in this configuration, and if so, is it using the processed data from the qvio server to refine the state estimates?

This leads to another question about the distinction between the outputs of the qvio server and the EKF algorithm on PX4. Are they operating independently, each producing separate state estimates based on common inputs (IMU and tracking camera data), or is there a fusion of data from both for enhanced accuracy in state estimation?

Finally, I'm trying to pinpoint the source of the pose data when interfacing Mavros with voxl-vision-px4. Specifically mavros/local_position/pose output, which algorithm (qvio or EKF2 on PX4) is providing this state estimate?

I apologize if my questions seem a bit convoluted; I'm just trying to get a clear understanding of how these systems work together for state estimation. Any insights or clarifications would be greatly appreciated!

Thanks in advance for your help!

aashry

Hello!

Is there any workaround that I can do to get vscode remote ssh to work again with voxl? It was working fine a couple of months ago, but now it seems like it's not working anymore. I checked from here and it seems that the current SDK running on my voxl runs on an older Kernel which is not supported anymore for vscode ssh remote connection? I suspect it's due to the VOXL running on an outdated kernel. I'm guessing that because the error that it throws away while trying to connect says:

The remote host may not meet VS Code Server's prerequisites for glibc and libstdc++

/home/root/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/bin/helpers/check-requirements.sh: line 22: getconf: not found

Did anyone face a similar issue and manage to solve it?

Below is the output of voxl-version command on my current setup.

--------------------------------------------------------------------------------
system-image: 4.0.0
kernel:       #1 SMP PREEMPT Thu Oct 13 17:57:05 UTC 2022 3.18.71-perf
--------------------------------------------------------------------------------
hw version:   VOXL
--------------------------------------------------------------------------------
voxl-suite:   0.9.4
--------------------------------------------------------------------------------
Packages:
Repo: http://voxl-packages.modalai.com/dists/apq8096/sdk-0.9/binary-arm64/
Last Updated: 1970-01-01 00:05:23
List:
	apq8096-dfs-server       0.3.1
	apq8096-imu-server       1.0.3
	apq8096-libpng           1.6.38-1
	apq8096-system-tweaks    0.1.3
	apq8096-tflite           2.8.3-1
	libapq8096-io            0.6.0
	libmodal-cv              0.2.3
	libmodal-exposure        0.0.7
	libmodal-journal         0.2.1
	libmodal-json            0.4.3
	libmodal-pipe            2.6.0
	libvoxl-cci-direct       0.1.5
	libvoxl-cutils           0.1.1
	voxl-camera-calibration  0.2.3
	voxl-camera-server       1.3.5
	voxl-ceres-solver        1.14.0-9
	voxl-cpu-monitor         0.3.0
	voxl-docker-support      1.2.4
	voxl-eigen3              3.4.0
	voxl-gphoto2-server      0.0.10
	voxl-jpeg-turbo          2.1.3-4
	voxl-libgphoto2          0.0.4
	voxl-libuvc              1.0.7
	voxl-logger              0.3.4
	voxl-mapper              0.1.9
	voxl-mavlink             0.1.0
	voxl-mavlink-server      0.2.0
	voxl-modem               0.16.1
	voxl-mongoose            7.7.0-1
	voxl-mpa-to-ros          0.3.6
	voxl-mpa-tools           0.7.6
	voxl-nlopt               2.5.0-4
	voxl-opencv              4.5.5-1
	voxl-portal              0.4.2
	voxl-qvio-server         0.8.2
	voxl-streamer            0.4.1
	voxl-suite               0.9.4
	voxl-system-image        4.0-r0
	voxl-tag-detector        0.0.4
	voxl-tflite-server       0.3.1
	voxl-utils               1.2.2
	voxl-uvc-server          0.1.3
	voxl-vision-px4          1.4.0
	voxl-voxblox             1.1.3
	voxl-vpn                 0.0.6
--------------------------------------------------------------------------------

I think an SDK upgrade might be the solution, but I'm looking for a quick fix to tide me over until my current project is completed.

Thanks in advance for your help.

aashry

Anyone can help?

aashry

@aashry

Edit1: Everything above applies to the h264 encoding as well, I haven't been able to view it in the portal and it has the same problem with the ros node.

aashry

Also, it seems like voxl-camera-server randomly stops working. Is SDK 1.0 completely unusable so far? or Is there a workaround that we can do?

|           Pipe Name |  bytes  | wide |  hgt |exp(ms)| gain | frame id |latency(ms)|  fps |  mbps  t
|   hires_large_color | Server Disconnected
| hires_large_encoded | Server DisconnectedMost likely the server stopped without cleaning up
Client is cleaning up pipes for the serverdMost likely the server stopped without cleaning up
ERROR in pipe_client_init_channel opening request pipe: No such device or addressing request pipe: Ns
| hires_small_encoded | Server DisconnectedERROR in pipe_client_init_channel opening request pipe: Ns
|    hires_small_grey | Server DisconnectedClient is cleaning up pipes for the server
ERROR in pipe_client_init_channel opening request pipe: No such device or address cleaning up
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the serverdMost likely the server stopped without cleaning up
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning upn pipe_client_init_channel opening request pipe: Ns
Client is cleaning up pipes for the serveraning up
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the server
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the server
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the server
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the server

aashry

@aashry the following is the voxl-camera-server.conf file as well for your reference. Should I change anything here to fix that issue? Please elaborate.

/**
 * voxl-camera-server Configuration File
 *
 * Each camera has configurations for up to 4 HAL3 streams:
 *    - `preview` stream for raw unprocessed images from CV cameras
 *    - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming
 *    - `large_video` 4k (ish) h264/h265 for onboard video recording to disk
 *    - `snapshot` ISP-processed JPG snapshots that get saved to disk
 *
 * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled
 *
 * This file is generated from default values by voxl-configure-cameras.
 * Do not expect arbitrary resolutions to work, the ISP and video compression
 * pipelines only support very specific resolutions.
 *
 * The default video compression mode is cqp or Constant Quantization Parameter
 *
 *
 *
 */
{
	"version":	0.1,
	"cameras":	[{
			"type":	"pmd-tof",
			"name":	"tof",
			"enabled":	true,
			"camera_id":	0,
			"fps":	5,
			"en_preview":	true,
			"preview_width":	224,
			"preview_height":	1557,
			"pre_format":	"tof",
			"ae_mode":	"off",
			"standby_enabled":	false,
			"decimator":	5
		}, {
			"type":	"imx214",
			"name":	"hires",
			"enabled":	true,
			"camera_id":	2,
			"fps":	30,
			"en_preview":	false,
			"preview_width":	640,
			"preview_height":	480,
			"pre_format":	"nv21",
			"en_small_video":	true,
			"small_video_width":	1024,
			"small_video_height":	768,
			"small_venc_mode":	"h265",
			"small_venc_br_ctrl":	"cqp",
			"small_venc_Qfixed":	30,
			"small_venc_Qmin":	15,
			"small_venc_Qmax":	40,
			"small_venc_nPframes":	9,
			"small_venc_mbps":	2,
			"en_large_video":	true,
			"large_video_width":	4096,
			"large_video_height":	2160,
			"large_venc_mode":	"h265",
			"large_venc_br_ctrl":	"cqp",
			"large_venc_Qfixed":	38,
			"large_venc_Qmin":	15,
			"large_venc_Qmax":	50,
			"large_venc_nPframes":	29,
			"large_venc_mbps":	30,
			"en_snapshot":	true,
			"en_snapshot_width":	4160,
			"en_snapshot_height":	3120,
			"ae_mode":	"isp"
		}, {
			"type":	"ov7251",
			"name":	"tracking",
			"enabled":	true,
			"camera_id":	1,
			"fps":	30,
			"en_preview":	true,
			"preview_width":	640,
			"preview_height":	480,
			"pre_format":	"raw8",
			"ae_mode":	"lme_msv",
			"ae_desired_msv":	60,
			"ae_filter_alpha":	0.600000023841858,
			"ae_ignore_fraction":	0.20000000298023224,
			"ae_slope":	0.05000000074505806,
			"ae_exposure_period":	1,
			"ae_gain_period":	1
		}]
}

aashry

Hello Team,
I'm trying to get encoded hires images using sdk 1.0 and camera_server v1.6.2 from a voxl2 board with configuration 6 (TOF, HIRES and Tracking). The following is the output from voxl-version

voxl2:~$ voxl-version
--------------------------------------------------------------------------------
system-image: 1.6.2-M0054-14.1a-perf
kernel:       #1 SMP PREEMPT Fri May 19 22:19:33 UTC 2023 4.19.125
--------------------------------------------------------------------------------
hw version:   M0054
--------------------------------------------------------------------------------
voxl-suite:   1.0.0
--------------------------------------------------------------------------------
Packages:
Repo:  http://voxl-packages.modalai.com/ ./dists/qrb5165/sdk-1.0/binary-arm64/
Last Updated: 2023-11-12 06:55:14
List:
	libmodal-cv              0.3.2
	libmodal-exposure        0.0.8
	libmodal-journal         0.2.2
	libmodal-json            0.4.3
	libmodal-pipe            2.8.5
	libqrb5165-io            0.2.0
	libvoxl-cci-direct       0.1.5
	libvoxl-cutils           0.1.1
	mv-voxl                  0.1-r0
	qrb5165-bind             0.1-r0
	qrb5165-dfs-server       0.1.0
	qrb5165-imu-server       0.6.0
	qrb5165-slpi-test-sig    01-r0
	qrb5165-system-tweaks    0.2.2
	qrb5165-tflite           2.8.0-2
	voxl-bind-spektrum       0.1.0
	voxl-camera-calibration  0.4.0
	voxl-camera-server       1.6.2
	voxl-configurator        0.2.7
	voxl-cpu-monitor         0.4.6
	voxl-docker-support      1.2.5
	voxl-eigen3              3.4.0
	voxl-elrs                0.0.7
	voxl-esc                 1.2.2
	voxl-feature-tracker     0.2.3
	voxl-flow-server         0.3.3
	voxl-gphoto2-server      0.0.10
	voxl-jpeg-turbo          2.1.3
	voxl-lepton-server       1.1.2
	voxl-libgphoto2          0.0.4
	voxl-libuvc              1.0.7
	voxl-logger              0.3.4
	voxl-mavcam-manager      0.5.1
	voxl-mavlink             0.1.1
	voxl-mavlink-server      1.2.0
	voxl-modem               1.0.5
	voxl-mongoose            7.7.0-1
	voxl-mpa-to-ros          0.3.6
	voxl-mpa-tools           1.0.4
	voxl-opencv              4.5.5-1
	voxl-portal              0.5.9
	voxl-px4                 1.14.0-2.0.34
	voxl-px4-imu-server      0.1.2
	voxl-px4-params          0.1.8
	voxl-qvio-server         0.9.3
	voxl-remote-id           0.0.8
	voxl-slpi-uart-bridge    1.0.1
	voxl-streamer            0.7.1
	voxl-suite               1.0.0
	voxl-tag-detector        0.0.4
	voxl-tflite-server       0.3.1
	voxl-utils               1.3.1
	voxl-uvc-server          0.1.6
	voxl-vision-hub          1.6.6
	voxl2-system-image       1.6.2-r0
	voxl2-wlan               1.0-r0
--------------------------------------------------------------------------------

I was able to get the camera-server to work with configuration 6 and the pipes are showing up in voxl-inspect-cam -a as shown below

|           Pipe Name |  bytes  | wide |  hgt |exp(ms)| gain | frame id |latency(ms)|  fps |  mbps  | format
|   hires_large_color |13271040 | 4096 | 2160 | 33.23 |  431 |      489 |     77.1  | 30.0 | 3190.0 | NV12
| hires_large_encoded |   26549 | 4096 | 2160 | 33.23 |  431 |      489 |     89.3  | 30.0 |    7.2 | H265 (P)   
|    hires_large_grey | 8847360 | 4096 | 2160 | 33.23 |  431 |      489 |     71.8  | 30.0 | 2126.6 | RAW8
|   hires_small_color | 1179648 | 1024 |  768 | 33.23 |  431 |      490 |     60.6  | 30.0 |  283.5 | NV12
| hires_small_encoded |    1528 | 1024 |  768 | 33.23 |  431 |      490 |     63.0  | 30.0 |    0.5 | H265 (P)   
|    hires_small_grey |  786432 | 1024 |  768 | 33.23 |  431 |      490 |     59.4  | 30.0 |  189.0 | RAW8
|      hires_snapshot |
|            tof_conf |   38528 |  224 |  172 |  0.00 |    0 |     1669 |     29.6  |  5.0 |    1.5 | RAW8
|           tof_depth |   38528 |  224 |  172 |  0.00 |    0 |     1669 |     29.3  |  5.0 |    1.5 | RAW8
|              tof_ir |   38528 |  224 |  172 |  0.00 |    0 |     1669 |     29.0  |  5.0 |    1.5 | RAW8
|            tracking |  307200 |  640 |  480 |  5.61 |  303 |    10016 |     11.6  | 30.0 |   73.7 | RAW8

However, I'm not able to see the output of the encoded hires neither in voxl-portal not when I subscribe to the encoded pipeline in the published topics from voxl-mpa-to-ros. In fact, when I try to subscribe to any encoded image topic, the node crashes immediately as shown below.

voxl2:~$ roslaunch voxl_mpa_to_ros voxl_mpa_to_ros.launch 
... logging to /home/root/.ros/log/c26582ac-8191-11ee-99cc-9c5322cc8a09/roslaunch-m0054-8630.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://m0054:43361/

SUMMARY
========

PARAMETERS
 * /rosdistro: melodic
 * /rosversion: 1.14.13

NODES
  /
    voxl_mpa_to_ros_node (voxl_mpa_to_ros/voxl_mpa_to_ros_node)

auto-starting new master
process[master]: started with pid [8644]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to c26582ac-8191-11ee-99cc-9c5322cc8a09
process[rosout-1]: started with pid [8655]
started core service [/rosout]
process[voxl_mpa_to_ros_node-2]: started with pid [8658]


MPA to ROS app is now running

Found new interface: hires_large_color
Found new interface: hires_large_encoded
Found new interface: hires_large_grey
Found new interface: hires_small_color
Found new interface: hires_small_encoded
Found new interface: hires_small_grey
Found new interface: hires_snapshot
Found new interface: tof_conf
Found new interface: tof_depth
Found new interface: tof_ir
Found new interface: tracking
Found new interface: imu_apps
Found new interface: tof_pc
Found new interface: voa_pc_out
Interface tof_depth now publishing
Interface hires_small_grey now publishing
Interface tof_depth ceasing to publish
Interface hires_large_encoded now publishing
Interface hires_small_grey ceasing to publish
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
[voxl_mpa_to_ros_node-2] process has died [pid 8658, exit code -6, cmd /opt/ros/melodic/lib/voxl_mpa_to_ros/voxl_mpa_to_ros_node __name:=voxl_mpa_to_ros_node __log:=/home/root/.ros/log/c26582ac-8191-11ee-99cc-9c5322cc8a09/voxl_mpa_to_ros_node-2.log].
log file: /home/root/.ros/log/c26582ac-8191-11ee-99cc-9c5322cc8a09/voxl_mpa_to_ros_node-2*.log

Is there any possible solution for this issue? Please help!

aashry

Hello @hmlow, have you been able to solve this issue? or are you still facing similar issues with the camera-server with sdk 1.0?

aashry

@Jetson-Nano hello!
which tutorial are you referring to?

aashry

@tpavlic Thank you, glad it helped!

aashry

@tom sorry for the late reply, I've been distant from work for one week. I'm using m500 drone with the voxl flight deck. Below is the voxl-version output as you requested. I will probably try to follow what @jonathankampia mentioned and see if it works. I will update the thread here accordingly. Thank you guys for your help, appreciate it.

--------------------------------------------------------------------------------
system-image: 4.0.0
kernel:       #1 SMP PREEMPT Thu Oct 13 17:57:05 UTC 2022 3.18.71-perf
--------------------------------------------------------------------------------
hw version:   VOXL
--------------------------------------------------------------------------------
voxl-suite:   0.9.4
--------------------------------------------------------------------------------
Packages:
Repo: http://voxl-packages.modalai.com/dists/apq8096/sdk-0.9/binary-arm64/
Last Updated: 1970-01-01 00:05:23
List:
	apq8096-dfs-server       0.3.1
	apq8096-imu-server       1.0.3
	apq8096-libpng           1.6.38-1
	apq8096-system-tweaks    0.1.3
	apq8096-tflite           2.8.3-1
	libapq8096-io            0.6.0
	libmodal-cv              0.2.3
	libmodal-exposure        0.0.7
	libmodal-journal         0.2.1
	libmodal-json            0.4.3
	libmodal-pipe            2.6.0
	libvoxl-cci-direct       0.1.5
	libvoxl-cutils           0.1.1
	voxl-camera-calibration  0.2.3
	voxl-camera-server       1.3.5
	voxl-cpu-monitor         0.3.0
	voxl-docker-support      1.2.4
	voxl-gphoto2-server      0.0.10
	voxl-jpeg-turbo          2.1.3-4
	voxl-libgphoto2          0.0.4
	voxl-libuvc              1.0.7
	voxl-logger              0.3.4
	voxl-mavlink             0.1.0
	voxl-mavlink-server      0.2.0
	voxl-modem               0.16.1
	voxl-mongoose            7.7.0-1
	voxl-mpa-to-ros          0.3.6
	voxl-mpa-tools           0.7.6
	voxl-opencv              4.5.5-1
	voxl-portal              0.4.2
	voxl-qvio-server         0.8.2
	voxl-streamer            0.4.1
	voxl-suite               0.9.4
	voxl-system-image        4.0-r0
	voxl-tag-detector        0.0.4
	voxl-tflite-server       0.3.1
	voxl-utils               1.2.2
	voxl-uvc-server          0.1.3
	voxl-vision-px4          1.4.0
	voxl-vpn                 0.0.6
--------------------------------------------------------------------------------

aashry

@tom can you help me with that, or do you know someone who can help? Thank you so much in advance!

aashry

@aashry If anyone has any insights or suggestions on how to solve this, please do share. Your help would be much appreciated.

aashry

@SHofmeester I think you can choose the default configuration for your hardware from voxl-configure-cameras , and then you can turn off the cameras you don't want to use by modifying the config file directly at etc/modalai/voxl-camera-server.conf , and change the flags for the cameras you don't want to use to be false instead of true.