@Alex-Kushleyev
I have tested different layouts with different cameras disabled and it didn't work either.
for some reason, the voxl won't accept any other frame rate in the config file other than 30.
Latest posts made by giladWDS
-
RE: cam does not seem to be alive issue
-
RE: cam does not seem to be alive issue
@Alex-Kushleyev
Hi, after I had to work on other projects I have finally come back to this and I managed to make the voxl read all cameras when I moved the tracking camera to j8-lower.
now the issue I'm facing is with a frame rate reduction.
it seems that whenever I try to change the frame rate of one of the high-res cameras the server crashes.
this is the server log:Oct 27 12:35:13 root systemd[1]: Started voxl-camera-server. Oct 27 12:35:13 root voxl-camera-server[14170]: detected system image 1.7.6 Oct 27 12:35:13 root voxl-camera-server[14170]: using new imx412 defaults Oct 27 12:35:13 root voxl-camera-server[14170]: detected system image 1.7.6 Oct 27 12:35:14 root voxl-camera-server[14170]: using new imx412 defaults Oct 27 12:35:14 root voxl-camera-server[14170]: ================================================================= Oct 27 12:35:14 root voxl-camera-server[14170]: configuration for 3 cameras: Oct 27 12:35:14 root voxl-camera-server[14170]: cam #0 Oct 27 12:35:14 root voxl-camera-server[14170]: name: hires2 Oct 27 12:35:14 root voxl-camera-server[14170]: sensor type: imx412 Oct 27 12:35:14 root voxl-camera-server[14170]: isEnabled: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: camId: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: camId2: -1 Oct 27 12:35:14 root voxl-camera-server[14170]: fps: 10 Oct 27 12:35:14 root voxl-camera-server[14170]: en_rotate: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: en_rotate2: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: en_preview: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: pre_width: 4056 Oct 27 12:35:14 root voxl-camera-server[14170]: pre_height: 3040 Oct 27 12:35:14 root voxl-camera-server[14170]: en_raw_preview: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: en_small_video: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: small_video_width: 1024 Oct 27 12:35:14 root voxl-camera-server[14170]: small_video_height: 768 Oct 27 12:35:14 root voxl-camera-server[14170]: en_large_video: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: large_video_width: 4056 Oct 27 12:35:14 root voxl-camera-server[14170]: large_video_height: 3040 Oct 27 12:35:14 root voxl-camera-server[14170]: en_snapshot: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: snap_width: 4056 Oct 27 12:35:14 root voxl-camera-server[14170]: snap_height: 3040 Oct 27 12:35:14 root voxl-camera-server[14170]: ae_mode: isp Oct 27 12:35:14 root voxl-camera-server[14170]: standby_enabled: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: decimator: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: independent_exposure:0 Oct 27 12:35:14 root voxl-camera-server[14170]: cam #1 Oct 27 12:35:14 root voxl-camera-server[14170]: name: hires Oct 27 12:35:14 root voxl-camera-server[14170]: sensor type: imx412 Oct 27 12:35:14 root voxl-camera-server[14170]: isEnabled: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: camId: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: camId2: -1 Oct 27 12:35:14 root voxl-camera-server[14170]: fps: 10 Oct 27 12:35:14 root voxl-camera-server[14170]: en_rotate: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: en_rotate2: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: en_preview: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: pre_width: 4056 Oct 27 12:35:14 root voxl-camera-server[14170]: pre_height: 3040 Oct 27 12:35:14 root voxl-camera-server[14170]: en_raw_preview: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: en_small_video: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: small_video_width: 1024 Oct 27 12:35:14 root voxl-camera-server[14170]: small_video_height: 768 Oct 27 12:35:14 root voxl-camera-server[14170]: en_large_video: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: large_video_width: 4056 Oct 27 12:35:14 root voxl-camera-server[14170]: large_video_height: 3040 Oct 27 12:35:14 root voxl-camera-server[14170]: en_snapshot: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: snap_width: 4056 Oct 27 12:35:14 root voxl-camera-server[14170]: snap_height: 3040 Oct 27 12:35:14 root voxl-camera-server[14170]: ae_mode: isp Oct 27 12:35:14 root voxl-camera-server[14170]: standby_enabled: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: decimator: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: independent_exposure:0 Oct 27 12:35:14 root voxl-camera-server[14170]: cam #2 Oct 27 12:35:14 root voxl-camera-server[14170]: name: tracking Oct 27 12:35:14 root voxl-camera-server[14170]: sensor type: ar0144 Oct 27 12:35:14 root voxl-camera-server[14170]: isEnabled: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: camId: 2 Oct 27 12:35:14 root voxl-camera-server[14170]: camId2: -1 Oct 27 12:35:14 root voxl-camera-server[14170]: fps: 30 Oct 27 12:35:14 root voxl-camera-server[14170]: en_rotate: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: en_rotate2: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: en_preview: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: pre_width: 1280 Oct 27 12:35:14 root voxl-camera-server[14170]: pre_height: 800 Oct 27 12:35:14 root voxl-camera-server[14170]: en_raw_preview: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: en_small_video: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: small_video_width: -1 Oct 27 12:35:14 root voxl-camera-server[14170]: small_video_height: -1 Oct 27 12:35:14 root voxl-camera-server[14170]: en_large_video: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: large_video_width: -1 Oct 27 12:35:14 root voxl-camera-server[14170]: large_video_height: -1 Oct 27 12:35:14 root voxl-camera-server[14170]: en_snapshot: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: snap_width: -1 Oct 27 12:35:14 root voxl-camera-server[14170]: snap_height: -1 Oct 27 12:35:14 root voxl-camera-server[14170]: ae_mode: lme_msv Oct 27 12:35:14 root voxl-camera-server[14170]: standby_enabled: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: decimator: 1 Oct 27 12:35:14 root voxl-camera-server[14170]: independent_exposure:0 Oct 27 12:35:14 root voxl-camera-server[14170]: fsync_en: 0 Oct 27 12:35:14 root voxl-camera-server[14170]: fsync_gpio: 111 Oct 27 12:35:14 root voxl-camera-server[14170]: ================================================================= Oct 27 12:35:14 root voxl-camera-server[14170]: connected to GPS mavlink pipe Oct 27 12:35:14 root voxl-camera-server[14170]: Starting Camera: hires2 (id #0) Oct 27 12:35:14 root voxl-camera-server[14170]: Connected to cpu-monitor Oct 27 12:35:14 root voxl-camera-server[14170]: Using gain limits min: 100 max: 1600 Oct 27 12:35:14 root voxl-camera-server[14170]: gbm_create_device(156): Info: backend name is: msm_drm Oct 27 12:35:14 root voxl-camera-server[14170]: Starting Camera: hires (id #1) Oct 27 12:35:14 root voxl-camera-server[14170]: Using gain limits min: 100 max: 1600 Oct 27 12:35:14 root voxl-camera-server[14170]: ERROR: Received "Result" error from camera: hires2 Oct 27 12:35:14 root systemd[1]: voxl-camera-server.service: Main process exited, code=killed, status=6/ABRT Oct 27 12:35:14 root systemd[1]: voxl-camera-server.service: Failed with result 'signal'. Oct 27 12:35:16 root systemd[1]: voxl-camera-server.service: Service hold-off time over, scheduling restart. Oct 27 12:35:16 root systemd[1]: voxl-camera-server.service: Scheduled restart job, restart counter is at 1.
and this is the config file:
cat /etc/modalai/voxl-camera-server.conf /** * voxl-camera-server Configuration File * * Each camera has configurations for up to 4 HAL3 streams: * - `preview` stream for raw unprocessed images from CV cameras * - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming * - `large_video` 4k (ish) h264/h265 for onboard video recording to disk * - `snapshot` ISP-processed JPG snapshots that get saved to disk * * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled * * This file is generated from default values by voxl-configure-cameras. * Do not expect arbitrary resolutions to work, the ISP and video compression * pipelines only support very specific resolutions. * * The default video compression mode is cqp or Constant Quantization Parameter * * * */ { "version": 0.1, "fsync_en": false, "fsync_gpio": 111, "cameras": [{ "type": "imx412", "name": "hires2", "enabled": true, "camera_id": 0, "fps": 10, "en_preview": true, "preview_width": 4056, "preview_height": 3040, "en_raw_preview": false, "en_small_video": false, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h265", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "en_large_video": false, "large_video_width": 4056, "large_video_height": 3040, "large_venc_mode": "h265", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 38, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 30, "en_snapshot": false, "en_snapshot_width": 4056, "en_snapshot_height": 3040, "ae_mode": "isp" }, { "type": "imx412", "name": "hires", "enabled": true, "camera_id": 1, "fps": 10, "en_preview": true, "preview_width": 4056, "preview_height": 3040, "en_raw_preview": false, "en_small_video": false, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h265", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "en_large_video": false, "large_video_width": 4056, "large_video_height": 3040, "large_venc_mode": "h265", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 38, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 30, "en_snapshot": false, "en_snapshot_width": 4056, "en_snapshot_height": 3040, "ae_mode": "isp" }, { "type": "ar0144", "name": "tracking", "enabled": true, "camera_id": 2, "fps": 30, "en_rotate": false, "en_preview": true, "preview_width": 1280, "preview_height": 800, "en_raw_preview": true, "ae_mode": "lme_msv", "ae_desired_msv": 60, "exposure_min_us": 20, "exposure_max_us": 33000, "gain_min": 54, "gain_max": 8000, "exposure_soft_min_us": 5000, "ae_filter_alpha": 0.600000023841858, "ae_ignore_fraction": 0.20000000298023224, "ae_slope": 0.05000000074505806, "ae_exposure_period": 1, "ae_gain_period": 1 }] }
And when I change it back to 30 fps it works. the problem is the CPU toll 30 takes.
-
RE: CP with custom sensors
@James-Strawson
Thank you for the response.
we want to use an array of 4 vl53l5cx sensors to capture data around the drone. I know that there is existing support for vl53l1cx sensors. since it is crucial to know of possible obstacles from multiple angles we need the 4x4 version.
in this case, I have a point cloud of 64 points around the drone. so should I send those on the pointcloud pipe?for the few services:
this is the service inspect output:Service Name | Enabled | Running | CPU Usage ------------------------------------------------------------------- docker-autorun | Enabled | Not Running | modallink-relink | Disabled | Not Running | voxl-camera-server | Enabled | Running | 2.2% voxl-cpu-monitor | Enabled | Running | 0.6% voxl-dfs-server | Disabled | Not Running | voxl-feature-tracker | Disabled | Not Running | voxl-flow-server | Disabled | Not Running | voxl-imu-server | Enabled | Running | 0.1% voxl-lepton-server | Disabled | Not Running | voxl-mavcam-manager | Disabled | Not Running | voxl-mavlink-server | Enabled | Running | 2.9% voxl-modem | Disabled | Not Running | voxl-neopixel-manager | Disabled | Not Running | voxl-open-vins-server | Disabled | Not Running | voxl-portal | Enabled | Running | 0.1% voxl-px4-imu-server | Disabled | Not Running | voxl-px4 | Enabled | Running | 30.5% voxl-qvio-server | Disabled | Not Running | voxl-rangefinder-server | Disabled | Not Running | voxl-remote-id | Disabled | Not Running | voxl-softap | Disabled | Not Running | voxl-static-ip | Disabled | Not Running | voxl-streamer | Disabled | Not Running | voxl-tag-detector | Disabled | Not Running | voxl-tflite-server | Disabled | Not Running | voxl-time-sync | Disabled | Not Running | voxl-uvc-server | Disabled | Not Running | voxl-vision-hub | Enabled | Running | 1.7% voxl-wait-for-fs | Enabled | Completed |
-
CP with custom sensors
Hi,
I want to integrate collision prevention into my system.
currently, I have a microcontroller that manages all the sensor data and is connected to the voxl2 through serial.
now I debate two ways to implement this and I don't know that is the best approach.- using mavlink directly to px4. I saw that the px4 exposes a UDP port to send mavlink msgs directly to px4. this feels sub-optimal as this bypasses the integrated VOA system and the voxel-vision-hub.
- make a pipe client and transform the data from the serial port to some kind of TOF sensor and publish it.
Is the second option the way to go or is the first option viable as well?
Furthermore, I don't see all the voxl-vision-hub pipes:
voxl2:~$ voxl-list-pipes -t camera_image_metadata_t hires_large_color hires_large_encoded hires_large_grey hires_small_color hires_small_encoded hires_small_grey hires_snapshot cpu_stats_t cpu_monitor imu_data_t imu_apps imu_fft_data_t imu_apps_fft mavlink_message_t mavlink_ap_heartbeat mavlink_attitude mavlink_from_gcs mavlink_gps_raw_int mavlink_local_position_ned mavlink_onboard mavlink_sys_status mavlink_to_gcs point_cloud_metadata_t voa_pc_out pose_vel_6dof_t vvhub_body_wrt_fixed vvhub_body_wrt_local string gcs_ip_list text horizon_cal_io
Is this presenting all server pipes that have clients?
-
RE: custom uart usage
@Eric-Katzfey
Thank you for the reply
The only board I have no for testing is:
MCCA-M0151-1
I use the USB portion for the wifi and see that it exposes a uart port- HS1 for the external USB board.
I can't seem to communicate with this HS0 port as a standalone port.
Is it only for external Flight controllers?I looked into this:
https://docs.modalai.com/usb2-type-a-breakout-add-on/
but it seems to expose the same uart I can not talk to.
Can you please recommend me a board that won't affect the wifi capability but would add uart ports that I desperately need?- EDIT:
also if push comes to shove I would appreciate sources on how to write and compile a px4 driver that reads from the uart on j19.
Thank you and best regards,
Gilad - EDIT:
-
RE: custom uart usage
@giladWDS To specify. my sensor is connected to a microcontroller. so also I prefer UART I can do i2c(I have tried and it didn't work as well) and I can do SPI if it works better.
-
custom uart usage
Hi,
I wanted to add a custom sensor to my voxel-2 through j19 UART.
now I see that this UART is reserved to RC-inputs. but even when I disabled those in /etc/modalai/voxl-px4.conf, I still can't access the port.
I looked a bit and found this:
https://gitlab.com/voxl-public/voxl-sdk/core-libs/libqrb5165-io/-/tree/master/python?ref_type=heads
and tried using it with port 7.
when I tried to write it gave me the following:
ERROR: voxl_uart_write: Bus '17' is not initialized
and on init, it gave:
Sending library name request: libslpi_qrb5165_io.so
Sending initialization request
Received standard error event SNS_STD_ERROR_NOT_SUPPORTED
Couldn't configure flight_controller sensor
ERROR: fc_sensor_initialize failed
ERROR: Failed to initialize slpi
ERROR: Encountered error while initializing bus 17so it looks like it is still looking for some RC sensor.
I guess my real question is how can I access SLPI UART as a user since I don't see them in /dev/tty*
Thank you -
RE: cam does not seem to be alive issue
@Alex-Kushleyev
Thank you for the help, I have tried a different cable and it didn't work either, plus this flex is working when I am using it on a different voxl2 -
RE: cam does not seem to be alive issue
EDIT:
If it helps this is my voxl-version-------------------------------------------------------------------------------- system-image: 1.7.6-M0054-14.1a-perf kernel: #1 SMP PREEMPT Wed Mar 20 19:48:52 UTC 2024 4.19.125 -------------------------------------------------------------------------------- hw platform: M0054 mach.var: 1.0 -------------------------------------------------------------------------------- voxl-suite: 1.2.0 -------------------------------------------------------------------------------- Packages: Repo: http://voxl-packages.modalai.com/ ./dists/qrb5165/sdk-1.2/binary-arm64/ Last Updated: 2023-03-02 13:19:12 List: kernel-module-voxl-fsync-mod-4.19.125 1.0-r0 kernel-module-voxl-gpio-mod-4.19.125 1.0-r0 kernel-module-voxl-platform-mod-4.19.125 1.0-r0 libfc-sensor 1.0.5 libmodal-cv 0.4.0 libmodal-exposure 0.1.0 libmodal-journal 0.2.2 libmodal-json 0.4.3 libmodal-pipe 2.10.0 libqrb5165-io 0.4.5 libvoxl-cci-direct 0.2.1 libvoxl-cutils 0.1.1 modalai-slpi 1.1.12 mv-voxl 0.1-r0 qrb5165-bind 0.1-r0 qrb5165-dfs-server 0.2.0 qrb5165-imu-server 1.0.1 qrb5165-rangefinder-server 0.1.1 qrb5165-slpi-test-sig 01-r0 qrb5165-system-tweaks 0.2.6 qrb5165-tflite 2.8.0-2 voxl-bind-spektrum 0.1.1 voxl-camera-calibration 0.5.4 voxl-camera-server 1.9.1 voxl-ceres-solver 2:1.14.0-10 voxl-configurator 0.5.6 voxl-cpu-monitor 0.4.8 voxl-docker-support 1.3.1 voxl-elrs 0.1.3 voxl-esc 1.4.0 voxl-feature-tracker 0.3.2 voxl-flow-server 0.3.3 voxl-fsync-mod 1.0-r0 voxl-gphoto2-server 0.0.10 voxl-gpio-mod 1.0-r0 voxl-jpeg-turbo 2.1.3-5 voxl-lepton-server 1.2.0 voxl-libgphoto2 0.0.4 voxl-libuvc 1.0.7 voxl-logger 0.4.0 voxl-mavcam-manager 0.5.3 voxl-mavlink 0.1.1 voxl-mavlink-server 1.4.0 voxl-microdds-agent 2.4.1-0 voxl-modem 1.0.9 voxl-mongoose 7.7.0-1 voxl-mpa-to-ros 0.3.7 voxl-mpa-to-ros2 0.0.3 voxl-mpa-tools 1.1.5 voxl-neopixel-manager 0.0.3 voxl-open-vins 0.4.4 voxl-open-vins-server 0.2.18 voxl-opencv 4.5.5-2 voxl-platform-mod 1.0-r0 voxl-portal 0.6.5 voxl-px4 1.14.0-2.0.68 voxl-px4-imu-server 0.1.2 voxl-px4-params 0.3.8 voxl-qvio-server 1.0.2 voxl-remote-id 0.0.9 voxl-ros2-foxy 0.0.1 voxl-streamer 0.7.4 voxl-suite 1.2.0 voxl-tag-detector 0.0.4 voxl-tflite-server 0.3.2 voxl-utils 1.3.9 voxl-uvc-server 0.1.6 voxl-vision-hub 1.7.4 voxl2-system-image 1.7.6-r0 voxl2-wlan 1.0-r0 --------------------------------------------------------------------------------
-
RE: cam does not seem to be alive issue
@Alex-Kushleyev
I can confirm that the ar0144 module works.
it works if i switch to J7L and then the imx connected to J7L wont work when it is connected to J7U.
I know the setup works because i am currently running it with the same hardware on another voxl2@Vinny
thank you for the comment, I will bring an ESD mat.