@modaltb Thankyou so much for the quick eyes on this!
We owe you some beers!
Posts made by Steve Turner
-
RE: VOXL2 Mini J10 UART
-
VOXL2(mini) Serial Number
Is there a way to get a unique serial number from VOXL2 hardware? We'd like to do checkouts on each VOXL2/mini platform and track units through assembly and be able to access this in software. I would imagine this gets overwritten the moment you also do a factory image flash. Just wanted to check!
It looks like maybe you guys had something like this working in: https://docs.modalai.com/Qualcomm-Flight-RB5-sdk-utils/#rb5-flight-versions
Perhaps based on: https://github.com/quic/sample-apps-for-robotics-platforms/blob/master/RB5/linux_kernel_4_x/platform-bringup/Device-info/src/qrb5165_info.c
Or just
cat /sys/bus/soc/devices/soc0/serial_number
and we associate that with our ModalAI Serial Number.What hardware are you using? VOXL2, VOXL2mini
What version of software are you using? SDK 1.1.1
How have you configured the software? Yes (mpa, etc)
Do you have any logs? Not required for this issue.
Can you share pictures of your setup? If needed -- applies to any VOXL2(mini) based hardware.
Have you looked at the source code? Yes -- did see that the RB5 has a utility that maybe can do this and the SKU utils. -
VOXL2 Mini Camera Configuration Instability
We are trying to get a 3xTracking and 1x Hi-Res configuration working. Based on testing, it seems this might not be a supported configuration.
- Based on stock bionic Modal kernel and image released
- Based on interface board m0135
- The Tracker sensor (ov7251) can only be connected to JU (furthest from Qualcomm chip)
- The HiRes sensor (Imx214) can only be connected to JL, (Closest to the Qualcomm chip)
- Connecting the wrong sensor to the JU/JL on Voxl2 mini J6 connector will fail to boot and normally require a fast boot to reimage
- Connecting to J7 does not boot, but generally recovers without fastboot reimage required.
- If using the m0135 inteface board, one must be fully populated with both a high res and a tracker sensor
- If using two, the second board should not populate the HiRes sensor
- Trying to connect two of the same sensors to the interface board will fail, and depending if J6 is used, a fast boot reimage may be required
Reviewing m0014 (tracker) shows the cable is 1 lane CSI with most pins not connected. There is no obvious reason for crashing when connecting to the wrong JU/JL.
Is there something that potentially is causing a hardware fault in this configuration?The highest sensor configuration we can get working is that above, 1 HiRes and 2 Trackers.
Thanks for any help Modal Team!
-
RE: Heating and Excessive CPU usage in VOXL 2 under normal operation
Similar to you both:
voxl-camera-server | Enabled | Running | 71.1% voxl-cpu-monitor | Enabled | Running | 0.1% voxl-dfs-server | Disabled | Not Running | voxl-feature-tracker | Disabled | Not Running | voxl-flow-server | Disabled | Not Running | voxl-imu-server | Enabled | Running | 1.5% voxl-lepton-server | Disabled | Not Running | voxl-mavcam-manager | Disabled | Not Running | voxl-mavlink-server | Enabled | Running | 0.2% voxl-modem | Disabled | Not Running | voxl-portal | Enabled | Running | 60.1%
This is with HIRES, Tracking, Stereo on SDK 1.0. I also disabled large encoding on the HIRES from the camera server. Just bumping to see if this is expected.
-
VOXL2 Image A/B Bootloading
Modal team (or others),
Before we put any effort in, was curious if anyone has looked into solutions like Mender for rolling updates out to the VOXL2 boards?
I would assume we would need full access to the bootloader and image builds in order to do so beyond what you are able to make available here: https://gitlab.com/voxl-public/system-image-buildThanks!
Steve -
RE: Camera server dying with new MIPI extension cables.
@viralp Make sure you confirm the cable orientation remains the same as with the shorter cables. If the pins flip, the camera server will start and fail.
Also, sometimes interference at a certain length can cause issues. Also a debug log of the camera server would be helpful by running the camera server with the debug flag.
-d, --debug-level : Log debug level (Default 2) 0 : Print verbose logs 1 : Print >= info logs 2 : Print >= warning logs 3 : Print only fatal logs
-
RE: voxl-mavlink-server external FC Bus Error
Zach,
Thanks for the quick response!
I can run both on the dev branch, or roll back to staging.I thought that voxl-vision-hub would talk to the voxl-mavlink-server which handles all communication with PX4.
From there voxl-mavlink-server forwards between visionhub, px4, and QGC once it connections.
I'll try running the new config tool again to see if that cleans anything up.
Anything else to look for for the bus error?
-
voxl-mavlink-server external FC Bus Error
We were running a VOXL-1 based system off of staging.
Aftern an
opkg update
andopkg upgrade
it looks like the MAVLINK server is broken. We're investigating now, just wanted to get this out there.voxl:~$ voxl-mavlink-server loading our own config file ================================================================= Parameters as loaded from config file: primary_static_gcs_ip: 192.168.8.10 secondary_static_gcs_ip: NULL autopilot_uart_backup_bus: 12 autopilot_uart_bus: 5 autopilot_uart_baudrate: 921600 udp_mtu: 0 en_external_ap_timesync: 1 ================================================================= Successfully opened primary mavparser on bus 5 Starting primary thread for bus 5 Successfully opened backup mavparser on bus 12 Starting backup thread for bus 12 got first packet on bus 5, stopping the other bus: 12 Adding primary GCS IP address from conf file to list: 192.168.8.10 Added new UDP connection to 192.168.8.10 Not connecting to secondary gcs ip: NULL Added new UDP connection to 10.147.20.136 CONNECTED to GCS at 10.147.20.136 Init complete, entering main loop Detected Autopilot Mavlink SYSID 1 thread for bus 12 trying to close thread for bus 12 Successfully closed Bus error
If I rollback the mavlink server to previous versions it seems happy but I can't connect to
voxl-vision-hub
through the mavlink server from something like QGC. -
RE: Voxl2 + flight core in custom frame
Will be the VOXL2 IMU's.
https://docs.modalai.com/configure-extrinsics/#voxl2 recommends using
imu_apps
for configuration. -
RE: Reconfiguring VIO
Use cam name = "stereo" and there should be an option for using either the left or right stereo camera.
As Chad pointed out you will need to also configure extrinsics if they aren't already for the stereo camera.
https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/blob/master/server/config_file.h#L155I have a PR to fix an issue with using a stereo camera here with the QVIO Server: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/merge_requests/10 that you will need as well I believe.
-
RE: Voxl2 + flight core in custom frame
Extrinsics and QVIO server configuration will determine camera orientation and which IMU to use respectively.
https://docs.modalai.com/configure-extrinsics/
You will need to measure from the IMU (Parent) to child (camera) for translation in meters and also the roll, pitch, and yaw in degrees. I recommend watching the video on the configure extrinsics page.
-
RE: Fusing optical flow together with VIO
Yes, you can. You may need to evaluate the performance of both at the same time and make tweaks.
-
RE: VIO flight capabilities
You can change the QVIO server to use a stereo camera if you want in the front, but these are much narrower FOV.
https://docs.modalai.com/configure-extrinsics/ would need to be adjusted to accommodate any camera location changes as well. This is not something supported out of the box with the current VOXL software and would require some work on your part to make it fit your platform and camera configuration.
-
RE: sysimg 4.0.0 voxl-camera-server crashing tracking
Resurrecting this topic. We have two drones with identical hardware builds.
Only camera's plugged in are tracking cameras to J4.
Both are running the latest system image.Drone 1's camera server never crashes while flying Indoors/outdoors.
Drone 2's camera server sometimes crashes and seems to be triggered by exposure changes when outside in bright environments. It is running a slightly newer version ofvoxl-camera-server.
The failing drone has this in the logs from
logcat | grep "camera"
:voxl:~$ logcat | grep camera 01-26 20:06:14.004 5732 5732 V HAL : loaded HAL id=camera path=/usr/lib/hw/camera.msm8996.so hmi=(nil) handle=0x64450 01-26 20:06:14.005 5732 5732 I QCamera : <MCI><INFO> get_num_of_cameras: 2767: dev_info[id=0,name='video2'] 01-26 20:06:14.005 5732 5732 I QCamera : <MCI><INFO> sort_camera_info: 2611: Camera id: 0 facing: 1, type: 1 is_yuv: 0 01-26 20:06:14.005 5732 5732 I QCamera : <MCI><INFO> sort_camera_info: 2617: Number of cameras 1 sorted 1 01-26 20:06:14.005 5732 5732 I QCamera : <MCI><INFO> get_num_of_cameras: 2785: num_cameras=1 01-26 20:06:14.006 5732 5732 I QCamera : <HAL><INFO> QCamera2Factory: 142: 1 camera devices detected! 01-26 20:06:14.107 5732 5732 E QCamera : <HAL><ERROR> initStaticMetadata: 9846: static int qcamera::QCamera3HardwareInterface::initStaticMetadata(uint32_t)@9846: from gCamCapability QCAMERA3_EXP_TIME_RANGE= (19425, 1020461640) 01-26 20:06:14.107 5732 5732 E QCamera : <HAL><ERROR> initStaticMetadata: 9877: static int qcamera::QCamera3HardwareInterface::initStaticMetadata(uint32_t)@9877: cameraId 0 raw_dim[0].w 640 raw_dim[0].h 480 01-26 20:06:14.107 5732 5732 I QCamera : <HAL><INFO> getCamInfo: 10272: camera 0 resource cost is 100 01-26 20:06:14.107 5732 5732 I QCamera : <HAL><INFO> cameraDeviceOpen: 414: Open camera id 0 API version 768 01-26 20:06:14.117 5732 5732 I QCamera : <HAL><INFO> openCamera: 851: [KPI Perf]: E PROFILE_OPEN_CAMERA camera id 0 01-26 20:06:14.197 5732 5732 I QCamera : <HAL><INFO> openCamera: 861: [KPI Perf]: X PROFILE_OPEN_CAMERA camera id 0, rc: 0 01-26 20:06:14.197 5732 5732 E QCamera : <HAL><ERROR> initParameters: 8374: int qcamera::QCamera3HardwareInterface::initParameters()@8374 HAL params are initialized 01-26 20:07:55.283 5732 5789 E QCamera : <MCI><ERROR> mm_camera_event_notify: 192: Camera Event DAEMON DIED received 01-26 20:07:55.284 5732 5788 E QCamera : <HAL><ERROR> camEvtHandle: 803: Fatal, camera daemon died
journalctl
on the drone with crashedvoxl-camera-server
shows:Jan 01 00:00:52 apq8096 systemd[1]: Started voxl-camera-server. Jan 26 20:01:17 apq8096 bash[2966]: ERROR: Recieved "Buffer" error from camera: tracking Jan 26 20:01:23 apq8096 bash[2966]: ERROR: Recieved "Buffer" error from camera: tracking Jan 26 20:01:24 apq8096 bash[2966]: ERROR: Recieved "Buffer" error from camera: tracking Jan 26 20:01:35 apq8096 bash[2966]: ERROR: Recieved "Buffer" error from camera: tracking Jan 26 20:01:37 apq8096 bash[2966]: ERROR: Recieved "Buffer" error from camera: tracking Jan 26 20:01:42 apq8096 bash[2966]: ERROR: Recieved Fatal error from camera: tracking Jan 26 20:01:42 apq8096 bash[2966]: ERROR: Sending request 7188, ErrorCode: -ENODEV Jan 26 20:01:42 apq8096 bash[2966]: WARNING: Thread: tracking request thread recieved ESTOP Jan 26 20:01:42 apq8096 bash[2966]: WARNING: Thread: tracking result thread recieved ESTOP Jan 26 20:01:42 apq8096 bash[2966]: ------ voxl-camera-server INFO: Camera server is now stopping Jan 26 20:01:42 apq8096 bash[2966]: WARNING: Deleting buffers: 24 of 32 still in use Jan 26 20:01:42 apq8096 bash[2966]: Segmentation fault: Jan 26 20:01:42 apq8096 bash[2966]: Fault thread: voxl-camera-ser(tid: 2966) Jan 26 20:01:42 apq8096 bash[2966]: Fault address: 0x8 Jan 26 20:01:42 apq8096 bash[2966]: Address not mapped. Jan 26 20:01:43 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Main process exited, code=killed, status=11/SEGV[[0m Jan 26 20:01:43 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Unit entered failed state.[[0m Jan 26 20:01:43 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Failed with result 'signal'.[[0m
They have not been crashed... and the failure happens regularly when in bright environments. I matched up software versions as well and the crash still happens, so it has to be some kind of hardware issue and the below can be disregarded.
Hardware problems again with expansion or something weird in this specific camera/cable?
Drone 1 package versions (a bit older/stable repo):
libvoxl-cci-direct - 0.1.5 libvoxl-cutils - 0.1.1 voxl-camera-calibration - 0.2.3 voxl-camera-server - 1.3.5 voxl-cpu-monitor - 0.3.0 voxl-docker-support - 1.2.4 voxl-gphoto2-server - 0.0.10 voxl-jpeg-turbo - 2.1.3-4 voxl-libgphoto2 - 0.0.4 voxl-libuvc - 1.0.7 voxl-logger - 0.3.4 voxl-mavlink - 0.1.0 voxl-mavlink-server - 0.2.0 voxl-modem - 0.16.1 voxl-mongoose - 7.7.0-1 voxl-mpa-to-ros - 0.3.6 voxl-mpa-tools - 0.7.6 voxl-opencv - 4.5.5-1 voxl-portal - 0.4.2 voxl-qvio-server - 0.9.0 voxl-streamer - 0.4.1 voxl-suite - 0.9.4 voxl-system-image - 4.0-r0 voxl-tag-detector - 0.0.4 voxl-tflite-server - 0.3.1 voxl-utils - 1.2.2 voxl-uvc-server - 0.1.3 voxl-vision-px4 - 1.4.0 voxl-vpn - 0.0.6
Drone 2 package versions (a bit newer, dev repos):
libvoxl-cci-direct - 0.1.5 libvoxl-cutils - 0.1.1 voxl-camera-calibration - 0.3.0 voxl-camera-server - 1.3.7 voxl-cpu-monitor - 0.4.1 voxl-docker-support - 1.2.5 voxl-gphoto2-server - 0.0.10 voxl-jpeg-turbo - 2.1.3-4 voxl-lepton-server - 1.0.1 voxl-libgphoto2 - 0.0.4 voxl-libuvc - 1.0.7 voxl-logger - 0.3.4 voxl-mavlink - 0.1.0 voxl-mavlink-server - 0.2.0 voxl-modem - 0.16.2 voxl-mongoose - 7.7.0-1 voxl-mpa-to-ros - 0.3.6 voxl-mpa-tools - 0.8.4 voxl-opencv - 4.5.5-1 voxl-portal - 0.4.3 voxl-qvio-server - 0.9.0 voxl-streamer - 0.4.3 voxl-suite - 1.0.0 voxl-system-image - 4.0-r0 voxl-tag-detector - 0.0.4 voxl-tflite-server - 0.3.1 voxl-utils - 1.2.3 voxl-uvc-server - 0.1.3 voxl-vision-px4 - 1.4.1 voxl-vpn - 0.0.6
-
RE: EV data causing the drone to crash when in stabilize mode
https://github.com/PX4/PX4-Autopilot/issues/20950
@Sem-Andeweg Just cross-posting here too for anyone that comes back to it. I'm interested in these issues as well.
-
RE: EV data causing the drone to crash when in stabilize mode
Sem, (I don't work for modal!)
A few ideas..
Have you tried disabling the QVIO service and adjusting the EKF2_AID_MASK values to ignore the vision inputs across the modes you've tested with PX4?
See how the platform behaves and that should either point to VIO or rule it out. In stabilized mode and without GPS and VIO inputs the platform should be relatively stable.
-
High-Resolution + TOF + Stereo on VOXL?
Is it possible to run High-Resolution + TOF + Stereo on the VOXL? Very familiar with the configs at https://docs.modalai.com/voxl-camera-config/
Apologies for a bit of a noob question -- just tried it out and getting a segfault when J4 is set to stereo.
If it doesn't work now, that's fine just curious if it is fixable in software or a different port config before we dive in too far.Thanks!
-
RE: Is there any the off-the-shelf AI perception models available for VOXL2?
I don't work for Modal but,
Questions 1,2,3:I recommend checking out https://docs.modalai.com/voxl-tflite-server/#voxl-2 and following along with deploying the TF Lite server on your VOXL2.
The source is available here https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server and includes many built-in models.
Questions 4:
Yes for both cameras, but likely will need some adaptation particularly for thermal. -
RE: voxl 2 flight deck with esc v2 few questions
I do not work for ModalAI, but will attempt to answer.
The V2 ESC supports serial and PWM input. https://docs.modalai.com/modal-escs/
I am unsure if it supports DSHOT but you could configure it to use PWM. That said, if you're using a VOXL the serial connection is a better option.As far enabling DShot in the firmware, you need to make sure the uart is disabled by setting
UART_ESC_CONFIG
to disabled. Then I believe the option for DShot/PWM will show up.
Check out https://docs.modalai.com/modal-esc-px4-user-guide/ for more info.https://www.modalai.com/products/voxl2-io?variant=40610514206771 for generating PWM from the VOXL2 to command a larger 4-1 ESC.