Posts made by James Strawson

voxl-openvins-server is under rapid development at the moment and I expect the entire configuration file to be cleaned up dramatically quite soon. After that point the configuration file contents will stabilize and we will update the documentation to match. Fully documenting the current state would be a wasted effort.

It should not be necessary to modify the configuration file. When flashing the 1.4.1 SDK a new default config file will have been made. What differs from drone-to-drone is the physical layout of the cameras and IMU which is defined by /etc/modalai/extrinsics.conf as well as the cameras that should be used for vio which is set up in /etc/modalai/vio_cams.conf

yes, that's the correct rotation matrix for a tag on the ground.

For a tag on the wall with a center 1m off the ground you can use:
"T_tag_wrt_fixed": [0, 0, -1],
"R_tag_to_fixed": [[0, 0, 1], [1, 0, 0], [0, 1, 0]]

The second video of this docs page walks you through the voxl-vision-hub command line args to help you debug relocalization.

Are the failed batteries the white 3000mah Gaoneng packs or the black 4000mah Upgrade Energy Packs?

@star123 I'll validate the extrinsics and check back with you in a bit. I know the rear camera was moved between starling Max Rev 1&2, I'll double check the little starling too.

The point inside the lens used is what the lens manufacturer claims is the optical center of the lens, 1.7mm inward from the front of the glass.

@groupo It is not on Sentinel.

@nuallaino

On more recent versions of the Starling 2 Max we moved the GPS to the back of the drone. I'll reach out to you via PM regarding updating your Starling Max.

We just discovered a source of EMI which is the cable running to the optional Lepton sensor socket. For a number of reasons we've not offered Lepton as an option of Starling 2 Max yet so you can safely disconnect the 8-pin JST SH connector on the front left side of your camera breakout board next to the barometer cap. This will improve GPS performance. Please see picture below.

@Bernhard When we released the 5G version of the Starling Max we also moved the GPS to the back of the drone and specified smaller LTE antennas which those landing gear were designed to fit. I'll reach out to you via PM regarding updating your Starling Max.

We just discovered a source of EMI which is the cable running to the optional Lepton sensor socket. For a number of reasons we've not offered Lepton as an option of Starling 2 Max yet so you can safely disconnect the 8-pin JST SH connector on the front left side of your camera breakout board next to the barometer cap. This will improve GPS performance. Please see picture below.

Hi Griffin,

DFS stands for depth-from-stereo and is only used on the M500 and Sentinel platforms that have stereo cameras.

Starlings use the PMD TOF camera which outputs 3D point points directly from voxl-camera-server. That data is in the reference frame of the camera such that Z points out the lens, X to the right of the image, and Y pointing downwards.

voxl-vision-hub then consumes that data and rotates it based on the mounting location described in /etc/modalai/extrinsics.conf and the current position/orientation of the drone as reported by VIO.

voxl-mapper is not installed by default as it's not part of the core SDK, however it is locally available and can be installed with apt install voxl-mapper without needing an internet connection.

Best,
James

@giladWDS

it looks like very few services are running. If you run voxl-inspect-services you can see what's enabled. You will want to enable voxl-vision-hub, voxl-mavlink-server and voxl-px4

What kind of sensors and data are you dealing with? voxl-vision-hub can subscribe to rangefinder, generic points clouds, and tof data. Since all the logic for compensating for roll/pitch to level out the point cloud data then slice the data into pie-slices to send to PX4 is done already in voxl-vision-hub I would suggest leveraging this. I think the most simple solution would be option 2 and then edit the voxl-vision-hub config file to point to your new server's pipe.

@TOMPLA

Starling 2 Max comes out of the box with PX4's EKF2 configured for GPS outdoor flight.

To fly indoors with VIO only you need to modify PX4's EKF2 parameters in QGC or with the voxl-configure-px4-params utility. In the EKF2_helpers menu, select "indoor_vio.params"

voxl2:/$ voxl-configure-px4-params -w
[INFO] Voxl-px4 service found and active.
Select an option:
[1] Quit
[2] Reset ALL PX4 params to default
[3] experimental_do_not_use
[4] EKF2_helpers
[5] battery_helpers
[6] ci_helpers
[7] voxl2_io_helpers
[8] other_helpers
[9] platforms
[10] radio_helpers
[11] voa_helpers
[12] calibration files
4

Select a EKF2 helper:
[1] Quit
[2] Back
[3] outdoor_flow.params
[4] outdoor_gps.params
[5] outdoor_gps_baro.params
[6] vio_gps_baro.params
[7] indoor_vio.params
[8] indoor_vio_missing_gps.params
7

[INFO] Loading file: /usr/share/modalai/px4_params/v1.14/EKF2_helpers/indoor_vio.params
[INFO] Scanning currently loaded parameters...
The following parameters about to be loaded differ from those currently loaded:
	NAME    		CURRENT    	NEW
[1]	EKF2_MAG_TYPE    	0.0		5.0
[2]	EKF2_EV_CTRL    	0.0		15.0
[3]	EKF2_HGT_REF    	1.0		3.0
[4]	EKF2_GPS_CTRL    	7.0		0.0
[5]	EKF2_EV_QMIN    	0.0		1.0

Would you like to continue with the parameter differences listed above? (y/n)
y

[INFO] Setting params...
[INFO] PX4 parameter configuration successful!
Select an option:
[1] Quit
[2] Reset ALL PX4 params to default
[3] experimental_do_not_use
[4] EKF2_helpers
[5] battery_helpers
[6] ci_helpers
[7] voxl2_io_helpers
[8] other_helpers
[9] platforms
[10] radio_helpers
[11] voa_helpers
[12] calibration files
1
voxl2:/$ 

Then reboot the Starling and PX4 should be configured to use the VIO data.

@LucaVertiq Great work!! I'm going to be OOO for a while but as we discussed before I'm happy to review it when I'm back and integrate into the SDK.

@Jetson-Nano If you are using a stereo-pair that are being published together on the same pipe like on a Seeker, M500, or Sentinel, then you should just need to set up the following config files:

/etc/modalai/voxl-qvio-server.conf:

    "cam_name":     "stereo", (or stereo_front for sentinel)
    "lens_cal_path":        "/data/modalai/opencv_stereo_intrinsics.yml", (or stereo_front_intrinsics for Sentinel)
    "cam_extrinsics_name":  "stereo_l", (or stereo_front_l for sentinel)
    "use_second_cam_if_stereo":     false,

To use the right camera, you can swap "use_second_cam_is_stereo" to true, and qvio will pull in instrinsic lens params from the right camera which is also part of the same intrinsics file. Note that we only have camera locations in the extrinsics files for the left camera, but you can add to /etc/modalai/extrinsics.conf an entry for the right camera if you wish.

@rogerli vibration and lead to the lenses rotating. I often use a dab of hot-glue to hold the lens in place after focusing. On the new Starling 2 and Starling 2 Max drones we now use a clamping ring around the entire camera and lens assembly to lock them in place.

@wilkinsaf voxl-vision-hub's configuration file can be modified to subscribe to multiple sources of relevant data. By default it it set up to consume point cloud data from voxl-dfs-server (stereo cameras on Sentinel and M500), TOF data from PMD TOF sensors (Seeker, Starling, and Starling 2), as well as VL53L1CX mini rangefinders. Data from all concurrent sources are merged together and passed to the autopilot as an obstacle_distance message over mavlink.

Multiple of the VL53L1CX rangefinders can be set up using the open-source voxl-rangefinder-server https://docs.modalai.com/rangefinders/

However, you can integrate any sensor you like by publishing one of the above 3 MPA types that voxl-vision-hub can consume.

@Kris I found and fixed a bug in voxl-camera-server that seems to be the culprit. I flashed my seeker with SDK1.2.0 and sideloaded the following voxl-camera-server with the bugfix to re-enable ingestion of TOF data into voxl-vision-hub

http://voxl-packages.modalai.com/dists/apq8096/dev/binary-arm64/voxl-camera-server_1.9.7_202407111658.ipk

Note VOA simply does not work in the default PX4 acceleration-based position mode so MPC_POS_MODE needs to be set to 0 (velocity control) as done by the voa_enable px4 param helper file here:

https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-px4-params/-/blob/dev/params/v1.14/voa_helpers/voa_enable_indoor.params?ref_type=heads

@Jetson-Nano

In SDK 1.2.0 we did start pushing the default config of voxl-mapper a big harder with smaller voxl sizes, longer ray lengths, and double the UI update rate to better complement the newer TOF sensors on the Starling 2. Part of this update was also to lock some of the heavier parts of the mapper processing to cores 4,5,6 to make sure VIO wasn't affected too much. However, if you are still seeing a difference in VIO performance between mapper being on and off then there may still be some optimization to be done.

Just to confirm you are seeing worse VIO performance when mapper is enabled, correct?

If so, please try increasing the voxl_size parameter in /etc/modalai/voxl-mapper.conf from the default 0.1 to 0.25, and set the cpu into performance mode on bootup by changing the normal_cpu_mode field to "performance" in /etc/modalai/voxl-cpu-monitor.conf. The map will be very course and ugly with such a big voxel size but I'm just curious if the power cpu use helps resolve the issue.

For reference, here is the commit history for voxl-mapper showing the recent tweaks for SDK 1.2 and newer that turn up some of the knobs.

Hi @Jetson-Nano

The I2C port on J19 is connected to the Sensors DSP and therefore can't be accessed from the main applications processor.

Here is a new documentation page describing how to hook up and use these sensors.

https://docs.modalai.com/rangefinders/

That's a typo on the website we've just corrected. The Starling 2 Max does not include a TOF sensor, and this sensor is not intended for outdoor use.

I spent some time tuning the TOF point cloud processing filter params on Friday to reduce noise. These changes are in the dev branch now: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commit/919c24bef21cbea2eb7be33853884f5f606a0763

These these changes are likely to help in the outdoor use case too so I suggest giving it a go.

@Jetson-Nano

It's interesting that you see a difference in performance between SDK1.0 and SDK1.1, between those releases was a change in the way IMU timestamps were calculated that now provide more consistent and accurate timestamps which should have improved vio performance.

Do you mind trying SDK 1.1.3 with both the newest qrb5165_imu_server (1.0.1) and the old qrb5165-imu-server (0.6.0) from the old SDK to see if the problem is tied to the IMU server or something else?

http://voxl-packages.modalai.com/dists/qrb5165/sdk-1.0/binary-arm64/qrb5165-imu-server_0.6.0_arm64.deb
http://voxl-packages.modalai.com/dists/qrb5165/sdk-1.1/binary-arm64/qrb5165-imu-server_1.0.1_arm64.deb

Thank you!

@frafrat

Hi. The sensor is integrated into the VOXL SDK and raw data can be viewed through ADB or SSH with the voxl-inspect-rangefinders command line tool. All PX4 Autonomy Dev kids have this sensor. That data is sent from qrb5165-rangefinder-server to PX4 via voxl-mavlink-server. To enable in PX4 set the PX4 EKF2_RNG_CTRL param to 1. You will likely need to fiddle with PX4's other rangefinder params to achieve the behavior you want.

Best,
James