@star123 I'll validate the extrinsics and check back with you in a bit. I know the rear camera was moved between starling Max Rev 1&2, I'll double check the little starling too.
The point inside the lens used is what the lens manufacturer claims is the optical center of the lens, 1.7mm inward from the front of the glass.
@groupo It is not on Sentinel.
On more recent versions of the Starling 2 Max we moved the GPS to the back of the drone. I'll reach out to you via PM regarding updating your Starling Max.
We just discovered a source of EMI which is the cable running to the optional Lepton sensor socket. For a number of reasons we've not offered Lepton as an option of Starling 2 Max yet so you can safely disconnect the 8-pin JST SH connector on the front left side of your camera breakout board next to the barometer cap. This will improve GPS performance. Please see picture below.
@Bernhard When we released the 5G version of the Starling Max we also moved the GPS to the back of the drone and specified smaller LTE antennas which those landing gear were designed to fit. I'll reach out to you via PM regarding updating your Starling Max.
We just discovered a source of EMI which is the cable running to the optional Lepton sensor socket. For a number of reasons we've not offered Lepton as an option of Starling 2 Max yet so you can safely disconnect the 8-pin JST SH connector on the front left side of your camera breakout board next to the barometer cap. This will improve GPS performance. Please see picture below.
Hi Griffin,
DFS stands for depth-from-stereo and is only used on the M500 and Sentinel platforms that have stereo cameras.
Starlings use the PMD TOF camera which outputs 3D point points directly from voxl-camera-server. That data is in the reference frame of the camera such that Z points out the lens, X to the right of the image, and Y pointing downwards.
voxl-vision-hub then consumes that data and rotates it based on the mounting location described in /etc/modalai/extrinsics.conf and the current position/orientation of the drone as reported by VIO.
voxl-mapper is not installed by default as it's not part of the core SDK, however it is locally available and can be installed with apt install voxl-mapper without needing an internet connection.
Best,
James
it looks like very few services are running. If you run voxl-inspect-services you can see what's enabled. You will want to enable voxl-vision-hub, voxl-mavlink-server and voxl-px4
What kind of sensors and data are you dealing with? voxl-vision-hub can subscribe to rangefinder, generic points clouds, and tof data. Since all the logic for compensating for roll/pitch to level out the point cloud data then slice the data into pie-slices to send to PX4 is done already in voxl-vision-hub I would suggest leveraging this. I think the most simple solution would be option 2 and then edit the voxl-vision-hub config file to point to your new server's pipe.
Starling 2 Max comes out of the box with PX4's EKF2 configured for GPS outdoor flight.
To fly indoors with VIO only you need to modify PX4's EKF2 parameters in QGC or with the voxl-configure-px4-params utility. In the EKF2_helpers menu, select "indoor_vio.params"
voxl2:/$ voxl-configure-px4-params -w
[INFO] Voxl-px4 service found and active.
Select an option:
[1] Quit
[2] Reset ALL PX4 params to default
[3] experimental_do_not_use
[4] EKF2_helpers
[5] battery_helpers
[6] ci_helpers
[7] voxl2_io_helpers
[8] other_helpers
[9] platforms
[10] radio_helpers
[11] voa_helpers
[12] calibration files
4
Select a EKF2 helper:
[1] Quit
[2] Back
[3] outdoor_flow.params
[4] outdoor_gps.params
[5] outdoor_gps_baro.params
[6] vio_gps_baro.params
[7] indoor_vio.params
[8] indoor_vio_missing_gps.params
7
[INFO] Loading file: /usr/share/modalai/px4_params/v1.14/EKF2_helpers/indoor_vio.params
[INFO] Scanning currently loaded parameters...
The following parameters about to be loaded differ from those currently loaded:
NAME CURRENT NEW
[1] EKF2_MAG_TYPE 0.0 5.0
[2] EKF2_EV_CTRL 0.0 15.0
[3] EKF2_HGT_REF 1.0 3.0
[4] EKF2_GPS_CTRL 7.0 0.0
[5] EKF2_EV_QMIN 0.0 1.0
Would you like to continue with the parameter differences listed above? (y/n)
y
[INFO] Setting params...
[INFO] PX4 parameter configuration successful!
Select an option:
[1] Quit
[2] Reset ALL PX4 params to default
[3] experimental_do_not_use
[4] EKF2_helpers
[5] battery_helpers
[6] ci_helpers
[7] voxl2_io_helpers
[8] other_helpers
[9] platforms
[10] radio_helpers
[11] voa_helpers
[12] calibration files
1
voxl2:/$
Then reboot the Starling and PX4 should be configured to use the VIO data.
@LucaVertiq Great work!! I'm going to be OOO for a while but as we discussed before I'm happy to review it when I'm back and integrate into the SDK.
@Jetson-Nano If you are using a stereo-pair that are being published together on the same pipe like on a Seeker, M500, or Sentinel, then you should just need to set up the following config files:
/etc/modalai/voxl-qvio-server.conf:
"cam_name": "stereo", (or stereo_front for sentinel)
"lens_cal_path": "/data/modalai/opencv_stereo_intrinsics.yml", (or stereo_front_intrinsics for Sentinel)
"cam_extrinsics_name": "stereo_l", (or stereo_front_l for sentinel)
"use_second_cam_if_stereo": false,
To use the right camera, you can swap "use_second_cam_is_stereo" to true, and qvio will pull in instrinsic lens params from the right camera which is also part of the same intrinsics file. Note that we only have camera locations in the extrinsics files for the left camera, but you can add to /etc/modalai/extrinsics.conf an entry for the right camera if you wish.
@rogerli vibration and lead to the lenses rotating. I often use a dab of hot-glue to hold the lens in place after focusing. On the new Starling 2 and Starling 2 Max drones we now use a clamping ring around the entire camera and lens assembly to lock them in place.
@wilkinsaf voxl-vision-hub's configuration file can be modified to subscribe to multiple sources of relevant data. By default it it set up to consume point cloud data from voxl-dfs-server (stereo cameras on Sentinel and M500), TOF data from PMD TOF sensors (Seeker, Starling, and Starling 2), as well as VL53L1CX mini rangefinders. Data from all concurrent sources are merged together and passed to the autopilot as an obstacle_distance message over mavlink.
Multiple of the VL53L1CX rangefinders can be set up using the open-source voxl-rangefinder-server https://docs.modalai.com/rangefinders/
However, you can integrate any sensor you like by publishing one of the above 3 MPA types that voxl-vision-hub can consume.
@Kris I found and fixed a bug in voxl-camera-server that seems to be the culprit. I flashed my seeker with SDK1.2.0 and sideloaded the following voxl-camera-server with the bugfix to re-enable ingestion of TOF data into voxl-vision-hub
Note VOA simply does not work in the default PX4 acceleration-based position mode so MPC_POS_MODE needs to be set to 0 (velocity control) as done by the voa_enable px4 param helper file here:
In SDK 1.2.0 we did start pushing the default config of voxl-mapper a big harder with smaller voxl sizes, longer ray lengths, and double the UI update rate to better complement the newer TOF sensors on the Starling 2. Part of this update was also to lock some of the heavier parts of the mapper processing to cores 4,5,6 to make sure VIO wasn't affected too much. However, if you are still seeing a difference in VIO performance between mapper being on and off then there may still be some optimization to be done.
Just to confirm you are seeing worse VIO performance when mapper is enabled, correct?
If so, please try increasing the voxl_size parameter in /etc/modalai/voxl-mapper.conf from the default 0.1 to 0.25, and set the cpu into performance mode on bootup by changing the normal_cpu_mode field to "performance" in /etc/modalai/voxl-cpu-monitor.conf. The map will be very course and ugly with such a big voxel size but I'm just curious if the power cpu use helps resolve the issue.
For reference, here is the commit history for voxl-mapper showing the recent tweaks for SDK 1.2 and newer that turn up some of the knobs.
Hi @Jetson-Nano
The I2C port on J19 is connected to the Sensors DSP and therefore can't be accessed from the main applications processor.
Here is a new documentation page describing how to hook up and use these sensors.
That's a typo on the website we've just corrected. The Starling 2 Max does not include a TOF sensor, and this sensor is not intended for outdoor use.
I spent some time tuning the TOF point cloud processing filter params on Friday to reduce noise. These changes are in the dev branch now: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commit/919c24bef21cbea2eb7be33853884f5f606a0763
These these changes are likely to help in the outdoor use case too so I suggest giving it a go.
It's interesting that you see a difference in performance between SDK1.0 and SDK1.1, between those releases was a change in the way IMU timestamps were calculated that now provide more consistent and accurate timestamps which should have improved vio performance.
Do you mind trying SDK 1.1.3 with both the newest qrb5165_imu_server (1.0.1) and the old qrb5165-imu-server (0.6.0) from the old SDK to see if the problem is tied to the IMU server or something else?
http://voxl-packages.modalai.com/dists/qrb5165/sdk-1.0/binary-arm64/qrb5165-imu-server_0.6.0_arm64.deb
http://voxl-packages.modalai.com/dists/qrb5165/sdk-1.1/binary-arm64/qrb5165-imu-server_1.0.1_arm64.deb
Thank you!
Hi. The sensor is integrated into the VOXL SDK and raw data can be viewed through ADB or SSH with the voxl-inspect-rangefinders command line tool. All PX4 Autonomy Dev kids have this sensor. That data is sent from qrb5165-rangefinder-server to PX4 via voxl-mavlink-server. To enable in PX4 set the PX4 EKF2_RNG_CTRL param to 1. You will likely need to fiddle with PX4's other rangefinder params to achieve the behavior you want.
Best,
James
Hi Eric,
The stereo camera pair intrinsics and extrinsics are saved to /data/modalai/opencv_stereo_intrinsics.yaml and /data/modalai/opencv_stereo_extrinsics.yaml
We then use the following opencv calls to generate the rectification maps for each left and right image
cv::Mat R1, P1, R2, P2;
cv::Mat map11, map12, map21, map22;
cv::Mat Q; // Output 4×4 disparity-to-depth mapping matrix, to be used later
double alpha = -1; // default scaling, similar to our "zoom" parameter
cv::stereoRectify(M1, D1, M2, D2, img_size, R, T, R1, R2, P1, P2, Q, cv::CALIB_ZERO_DISPARITY, alpha, img_size);
// make sure this uses CV_32_FC2 map format so we can read it and convert to our own
// MCV undistortion map format, map12 and 22 are empty/unused!!
cv::initUndistortRectifyMap(M1, D1, R1, P1, img_size, CV_32FC2, map11, map12);
cv::initUndistortRectifyMap(M2, D2, R2, P2, img_size, CV_32FC2, map21, map22);
Those maps then get converted into a format that can be used by the Qualcomm hardware acceleration blocks for image dewarping, but the fundamental matrices and undistortion coefficients stay the same. I think the above lines is what you are after.
You are correct that the disparity map is centered about the left camera, I believe this is typical in OpenCV land too.
Note that we do not do a precise calibration between the Stereo and Hires Cameras from the factory, depending on how precise you need to alignment to be you may need to use a tool like Kalibr to do that.
@Chase-Riley Glad to hear. Does it also perform as expected at lower values such as 100 or 200? a value as high as 500 will increase latency when only sending small mavlink messages.
0.9765625 = 32/32.768
The ICM42688 will run at roughly 32khz +- a few percent when running off its own internal oscillator. Optionally, you can drive it from a standard 32.768khz crystal which is what we opted to do on VOXL2 so the sample rate is consistent between boards.