Camera Intrinsics, VIO and more..

[Dinesh Varun Kandiyappan]

Dear Team,

1. I am looking to obtain the following intrinsic calibration values for the Starling 2, but I was unable to find them on the drone. It would be great if these calibration parameters are available: RGB HiRes IMX412, PMD ToF intrinsics.
2. I did find intrinsics for both the forward and downward-facing tracking cameras, as well as extrinsics for all onboard cameras, can I safely assume these values are accurate?
3. I also noticed that the ToF sensor performs poorly outdoors( which I understand is a known limitation), are there any recommended approaches for obtaining metric depth for outdoor SLAM, beyond using monocular depth estimation models? I came across documentation mentioning stereo modules, but I don't see them listed on the products page. Are there plans to make them available, and if so, what depth range can we expect from them?
4. I also noticed that both qVIO and Open-VINS are available onboard. Before I run empirical tests to compare them, do you have any recommendations regarding their usage in terms of accuracy and suitability for different conditions such as indoor vs. outdoor environments?
5. I noticed that the PX4 airframe XY positions appear to be swapped across all the drones. is this intentional?
6. Finally, we are trying to run a deep learning model on the VOXL2, what would be the recommended approach for reading and feeding inputs to the model at high frequency with low latency, while also sending control commands to PX4?

Thank you !

[Dinesh Varun Kandiyappan]

@alex-kushleyev

[Alex Kushleyev]

Hi @dinesh-varun-kandiyappan , please see responses below:

1. the high resolution cameras (IMX412) are not calibrated during production. PMD tof does have intrinsics avaiable and they are downloaded from the sensor directly when the voxl-camera-server strarts. We did make a feature branch to print out those intrinsics for the TOF sensor - we can revisit that (and mainline it). You can use our tool voxl-calibrate-camera to perform intrinsics calibration of the fisheye IMX412 camera. We also have some improvements to make the process more efficient for high resolution images - if you need that, we can share the update (on todo list to mainline). https://docs.modalai.com/calibrate-cameras/

2. all the extrinsics are obtained from the 3D CAD models (not calibrated). they should be pretty accurate, but we do not have any data / analysis on the accuracy

3. Yes, TOF sensor is an indoor sensor because outdoors the IR light will interfere with the sensor. We used to have drones that used DFS (Depth From Stereo, which used an algorithm from Qualcomm), but that stereo camera setup (using old ov7251 cameras) is EOL and we are not actively developing / testing DFS. However, you could potentially put together two AR0144 tracking cameras in a stereo configration and try DFS.

4. QVIO only works using a single camera, but Open-Vins supports multiple cameras and that provides more reliable VIO solution. We are not developing QVIO, but Open-Vins is under active development and testing, so you should probably start with that. Also, QVIO is closed source (and not being actively developed), so it has no potential for updates / improvements.

5. Can you please clarify? the PX4 params should be set up correctly for each drone after you run voxl-configure-mpa

6. Can you please clarify? Is the goal for your model to control the vehicle's flight? (if so, what type of controls?)

Alex