@alex-kushleyev Thanks for the response. I missed this since I didn't receive any notification. It would be nice if it gets notified.
I used voxl-calibrate-camera with the -f flag since the lens looked like a fisheye, but it always errored out. Without the fisheye parameter, it worked fine, with a reprojection error of 0.43. I would love to get your update on more efficient processing. May I know which feature branch prints the intrinsics for the ToF camera?
Okay, thanks for confirming.
I'm building a monocular depth model that predicts relative disparity, which I then scale to metric using VIO features + ToF as anchors. The hard part is reprojecting tracking-front features into the hires frame, since the depth model runs on hires (RGB works much better for these models). Hires has a narrower FOV, so most features fall outside once reprojected.
I tried features from both qVIO and OpenVINS. qVIO consistently gives far more. Measured over 250 frames from the same log:
qVIO: ~64 features published, ~26 land in the hires FOV
OpenVINS: ~17 published, ~2 land in the hires FOV
Looking at starling2/estimator_config.yaml, I'm guessing this comes from num_pts: 50, max_slam: 30, max_msckf_in_update: 10. Is that right, and is raising them the correct way to get more features published? The transport caps at 64 (VIO_MAX_REPORTED_FEATURES), so I assume that's the ceiling.
CA_ROTOR*_PX and CA_ROTOR*_PY look swapped relative to the physical frame. Measuring motor-center to motor-center: front-to-back (X, forward) ≈ 0.066 m, side-to-side (Y) ≈ 0.088 m, so forward is the shorter dimension. But by default the params have it the other way round on all drones we have:
CA_ROTOR0_PX [87,258] : 0.0850
CA_ROTOR0_PY [88,259] : 0.0625
The end goal is an RL policy subscribing to odometry and image topics, outputting PX4 offboard commands.
A few other things I've run into:
I want to add the hires stream to OpenVINS as an extra camera, so I don't have to reproject from tracking front. I hope rolling shutter won't affect it much. Also, does it strictly need misp ion?
I tried some transformer-based models, but tflite doesn't support several ops, technically it does, but not for the specific shapes I need. Is there a newer voxl-tflite-server with broader op/shape coverage? I also spoke to Qualcomm, and it sounds like the QRB5165 is old enough that anything beyond CNNs is difficult.
We are in a situation to run the following at once: custom object detection model, depth estimation, an RL control policy consuming the depth output, GPS, plus streaming hires to the ground station over gstreamer at high rate. Still building, but I'm doubtful all of it fits comfortably on VOXL2. I'm surprised that thermal throttling started with just the single depth model in tflite-server, while GPU utilisation was only ~50%. We're looking at heatsinks and hoping propeller airflow plus a heatsink together are enough and would work in sync.
Worst case, is it possible to pair and run two VOXL2s on a Starling 2? Any guidance would be appreciated, and hopefully it doesn't add much payload (11 g plus connectors for the VOXL Mini, I guess). I also read about VOXL3, and it looks very promising. Any idea when it's releasing, and is there a pre-release programme for academic research labs?
