Redundant VIO via AprilTags?

[Evan Hertafeld]

Hi all,

I've had a pleasant time setting up my Voxl Flight for offboard vision-based navigation but the occasional QVIO blowup makes me seek a level of redundancy for vision tracking. The apriltag relocalization feature is great, but if we blowup while relocalizing, then the setpoints leap to the new reference frame before the next tag is detected and we jerk/crash.

I'm sure there are things I can do do reduce the chance of a blowup but it seems unlikely I could eliminate the risk entirely - loss of light, debris on the camera, the sensor shifting, could all occur if this system is deployed at scale.

Currently the PX4 EKF seems to only take in a single vision stream, so my thoughts are focused around adding this redundancy upstream (eg within the Voxl system). The general idea would be to generate some other stream(s) of Odometry data with another vision source, and be able to switch to that if the main source fails. Most likely by passing both to some monitor process that in turn sends the "clean" data out the actual QVIO pipe for PX4.

So my questions are:

1. Is this a sensible idea? Gotchas to consider? Or aught I be able to entirely mitigate QVIO blowups with a single sensor?
2. Would a second tracking camera be able to run a separate instance of QVIO? This would be the idea situation since it seems the easiest.
3. Would a non-tracking camera doing AprilTag detection, then relocalizing these detections and providing odometry data directly, be a reasonable idea? Eg my stereo cam, which isn't suitable for QVIO (bummer), could deduce position from known locations of detected tags and compute NED-frame odometry.
4. Is there existing work in this area (redundant vision on Voxl) or other ideas I should look into?

Thanks so much!

[Chad Sweet]

This is a big topic. We are constantly working to improve performance in multiple ways. We hope to have a multi-sensor VIO system working in a few months, but it is a lot of work.

You can definitely use the stereo sensor for VIO though, you just can't yaw as quickly. You can also replace the default stereo sensors with fisheye sensors found here. If you use depth from stereo you will of course need to recalibrate, etc.

[Evan Hertafeld]

Thanks for the reply! We'll definitely stay tuned. Curious about your thoughts on my specific approach - namely, are there enough resources to run 2 simultaneous instances of the VIO system?

[Chad Sweet]

Yes, but it's a lot of math to get them merged properly.

[Evan Hertafeld]

I was thinking of something simple, eg:

Use tracking cam qvio until reported quality drops below a particular value (near blowup). If it does, switch to stereo qvio feed as long as the position + velocity were recently in agreement within some threshold and its reported quality is sufficient. If not, switch to manual mode and call an adult. Once the main qvio recovers, store the position offset (for future switchovers) and switch back.

I'm sure proper fusion will obtain a smoother and more robust result, but I'd just like something that can handle a blowup without crashing.