How much can cameras be obscured?

Alex Kushleyev

You should not have any unmasked static objects appear in the camera view because they may have features that will be tracked and will confuse the algorithm because these features will be stationary with respect to the drone even if the drone is moving.

open vins should support masking parts of the camera(s) to ignore any features in this region. I am not sure how to enable the mask, will double check with colleagues (you could also do a search..).

worst care scenario, the part of the image to be ignored should be blacked out before feeding into open vins - that would be the solution if the mask is not supported. Hopefully that is not needed and mask can be used.

Alex

Myles Levine

@Alex-Kushleyev Thanks Alex. I did a search for masking and found this:
use_mask Masking configuration moved to OpenVINS YAML configs

But I have been unable to find more information about YAML or masking within it. Do you have instructions somewhere for this?

zauberflote1

@Myles-Levine
Hello There,

Please check your SDK version, VINS is under constant development, so always prefer the latest tag available.

Regarding occlusion during takeoff, please check your /etc/modalai/vio_cams.conf, if you have the occlusion flag during takeoff enabled, I would attempt disabling and confirm if the behaviour persists...

Regarding Arbitrary mask, please refer to this past forum question, if you have any further questions, please don't hesitate to post.

All the best,
ZBFT

zauberflote1

@zauberflote1
You could also disable the bottom cam in vio_cams.conf for VIO only

Myles Levine

@zauberflote1 Hi, we figured out how to create and import masking files. I created 2 masking files, one each for the front and bottom tracking cameras. I tested it by removing the payload but keeping the masking files in place, then trying to fly a simple script (go up 1 meter, hover 10 seconds, land). However, the drone is exhibiting inconsistent behavior, sometimes rising far too high and running into the ceiling of the cage around it, another time it got to the right height then immediately started slowly lowering before hitting the ground and trying to go up again.

The front mask file only masks out a shape about 20% of the pixels, but the bottom masking file masks about 90% of the pixels. Could the drastic masking be causing this?

zauberflote1

@Myles-Levine Absolutely, you're forcing the tracker to reduce the tracking continuity window, less persistent features -> SLAM feats drop.

Perhaps, you could try reducing the spacing between features in the estimator_config yaml (track opts) to bump MSCKF feats and potentially SLAM feats; however, these are advanced settings. Please exercise your best judgment when changing this, modifications are not recommended for general/default use-case

Myles Levine

@zauberflote1 Is there an approximate cutoff for how much can be masked before performance starts getting iffy? I have the ability to modify the payload to allow more camera view, but the more I open it, the worse it is for the application. So, finding the minimum opening is helpful, and I'm hoping that I don't have to just try different sized masks and test until it becomes reliable.

Myles Levine

@zauberflote1 Also, thank you for your quick responses!!!

zauberflote1

@Myles-Levine
Performance is evaluated with no external masks -- as we don't ship them. With masks, the answer is very dependent on use-case. Honestly, based on practical tests, it might be better to fly single-cam (full-view) with tight params for robustness than to fly multi-cam with obstructed view. Furthermore, you could try changing the location of your cameras, and doing an extrinsic IMU-CAM calibration with our Kalibr setup (guides available in past answered questions)

Alex Kushleyev

@Myles-Levine , if you are masking out 90% of down-facing camera, then it is probably useless any movement of the features will get it out of the unmasked region very easily and feature will be dropped.. It may hurt VIO as there may be features going in and out, only trackable for a few frames (just adds to the complexity).

You should try to move the down-facing camera to free up it's FOV or even have it facing slightly angled to the back of the drone..

Alex