How much can cameras be obscured?
-
Hello,
I am trying to attach a payload to a Starling 2 Max. I started having flight control issues (specifically the drone shooting higher than its designated hovering height, and going in random directions instead of down for landing), and tracked it back to VIO and obscuring some of the forward cameras' downward view. Which view is critical for flight stability, the tracking camera or the other camera? Approximately how much of the view of the relevant camera can be obscured before it impacts flight stability?
You should not have any unmasked static objects appear in the camera view because they may have features that will be tracked and will confuse the algorithm because these features will be stationary with respect to the drone even if the drone is moving.
open vins should support masking parts of the camera(s) to ignore any features in this region. I am not sure how to enable the mask, will double check with colleagues (you could also do a search..).
worst care scenario, the part of the image to be ignored should be blacked out before feeding into open vins - that would be the solution if the mask is not supported. Hopefully that is not needed and mask can be used.
Alex
-
You should not have any unmasked static objects appear in the camera view because they may have features that will be tracked and will confuse the algorithm because these features will be stationary with respect to the drone even if the drone is moving.
open vins should support masking parts of the camera(s) to ignore any features in this region. I am not sure how to enable the mask, will double check with colleagues (you could also do a search..).
worst care scenario, the part of the image to be ignored should be blacked out before feeding into open vins - that would be the solution if the mask is not supported. Hopefully that is not needed and mask can be used.
Alex
@Alex-Kushleyev Thanks Alex. I did a search for masking and found this:
use_mask Masking configuration moved to OpenVINS YAML configsBut I have been unable to find more information about YAML or masking within it. Do you have instructions somewhere for this?
-
@Alex-Kushleyev Thanks Alex. I did a search for masking and found this:
use_mask Masking configuration moved to OpenVINS YAML configsBut I have been unable to find more information about YAML or masking within it. Do you have instructions somewhere for this?
@Myles-Levine
Hello There,Please check your SDK version, VINS is under constant development, so always prefer the latest tag available.
Regarding occlusion during takeoff, please check your /etc/modalai/vio_cams.conf, if you have the occlusion flag during takeoff enabled, I would attempt disabling and confirm if the behaviour persists...
Regarding Arbitrary mask, please refer to this past forum question, if you have any further questions, please don't hesitate to post.
All the best,
ZBFT -
@Myles-Levine
Hello There,Please check your SDK version, VINS is under constant development, so always prefer the latest tag available.
Regarding occlusion during takeoff, please check your /etc/modalai/vio_cams.conf, if you have the occlusion flag during takeoff enabled, I would attempt disabling and confirm if the behaviour persists...
Regarding Arbitrary mask, please refer to this past forum question, if you have any further questions, please don't hesitate to post.
All the best,
ZBFT@zauberflote1
You could also disable the bottom cam in vio_cams.conf for VIO only -
@zauberflote1
You could also disable the bottom cam in vio_cams.conf for VIO only@zauberflote1 Hi, we figured out how to create and import masking files. I created 2 masking files, one each for the front and bottom tracking cameras. I tested it by removing the payload but keeping the masking files in place, then trying to fly a simple script (go up 1 meter, hover 10 seconds, land). However, the drone is exhibiting inconsistent behavior, sometimes rising far too high and running into the ceiling of the cage around it, another time it got to the right height then immediately started slowly lowering before hitting the ground and trying to go up again.
The front mask file only masks out a shape about 20% of the pixels, but the bottom masking file masks about 90% of the pixels. Could the drastic masking be causing this?
-
@zauberflote1 Hi, we figured out how to create and import masking files. I created 2 masking files, one each for the front and bottom tracking cameras. I tested it by removing the payload but keeping the masking files in place, then trying to fly a simple script (go up 1 meter, hover 10 seconds, land). However, the drone is exhibiting inconsistent behavior, sometimes rising far too high and running into the ceiling of the cage around it, another time it got to the right height then immediately started slowly lowering before hitting the ground and trying to go up again.
The front mask file only masks out a shape about 20% of the pixels, but the bottom masking file masks about 90% of the pixels. Could the drastic masking be causing this?
@Myles-Levine Absolutely, you're forcing the tracker to reduce the tracking continuity window, less persistent features -> SLAM feats drop.
Perhaps, you could try reducing the spacing between features in the estimator_config yaml (track opts) to bump MSCKF feats and potentially SLAM feats; however, these are advanced settings. Please exercise your best judgment when changing this, modifications are not recommended for general/default use-case
-
@Myles-Levine Absolutely, you're forcing the tracker to reduce the tracking continuity window, less persistent features -> SLAM feats drop.
Perhaps, you could try reducing the spacing between features in the estimator_config yaml (track opts) to bump MSCKF feats and potentially SLAM feats; however, these are advanced settings. Please exercise your best judgment when changing this, modifications are not recommended for general/default use-case
@zauberflote1 Is there an approximate cutoff for how much can be masked before performance starts getting iffy? I have the ability to modify the payload to allow more camera view, but the more I open it, the worse it is for the application. So, finding the minimum opening is helpful, and I'm hoping that I don't have to just try different sized masks and test until it becomes reliable.
-
@Myles-Levine Absolutely, you're forcing the tracker to reduce the tracking continuity window, less persistent features -> SLAM feats drop.
Perhaps, you could try reducing the spacing between features in the estimator_config yaml (track opts) to bump MSCKF feats and potentially SLAM feats; however, these are advanced settings. Please exercise your best judgment when changing this, modifications are not recommended for general/default use-case
@zauberflote1 Also, thank you for your quick responses!!!
-
@zauberflote1 Is there an approximate cutoff for how much can be masked before performance starts getting iffy? I have the ability to modify the payload to allow more camera view, but the more I open it, the worse it is for the application. So, finding the minimum opening is helpful, and I'm hoping that I don't have to just try different sized masks and test until it becomes reliable.
@Myles-Levine
Performance is evaluated with no external masks -- as we don't ship them. With masks, the answer is very dependent on use-case. Honestly, based on practical tests, it might be better to fly single-cam (full-view) with tight params for robustness than to fly multi-cam with obstructed view. Furthermore, you could try changing the location of your cameras, and doing an extrinsic IMU-CAM calibration with our Kalibr setup (guides available in past answered questions) -
@Myles-Levine
Performance is evaluated with no external masks -- as we don't ship them. With masks, the answer is very dependent on use-case. Honestly, based on practical tests, it might be better to fly single-cam (full-view) with tight params for robustness than to fly multi-cam with obstructed view. Furthermore, you could try changing the location of your cameras, and doing an extrinsic IMU-CAM calibration with our Kalibr setup (guides available in past answered questions)@Myles-Levine , if you are masking out 90% of down-facing camera, then it is probably useless any movement of the features will get it out of the unmasked region very easily and feature will be dropped.. It may hurt VIO as there may be features going in and out, only trackable for a few frames (just adds to the complexity).
You should try to move the down-facing camera to free up it's FOV or even have it facing slightly angled to the back of the drone..
Alex
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login