@Myles-Levine , if you are masking out 90% of down-facing camera, then it is probably useless any movement of the features will get it out of the unmasked region very easily and feature will be dropped.. It may hurt VIO as there may be features going in and out, only trackable for a few frames (just adds to the complexity).
You should try to move the down-facing camera to free up it's FOV or even have it facing slightly angled to the back of the drone..
Alex