Bad DFS disparity

[afdrus]

Hi everyone,

I am using a Sentinel (1) drone upgraded to the SDK 1.1.0. I was trying to get the disparity out of the frontal stereo however the results are quite bad, please see below the output of the sentinel (greyscale) vs the output of a an intel realsense (colored). For reference the object in view is a library.

I tried to play around with the parameters in voxl-dfs-server, but I can't get a decent result.

1. Do you have any suggestions? I have a feeling that it might be something macroscopic like: camera mounting, stereo out of focus or bad calibration (although I repeated it quite a lot).
2. Could you please post a good disparity taken with one of your stereo setup along with the respective grey scale stereo images? This would help to have a benchmark that I can compare with?

Thank you in advance for your response.

[A Former User]

@afdrus

Sorry to hear you're having issues with DFS disparity, I may be able to help you out. We changed some things in the DFS server for SDK 1.1.0 so I'm wondering if that's creating an issue. Would you mind installing an older SDK (like SDK 0.9) and seeing if that gives better quality? If so, I may need to revisit the changes I made. If that doesn't fix it, bad calibration often creates issues for DFS so I'd check there first.

I don't have a good DFS image on-hand but you can check out our documentation page for an example.

Let me know how this goes!!

Thomas Patton

[afdrus]

@thomas Thank you for your response, On the SDK 0.9 I have the same issue. I tried to re-focus the cameras and re-calibrate them and the dfs has improved maybe slightly:
no parameter tuning

post_median_filter=15

post_median_filter=15, blur_filter=5

post_median_filter=15, blur_filter=5, dfs_pair_0_cost_threshold=80

However, I am not able to get a uniform disparity as the one that I would get from the intel sensor for example, there are always a lot of holes over the objects in the scenes.

Is there a way to fine tune this? Maybe a parameter in voxl-dfs-server that I did not yet tweaked?
Could this be related to other macroscopic issues?

Thank you in advance for your response!

[A Former User]

@afdrus

Can I ask what model of Intel Realsense you're using?

Thomas

[afdrus]

@thomas Sure, the model that I am using is the D455.

I was just trying to understand if the disparity that I am getting from voxl-dfs-server (now) is the best that I can get (or at least not too far from the best that I can get). This is why I asked if you could post a good disparity taken with one of your stereo setup. The one in the website https://docs.modalai.com/voxl-dfs-server-0_9/ looks very similar to what I am getting ATM: good disparity along the edges of the objects, but no information in uniform areas.

Thanks again for the reply!

[A Former User]

@afdrus

Yeah, for DFS we actually invoke a lower-level library to compute it for us so it's kind of a black box in the sense that we don't have a lot of control over the algorithmic process. If you tried out DFS Server on SDK 0.9 (like in the docs page) and the result wasn't good there isn't a whole lot I can say. Definitely re-check your camera calibration, camera mounting, and focus - these things all play a big role in the result.

Sorry this isn't that informative of an answer, hope it at least helps some!

Thomas Patton

[Moderator]

@thomas depth from stereo works by correlating features across two image sensors. Against a uniform surface, such as a white wall, it will never generate depth. You need an active sensor, like TOF or LiDar, to measure depth of a flat, white wall

[afdrus]

@Moderator thank you for your response!
I agree with what you suggest, but we are not discussing about the computation of the disparity in an edge case scenario of a white wall. I am just confused, for instance take the example below: why the voxl-disparity is so sparse (first top picture) with respect to the other done via opencv (second bottom picture)?

top-disparity

bottom-disparity

In both cases I am using the same grey images, with the same extrinsics/intrinsics parameters, and yet the disparity from opencv is able to fill in the areas within the edges, whereas voxl-disparity is not. Are there any parameters that I can tune to get the top-disparity to look more similar to the bottom-disparity?

[A Former User]

@afdrus

voxl-dfs-server has a couple adjustable parameters, you can check them out in /etc/modalai/voxl-dfs-server.conf. But yeah, not everything is tweakable as we're using a hardware-optimized API to compute these images. If you're happier with the OpenCV ones, you could always just fork voxl-dfs-server and make a small change to the implementation to have it computed in OpenCV instead of with the usual libmodal-cv call. I can help you out with this if you'd like.

Hope this helps!

Thomas Patton

[afdrus]

@thomas Thank you for your response! That would be great, could you please show me a bit more precisely where can I start to edit the code to make this adjustments?

[A Former User]

@afdrus

Assuming you're on VOXL2, start by forking the DFS Server repo here. Then take a look at the _image_callback function here. This defines what happens when each image comes in. These lines extract that into a left/right OpenCV image which you can then use to compute DFS. Then you should push out the results over some pipe using a similar format to what we do, consult the libmodal-pipe documentation if you need any help with this.

Hope this helps, let me know if you have any other questions!

Thomas Patton

[afdrus]

@thomas Thanks a lot for the support! I'll dig into it and go back to you in case something is not clear!

[A Former User]

@afdrus

Sounds good, let me know if I can help in any way!