Posts made by MattK

@dlee Ah, no I have not. I'll give that a try when I get the chance. Thanks!

@Moderator Oh... interesting. You might want to remove this bullet point from its sales page then. https://www.modalai.com/products/starling-2-max?variant=48172375310640

Would you be able to recommend a drone with a working VOA solution? We're interesting in getting one to get a baseline for the quality we can expect from VOXL2's built in VOA SDK. I'm particularly interested in assessing depth-from-stereo, I've gotten good enough results for now from the TOF v2 sensor. But TOF recommendations are welcome as well.

Thanks, @Alex-Kushleyev .

If that's the case, then what kind of VOA solution is present on the Starling 2 Max?

@James-Strawson Hmmm, unfortunate typo. Does the Starling Max 2 use its dual IMX412 color image sensors to get depth-from-stereo, in order to do outdoor VOA? I wasn't too impressed with the quality of outdoor DFS that I got from the M0015, but perhaps the IMX412s are better?

Appreciate the extra help/guidance on TOF processing, I'll try take a look. Even though the outdoor use case is not ideal, I'm still leaning towards trying to make the TOF sensors work for us, as the results I'm getting still seem more promising than what I was getting from stereo.

So I actually was able to filter out the noise pretty well by adding a temporal filter to confidence and depth values in PerCameraMgr::RoyaleDataDone in hal3_camera_mgr.cpp.

^ I used an exponential moving average with alpha = 0.25. After updating the EMA buffer, I cleaned things up further by reducing all confidence values below an ~80% threshold (200/255) to zero,

That seems to be working pretty well at reducing/ignoring noise artifacts from sunlight while still being able to see large objects.

I've only tested it out on the ground so far though, and expect to encounter reliability issues in the air when everything is in motion. I've already noticed that when I walk around in frame, I ghost and become lost in the noise due to the temporal nature of the filter. I'll probably tweak things more and try to experiment with median/spatial filters as well, but for now this at least seems like it should work for things like large trees, buildings, and the ground, when the drone is flying slowly.

Still interested to know more about how outdoor obstacle avoidance is addressed with the Starling Max v2.

Hey there,

I've been testing out the new ToF v2 sensor and have been encountering a lot of noise when trying to use it outdoors in the sun.

Attaching some depth image screenshots of the noise that I'm referring to. In both images, you can see me standing on ground, with a bunch of staticky noise all over the place.

I'm not surprised that noise gets introduced when sunlight is added to the equation, but I also noticed that the new Starling Max v2, which is advertised as an outdoor drone, claims to rely on this same ToF sensor for obstacle avoidance:

The Starling Max v2 also has dual color image sensors, so I'm wondering if this was a misprint and it actually uses depth-from-stereo for outdoor obstacle avoidance. Either that, or maybe my ToF sensor is damaged?

I've started to work on implementing my own filtering to voxl-camera-server (looks promising so far, but still more work to do), but I wanted to sanity check myself in case something else is wrong.

@Alex-Kushleyev

Oh, very fascinating. Thank you for the update!

I'd actually somewhat given up on DFS and started testing your new ToF beta product, but when I can I'll try boot my DFS rig back up and see if the fix helps.

Edit: Nevermind, I just checked and my DFS rig is on SDK v1.1.2, so this fix shouldn't be applicable. I might still try it out just in case, though.

Thanks @Alex-Kushleyev ,

Here's the summaries from the successful intrinsic/extrinsic calibrations that I've been using:

I tried doing the monocular calibration process and it worked for the intrinsic calibrations, but when I tried to then do the normal extrinsic calibration it no longer worked.

Monocular intrinsic calibration successes:

Following stereo extrinsic calibration failures:

Is this the process you had in mind? The re-projection error and distance between cameras calculations are so off that I figured something must be inherently wrong. At first I thought that you wanted me to try and calibrate each stereo camera individually (stereo_front_L, stereo_front_R, stereo_rear_L, stereo_rear_R), but there didn't seem to be an option for that.

Might be worth noting that my stereo_front cameras are still incorrectly rotated in config, which might have something to do with why it was mistakenly identified as a vertical pair (you gave me the solution to this one here, I just haven't gotten around to fixing it yet). Though I wasn't getting that problem before, so it still seems to me that I'm just not doing what you envisioned.

And yes, please let me know if you find anything out regarding if the field of view offset could be effecting the quality of DFS.

I've been able to calibrate my stereo cameras properly (reprojection errors of 0.25-0.5, in the green), but I always notice an offset between the left and right cameras that would indicate that they are not horizontally aligned.

While digging around the forums, I came across this old thread where someone was having trouble with calibration and @Dobry-Kolacz advised to check to see if the tops of each camera's views align. I tried it for myself and discovered that there was a significant gap:

We have a bunch of M0015 stereo cameras in inventory. I've checked several of them and they all have similar offsets. Same thing occurs when I try to align with the bottom of the camera views.

I've been testing out depth-from-stereo for VOA and have been noticing way more false positives than I'd like. I've been playing around with filters and thresholds but it feels like a band-aid rather than an actual solution (in order to eliminate false positives, I have to restrict things so much that it starts ignoring true positives).

I wonder if this offset is hurting the quality of my DFS, and if there is any way to fix it? According to the old thread, it seems that it may be due to an issue with the internal construction of the image sensor being offset to its lens. It's concerning to me that this difference is observed on all the pairs that I've checked so far, despite my efforts to mount them perfectly each time. Should I start trying to mix and match individual sensors to make new pairs that match better?

@Alex-Kushleyev you were right! The stereo_front pair did have en_rotate set to true. Thanks!

I have an inverted custom setup where my OV7251 stereo cameras are mounted upside-down, as pictured below.

Setup Diagram: (By the way, is this really how L/R pairs are? Seems weird to me. I concluded this based on error responses I got from the calibration process.)

Strangely, in the web portal, my stereo_front camera feeds display as being right-side up, while my stereo_rear camera feeds are flipped upside-down.

Conversely, when I look at my voa_pc_out pointcloud, the cloud coming from the stereo_front cameras appears to be inverted and the stereo_rear cameras appear to be oriented correctly (matching what I expect based on the VOXL2 board's reference frame).

This leads me to think that my stereo_rear cameras are actually configured correctly. The image outputs are upside-down because that's what they are in reality, and the vision-hub is correctly processing that based on the configured extrinsics to output a correctly oriented pointcloud.

My Extrinsics File:

I've looked over my extrinsics file a few times to look for rotation matrix errors, but I don't see anything wrong and it passes the camera calibration process with flying colors. Am I missing something?

I also looked and found this thread from a few weeks ago that instructs on swapping some files in /usr/bin/camera that might fix my problem. However, I still have confusion due to my front and rear stereo pairs exhibiting different behavior. Should I replace all my com.qti.sensor.ov7251_combo files with their _flip alternates or just the ones related to my front or rear stereo pair?

After I swap files, I'm guessing that I then need to redo the rotation matrixes related to that stereo pair?

My /usr/lib/camera directory:

Any help here is extremely appreciated. I can give more details as well. Thank you!

@Chad-Sweet Great to hear, that's exactly what I was hoping for. Thank you!

Is the VOXL2's VIO/VOA functionality really not possible with ArduCopter?

From their documentation it seems that ArduCopter has VIO compatibility with the VOXL products, but specifically not VOXL2. I wonder why that is the case. Is it not sending the same MAVLink odometry packets to the flight controller? (I'm using a Pixhawk with ArduCopter 4.3.7)

VOA is more unclear to me. Does any VOXL product have working VOA compatibility with ArduCopter?

I'm fine with digging into it more to try and hack things together if needed as I like the VOXL2 and would like to order more for our fleet, but if it's a lost cause I don't want to waste a bunch of engineering time on it.

Any additional information/clarity is helpful. Thank you!

I saw a similar topic posted but it was asking about the voxl-modem driver, whereas I'd like to know about the actual firmware on the Quectel 5G modem.

I'm sourcing some RM502QAEAA-M20-SGASA 5G modems directly from Quectel but I'm unsure if their firmware version out of the box is compatible with the VOXL2 ecosystem.

I'm wondering is if there is a way to check the firmware version of the Quectel 5G modem using the VOXL2, and if there is a to upgrade/change it if I run into any issues?

Thank you

Wondering how this is setup by default. I'd like to try and use the VOXL2 to both do Sense and Avoid, as well as some object detection. If Sense and Avoid is able to run solely on the CPU, then I figured I would have the entire GPU available for object detection. Is this a feasible plan?

Thank you!