@blakekrpec As I understand the ECL/EKF library integration so far, you can run several instances that each will fuse different sensor combinations (with individual blending) to accommodate for estimation state errors and the like....

Have you followed the mavros tutorial here?

Hello, they are M1.6 x 6mm.

@Prasad-Antervedi Try following these: https://docs.modalai.com/calibrate-cameras/

Camera server has auto-exposure enabled by default on all the cameras, you can verify this by looking at /etc/modalai/voxl-camera-server.conf and by running voxl-inspect-cam hires_preview and shading the lens with your hand, watching the exposure go up and down.

@Alex-Gardner Worked, thanks!

Not modalai, but something that helped me debug this very same issue was installing voxl-portal, and using that to view the qvio stream, which will overlay the points onto the tracking camera screen. If it creates points at your drone's motors and feet, then I presume it would be a problem

Found the repository that stores the code here https://gitlab.com/voxl-public/old-projects/voxl-hal3-tof-cam-ros. If this link could be placed in the documentation for future viewers, that would be great

Thanks for printing out those resolutions. Very helpful! yocto:~$ voxl-camera-server -d 1 exising instance of voxl-camera-server found, attempting to stop it ------ voxl-camera-server INFO: Port J2 ------ voxl-camera-server: Done configuring hires camera Port : port_J2 Name : hires Enabled : 1 Type : hires Api : hal3 P-W : 640 P-H : 480 P-Fmt : nv21 FPS : 30 OverrideId : -1 AEAlgo : isp ------ voxl-camera-server INFO: Port J3 ------ voxl-camera-server: Done configuring stereo camera Port : port_J3 Name : stereo Enabled : 1 Type : stereo Api : hal3 P-W : 1280 P-H : 480 P-Fmt : nv21 FPS : 30 OverrideId : -1 AEAlgo : modalai =================MODALAI Auto Exposure Settings================== gain_min: 0 gain_max: 1000 exposure_min_us: 100 exposure_max_us: 33000 desired_msv: 58.000000 k_p_ns: 32000.000000 k_i_ns: 20.000000 max_i: 250.000000 p_good_thresh: 3 exposure_period: 1 gain_period: 2 display_debug: no exposure_offset_for_gain_calc: 8000 ================================================================= ------ voxl-camera-server INFO: Port J4 ------ voxl-camera-server: Done configuring tracking camera Port : port_J4 Name : tracking Enabled : 1 Type : tracking Api : hal3 P-W : 640 P-H : 480 P-Fmt : raw8 FPS : 30 OverrideId : -1 AEAlgo : modalai =================MODALAI Auto Exposure Settings================== gain_min: 0 gain_max: 1000 exposure_min_us: 100 exposure_max_us: 33000 desired_msv: 58.000000 k_p_ns: 32000.000000 k_i_ns: 20.000000 max_i: 250.000000 p_good_thresh: 3 exposure_period: 1 gain_period: 2 display_debug: no exposure_offset_for_gain_calc: 8000 ================================================================= Writing new configuration to /etc/modalai/voxl-camera-server.conf Created pipe: hires_preview channel: 0 Created pipe: stereo channel: 1 Created pipe: tracking channel: 2 Starting Camera: tracking SUCCESS: Camera module opened ----------- Number of cameras: 3 ------ voxl-camera-server INFO: Client: qvio-server0 connected to channel: 2 ------ voxl-camera-server ERROR: Client connected to invalid channel: 0 ------ voxl-camera-server ERROR: Client connected to invalid channel: 1 ----------- HiRes camera id: 0 ----------- Stereo camera id: 2 ----------- Mono camera id: 1 ------ voxl-camera-server INFO: Client: voxl-streamer0 connected to channel: 2 Available resolutions for camera: tracking: 640 x 480 320 x 240 Resolution: 640 x 480 Found! ------ voxl-camera-server: Preview buffer allocations Internal Exposure for: tracking 2 /builds/voxl-public/modal-pipe-architecture/voxl-camera-server/src/expgain_interface/expgain_interface_factory.cpp 45 Creating exposure interface Creating ModalAI exposure interface Port : port_J4 Name : tracking Enabled : 1 Type : tracking Api : hal3 P-W : 640 P-H : 480 P-Fmt : raw8 FPS : 30 OverrideId : -1 AEAlgo : modalai =================MODALAI Auto Exposure Settings================== gain_min: 0 gain_max: 1000 exposure_min_us: 100 exposure_max_us: 33000 desired_msv: 58.000000 k_p_ns: 32000.000000 k_i_ns: 20.000000 max_i: 250.000000 p_good_thresh: 3 exposure_period: 1 gain_period: 2 display_debug: no exposure_offset_for_gain_calc: 8000 ================================================================= Starting Camera: hires Available resolutions for camera: hires: 4160 x 3120 4056 x 3040 4000 x 3000 3040 x 3040 3016 x 3016 3840 x 2160 3648 x 2736 3264 x 2448 3200 x 2400 2976 x 2976 3044 x 1720 2704 x 2028 2704 x 1520 2592 x 1944 2688 x 1512 2028 x 1144 2160 x 2160 1920 x 2160 1920 x 1920 1880 x 1880 2048 x 1536 1920 x 1440 1920 x 1080 1600 x 1600 1600 x 1200 1520 x 1520 1440 x 1080 1080 x 1080 1280 x 960 1340 x 760 1280 x 768 1280 x 720 1200 x 1200 1280 x 640 1280 x 480 1040 x 780 1024 x 768 960 x 960 720 x 720 800 x 600 960 x 720 848 x 480 858 x 480 864 x 480 800 x 480 720 x 480 640 x 480 640 x 240 640 x 360 480 x 640 480 x 480 480 x 360 480 x 320 432 x 240 352 x 288 320 x 240 Resolution: 640 x 480 Found! ------ voxl-camera-server: Preview buffer allocations Port : port_J2 Name : hires Enabled : 1 Type : hires Api : hal3 P-W : 640 P-H : 480 P-Fmt : nv21 FPS : 30 OverrideId : -1 AEAlgo : isp Starting Camera: stereo Available resolutions for camera: stereo: 1280 x 480 848 x 480 858 x 480 864 x 480 800 x 480 720 x 480 640 x 480 640 x 240 640 x 360 480 x 480 480 x 360 480 x 320 432 x 240 352 x 288 320 x 240 Resolution: 1280 x 480 Found! ------ voxl-camera-server: Preview buffer allocations Internal Exposure for: stereo 1 /builds/voxl-public/modal-pipe-architecture/voxl-camera-server/src/expgain_interface/expgain_interface_factory.cpp 45 Creating exposure interface Creating ModalAI exposure interface Received raw10 frame, checking to see if is actually raw8 Frame was actually 8 bit, sending as is Port : port_J3 Name : stereo Enabled : 1 Type : stereo Api : hal3 P-W : 1280 P-H : 480 P-Fmt : nv21 FPS : 30 OverrideId : -1 AEAlgo : modalai =================MODALAI Auto Exposure Settings================== gain_min: 0 gain_max: 1000 exposure_min_us: 100 exposure_max_us: 33000 desired_msv: 58.000000 k_p_ns: 32000.000000 k_i_ns: 20.000000 max_i: 250.000000 p_good_thresh: 3 exposure_period: 1 gain_period: 2 display_debug: no exposure_offset_for_gain_calc: 8000 ================================================================= ------ voxl-camera-server: Camera server is now running

@colerose we selected mobilenetv2 because of its exceptional performance on embedded devices - currently, inference time in the voxl-tflite-server is ~22ms per frame with very high precision. Also, the mobilenet family is fairly easy to retrain with a custom dataset!

Hi @Ador , Unfortunately it's not that straight forward. Most of the code won't be portable, the first candidates to be ported would be from here: https://gitlab.com/voxl-public/core-libs The system image (what Yocto is building) has many hardware dependent items, and we aren't pulling the application layer in during the bitbake process, rather later on we install on top. As an example, part of our software stack run on Hexagon DSP processor (a MAVLink parser) which wouldn't exist on RPi etc so it wouldn't work.

https://forum.modalai.com/topic/460/integrating-flir-boson-as-usb-connected-camera-on-voxl/3 Issue with UVC cameras in general has moved here

That should work fine, keep in mind that you should change the extrinsic with relation to imu0 as well, and that you may need to change the translation if you're not able to rotate it perfectly in place. VIO will converge internally to the actual rotation/translation values as long as they're close enough, but it can lead to errors if they're not.

Oh thank you! I will try to flash this new image. Thanks for the fast response.

Solution: sometimes when just binding the RC controller, the inputs aren't shown. Make sure to restart drone after rebinding. Additionally, I've found different controllers have different lever/switch inputs, so its good to use QGC to look for the responsive switches and assign the ones you want accordingly

Thank you! Very helpful. I figured out where I had seen the checklist, it was available for download in an email you folks sent out July 20th.

Hi @simo , That's likely a question for NVIDIA We can help you with our companion computer though! In general, you'll want to get a 3.3V TTL port on the NVIDIA side, setup the correct baud rate, and then use their mavros software). Thanks! Travis

@Chad-Sweet The fix in the dev branch successfully populates the 0th degree bin for obstacle detection. We are now able to run VOA mostly successfully, with one observable bug with the voxl-camera-server/voxl-dfs-server. Please see this post for details: https://forum.modalai.com/topic/450/intermittent-crashing-of-voxl-camera-server-during-voa-testing?_=1631837674205

Hi Chad, I tried a different windows ssh software and it worked fine. I could ssh into the VOXL using "Termius". It could be an issue of encryption or inherent setup. Thanks,

This sounds similar to the R/C passthrough instructions for the gimbal found here