Posts made by Alex Kushleyev

Hello @Teon,

This is more of a general question, not specifically related to VOXL2. However, here is some information that you might find helpful.

Base station

• Raspberry Pi
• ZED-F9P receiver with dual band helical antenna, connected to RPI via USB
• configure ZED-F9P to be a in base station mode (will require data collection for localizing the base station precisely). Check Ublox docs
• configure ZED-F9P to output RTCM3 corrections over usb to RPI
• run NTRIP caster software (you will need to find one..) which will connect to USB port and serve the RTCM3 data to the clients via NTRIP protocol
  -- this would require an NTRIP client on the client side
• (alternatively) publish the RTCM data via Mavlink directly to the drone (MAVSDK).
• would need a client that would connect to Ublox via USB and send out mavlink messages to your drone
  -- https://mavsdk.mavlink.io/main/en/cpp/api_reference/classmavsdk_1_1_rtk.html
  -- https://github.com/mavlink/MAVSDK/issues/2352

Rover (VOXL2)

• wifi connection to the base station server
• based, on the following docs (https://docs.px4.io/main/en/advanced/rtk_gps) the PX4 gps driver will automatically forward RTCM packets to the attached GPS receiver. The Px4 GPS driver will need to receive the Mavlink GPS_RTCM_DATA packet.
• if you use the NTRIP caster in the step above, you could use a NTRIP client and have the client publish the mavlink GPS_RTCM_DATA message locally on VOXL2 .

I will ask around to see if we have a better documentation for this..

Alex

@mark , we have seen this issue when using 10- or 12-bit drivers for the AR0144 tracking cameras. These drivers are shipped with the SDK but are not stable (for this exact reason).

Can you please confirm which ar0144 sensormodules you are using - you can check in /usr/lib/camera.

Alex

Hi @Rowan-Dempster,

We have been looking at some optimizations to help reduce the overall cpu usage of the camera server (not yet in the SDK). Let me test your exact use case and see what can be done.

Just FYI, we recently added support for sharing ION buffers directly from camera server, which means the camera clients get the images using zero-copy approach. this allows to save the cpu cycles wasted on sending the image bytes over the pipe, especially when there are multiple clients.

If you would like to learn more how to use the ION buffers, I can post some examples soon. One the client side, the API for receiving and ION buffer vs regular buffer is almost the same. One thing that will be different is that the shared ION buffer has to be released by all clients before it can be re-used by the camera server (which makes sense).

Even without the ION buffer sharing there is room to reduce the cpu usage, so I will follow up after testing this a bit. Regarding your question whether sending the image to multiple clients should not cause significant extra cpu usage -- yes you are correct, ideally it should not. However, the reason why it is happening here is related to how we set up ION buffer cache policy and currently when the CPU accesses the buffers for the misp_norm images (coming from the gpu), the cpu reads are not cached and the read access is expensive. Reading the same buffer multiple times results in repeated CPU-RAM access (for the data that would normally be already fully cached after the first send to the client pipe). However, in some other cases (when the buffer is not used by the cpu, but is shared as ION buffer and client sends the buffer directly to GPU), this approach results in even lower CPU usage. So i think we need to resolve the buffer cache policy based on the use case.. More details will come soon..

Alex

@Seungjae-Baek , the resolution of the camera should depend on the use case. However, it is also important to keep in mind what exactly you are doing with the images coming from the hires cameras. For example, if you set the resolution to 4K and you try to view those uncompressed images using voxl-portal, this process will be very taxing on the CPU for the following reason: when you view uncompressed images in voxl-portal, these images are encoded with a software JPG encoder and then sent from voxl2 to your browser. This process is very cpu-heavy. On the contrary, if you use h264 / h265 stream, it should be perfectly fine to encode 4K30 video to disk or even stream, since H264 / H265 encoding is done by a hardware encoder.

voxl-portal, actually does support showing h264 (but not h265) 30FPS streams, so that would be a lot more efficient for the CPU, since there would be no jpg encoding. Otherwise, if you are using raw frames (not _encoded) in voxl-portal , please keep in mind that you will always have a lot of cpu overhead. voxl-portal is designed for debugging / development purposes, so it's not necessarily the most efficient solution for video streaming. For real video streaming use cases, you would use h264 or h265 encoding and save to disk on voxl2 + stream the encoded video for remote viewing. You could encode the same camera source with two different resolutions / codecs / bitrates.

If you need help setting up a specific use case, please provide some details and I can help you further.

Alex

@cbay ,

We support any camera that is compatible with UVC (plugged into a usb port). Please look into voxl-uvc-server. https://docs.modalai.com/voxl-uvc-server/

Regarding NIR, have you considered using a regular camera (most cameras are sensitive to NIR) and install a lens with a NIR pass filter (if you dont want to see visible light)? Then you could use one of our hi-res cameras like IMX412 or IMX664. I can get you spectral response plot of those sensors if you need.

Alex

@Aaky , 5A hover / 12A max (per motor) should be easy for M0129 in terms of current capacity. Just make sure you have some cooling. Sufficient cooling should be determined by testing and monitoring ESC temps, which are available in the ESC telemetry and PX4.

Alex

@cbay ,

The minimal components for testing M0166 camera with VOXL2 are:

• M0155 adapter
• coax camera cable (should come with the camera)

To run the solo configuration, you would need to do the following in software:

• copy correct ar0144 sensormodule driver to /usr/lib/camera/
• create (or copy/paste) a voxl-camera-server.conf file for that camera to /etc/modalai. The config can be copied for many of our standard configs.
  • as an example, you could run voxl-configure-cameras 26 to use our standard C26 config (see here : https://docs.modalai.com/voxl2-coax-camera-bundles/) and then just disable / remove (in the generated config file) the cameras that are not actually present.

For testing 1-3 M0166 cameras,

• option 1: 1-3 M0155 (unsync'ed), plugged into VOXL2 J6, J7, J8
• option 2: 1 M0173 (synchronized or unsynchronized), M0173 plugs into VOXL2 J6, J7.

Please take a look at this post, you will find some useful information about customization : https://forum.modalai.com/topic/4491/ar0144-rgb-output-on-voxl2 (even though your question is not related to a color version of M0166).

If you are using VOXL2 mini, you can still use two M0155. If you wanted 2-3 sync'ed cameras, you need to use either M0188 or M0195 (both of which are for VOXL2 mini only, just like M0173 is for VOXL2 only).

There are lots of options.. depending on the path you take, we can help you get the software set up.

Alex

@taiwohazeez , the image is indeed blurry. Did the camera come like this or did something happen to the camera / lens?

Also, i am assuming that the blur is not coming from the camera moving around (that is to say the camera is stationary during those image captures).

Please inspect the lens on the outside to make sure it's not scratched or damaged, as this can also contribute to the issue.

In any case, adjusting focus on a camera is usually as simple as turning the lens slightly CW or CCW. Since the image is almost focus, i expect that the adjustment would be within 1 full turn from the existing lens position in the lens holder. If the camera lens is being held by a plastic mount, it should have a screw on the side of the plastic mount, which clamps the mount around the lens. The screw needs to be loosened and then you can use the focus tool to turn the lens slightly while looking at the image view in voxl-portal. For this particular camera with a wide lens, the hyperfocal distance will be around 10cm, meaning that when properly focused, everything beyond 10cm away should appear in focus. With this in mind, you should some target (checkerboard is fine) 0.3 - 0.5m away and turn the lens back and forth (with the plastic tool) to try achieve the best focus. If you cannot achieve good focus (the best focus is still blurry), it probably means that either the lens is damaged dirty (front or back side of the lens).

If you do unscrew the lens all the way out (by turning counter-clockwise until the lens comes out), don't leave the camera exposed like this for a long period of time, so that dust does not get on the camera sensor.

Also, what is the exposure and gain reported in the stats below the image in the voxl-portal view, when you view the camera stream (without the calibration). The environment seems pretty dark, i would recommend more lighting, but let's double check the reported exposure and gain (from auto exposure). Also, please make sure your checkerboard pattern is as flat as possible (this is unrelated to blurry image, but will result in better calibration results).

Please try focusing the camera and let me know if you have any questions.

Alex

Hi @Seungjae-Baek,

It is hard to see the detail in the screenshot, it is very low resolution. Can you please clarify which cameras you are turning on / off?

Also, what streams are consumed from the camera server (by other processes) and what are the resolutions?

Also, keep in mind that your cpu may not run at a high frequency. Please check voxl-inspect-cpu for more details. If your cpu governor is in auto mode, it will reduce cpu core frequency, which will result in higher % usage of each core (at lower frequency).

If you sent your cpu to performance mode (voxl-set-cpu-mode perf), this will push the frequency of all cores to the maximum and will give you a better estimate of the cpu usage per process (out of the full core capacity). Please note that in performance mode, the cpu will consume more power (and heat up a bit more) to perform the same tasks (with lower latency).

Alex

@Peter-Milani , I see. this looks like a generic implementation of OpenCL for ARM from 3rd party (not Qualcomm), and i think it also overwrites the proprietary opencl libraries, disabling the GPU opencl support. However, you could make two separate docker images, one for each use case (cpu and gpu)

Alex

@Peter-Milani , it looks like Qualcomm CPU device is not supported by OpenCL library from Qualcomm.

clinfo may be confused, but installing and running clinfo natively on voxl2 does not return any platforms - the opencl libraries that may get installed by apt are most likely not compatible with the VOXL2 GPU.

Alex

Actually, I see from the log that your FPV camera is IMX214 (please confirm).

So then my next questions are:

• which Voxl2 SDK are you using?
• does the crash happen if you only view the Hadron EO stream (this is OV64B camera)
• can you please check the cpu temps when you view many cameras (using voxl-inspect-cpu)

The dmesg error suggests some sort of low level error during MIPI transmission, but it could be tied to the SoC overheating.

When voxl-portal is used to view images (especially in multi-view), the images on VOXL2 are converted from YUV to Jpeg using a software Jpeg encoder (which is very taxing on the cpu for large images). This can cause the cpu to overheat and start throttling operating frequencies to reduce the temperature). We are aware of this issue and it is on the list to fix the multi view to be more efficient and not overheat the CPU.

Alex

@ravi , can you please let me know which FPV camera you are using? also can you share the entire voxl-camera-server.conf ?

Alex

@ashley-miller , for this motor (28xx class), you should use the following ESC M0138 FPV ESC :

https://www.modalai.com/products/m0138
https://docs.modalai.com/voxl-fpv-esc-datasheet/

The mini ESC (M0129) is designed for small and lightweight drones and mini FPV racers (750g or less for slow flyers and 500g or less for mini FPV drones).

The peak power for one of these motor is almost 1kW each - the mini ESC cannot handle that for several reasons, such as peak current capacity, thermal.

If you would like to use the 2808 motor, please choose the M0138 FPV ESC and perform motor calibration using voxl-esc tools : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-tools/calibration.md

Also, please review the following FAQ: https://docs.modalai.com/voxl-escs/faq/

Alex

@jeremyrbrown5 , it is possible that overheating VOXL2 cpu could be causing your SoftAP performance slowdown. You can run voxl-inspect-cpu which is a tool that will show you cpu usage, temps and other stats. If the CPU cores reach 95 deg C, the thermal management will slow down the cpu clock, so all the processes will run slower, including SoftAP.

Alex

Hi @SKA ,

I believe i fixed this issue with full-follow mode in the camera server. Now EIS has a body->imu transform, which is read from the extrinsics file and is used to properly rotate the image, instead of hi-jacking the cam->cam_eis transform.

So now your cam->cam_eis transform should work. Please make sure the imu_apps->cam transform is correct, if your camera is rotated 90 degrees.

Please check the commits from Nov 10 : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commits/dev

Alex

Hello @jeremyrbrown5 ,

Can you please answer the following questions to help us diagnose the issue?

• Which wifi adapter are you using?
• Does rebooting VOXL2 Mini help?
• How are you measuring the connection quality?

Alex

Also, a quick note.. regarding what preview resolution to use with IMX412 camera with MISP enabled..

For best quality (non-EIS applications, where the output is 16:9 aspect ratio, such as 3840x2160, 1920x1080, 1280x720), you should select 3840x2160 as the preview resolution. For EIS applications, the full frame 4040x3040 resolution should be used because it will allow more stabilization margin. selecting 3840x2160 will tell the camera to crop this region from the full frame 4040x3040, since we don't need the rest of the image.

For low latency applications, you can use input resolution of 1936x1080. Note that this is slightly non-standard (as opposed to 1920x1080 because the 1936x1080 resolution allows faster processing. So you could have input resolution of 1936x1080 and output resolution of 1920x1080, 1280x720, etc.

Alex

@Jetson-Nano , this question is probably for @Eric-Katzfey

@Jetson-Nano , it seems that Vertiq has PX4 support, so you should be able to modify their driver to enable the turtle mode. The mode is triggered by checking the RC switch within the voxl-esc driver.

https://iqmotion.readthedocs.io/en/latest/tutorials/ifci_px4_flight_controller.html

If you disable the voxl-esc driver, PX4 should work fine, as long as you correctly update the PX4 parameters to use another ESC type.

Alex