@nl_vdi Here is a link to the latest beta: https://storage.googleapis.com/platform-beta-releases/voxl2/voxl2_SDK_1.6.4-beta8.tar.gz

Hi @RoyAzriel ,

Running voxl-inspect-cam -a puts significant stress on VOXL2 as all possible camera streams are generated and sent out to the voxl-inspect-cam application. With a small number of camera or low resolutions this can work fine, but in C29 config, this is not a sustainable test. There are quite a few very large images (4056x3040) per hires camera that are being sent uncompressed, causing a lot of cpu stress.

A better test would be to first run voxl-list-pipes to see which camera pipes are available and then run voxl-inspect-cam with arguments explicitly listing the camera streams to inspect -- ideally choosing one stream per camera.

example : voxl-inspect-cam tracking_front tracking_down hires_front_misp_color etc.

So to answer your question regarding "I wonder how it works well when you ship out C29 from the factory" -- we do not use voxl-inspect-cam -a for testing for the exact reason i described. Single stream per camera verification (can be simultaneous for all cameras) is sufficient to test the camera pipeline.

When voxl-inspect-cam was originally developed, we did not have as many cameras connected and the test worked fine, but started hitting system limits as the number of connected cameras (and their resolution) increased. In reality, you would never have to ship such large images across processes -- there is a better mechanism for sharing large buffers between processes (ION buffer) which avoids sending huge amounts of data over the linux pipes. ION buffer sharing is supported on voxl2 but that is outside of the context of this discussion, i believe

The rotation of the PMD sensor is done in software (camera server config, look for the rotation param) and it is done to match the actual rotation of the sensor in the camera mount. You can disable rotation in the camera server config.

Alex

@j-angel ,

We had a small batch of color AR0144, but those have run out and due to low demand, we do not have any stock.

I would like to clarify something. Motion blur is present when using any camera (global shutter or rolling shutter). Global shutter cameras also have exposure control and long exposure times will cause motion blur.

I think what you may be referring to, which is different in rolling shutter cameras (compared to global shutter), is the rolling shutter skew. Rolling shutter cameras use a technique where the start of exposure of each image row is offset slightly, so each row is essentially exposed over a different time frame. The offset is usually very small (4-30 microseconds per line, depends on the camera, mode), but from top to bottom, the rolling shutter effect can add up and be visible when there are moving features in the scene.

The fastest rolling shutter camera we have right now is IMX412 and in the 2x2 binning mode, the 1920x1080 resolution, the total frame readout time is 4ms. (https://docs.modalai.com/camera-video/low-latency-video-streaming/#camera-pipeline-latency-in-different-operating-modes). This means that the difference of the center of exposure between the first and last image row is 4ms.

Considering the information above, 4ms can seem a lot, but keep in mind that is the time delay across the whole image. depending on the size of features, the rolling shutter effect across a smaller image area will be smaller. Using the height of 1080, we can compute the time offset for each line, 4ms / 1080 ~ 4us.

Depending on the size of features that are being tracked, the rolling shutter may not create a lot of local skew. Additionally, the camera timing is deterministic, so it is possible to calculate the time offset of the center of exposure of each line (with respect to the first line). If you want to provide some more details on type of perception, I could provide more guidance.

Unfortunately, right now, we do not have a color global shutter camera option.

Alex

@nickanick , can you please clarify which exact WIFI Dongle you would like to use?

Are you able to unplug the wifi adapter from USB and plug it back in (while VOXL2 is on) and check what kernel reports in dmesg ?

@Jetson-Nano ,

Thank you for providing detailed description of the tests. I just requested access to the logs.

Please see the following comments:

• ESC temperature: the VOXL FPV ESC is rated for pretty high operating temperatures. For example, the MCUs are rated to run at up to 125C MCU temperature. I would not recommend running the ESC above 100C, however I am confident that the ESC will perform fine even above 100C. We typically recommend designing the system such that ESC temperature does not exceed 80C under normal flight conditions - this gives plenty of margin before a failure would occur. Did you take note of the ESC temps of VOXL ESC?
• VOXL FPV ESC reports two types of temperatures: two of the IDs (ID 0, 3) are measuring the ESC temps from separate temperature sensors located on top and bottom of the ESC. IDs 1 and 2 are reporting the MCU temps (the temperature of the MCU silicon). The MCU temps at steady state are typically a bit higher (5-7 deg C higher) because the MCU itself generates a little heat internally.
• VOXL ESC not responding after a crash : the likely reason is mechanical damage as a result of the crash. Components can be knocked off during the crash. If the ESC powers on briefly (you see the leds blink initially, but then the ESC turns off), then the likely issue is some kind of short or high load on one of the regulated ESC power rails (5V or 3.3V) -- the high current drain (due to some issue) results in the voltage regulator to turn off due to over-current, resulting in the ESCs shutting down. If only 3 out of 4 LEDs blink, then it is possible that one ESC channel has some sort of a short (due to mechanical damage) and it does not turn on, eventually causing the power rails to be disabled due to over-current.
• motor suddenly stopping in flight: this is often a result of a mechanical issue related to mounting the motor to the frame. Sometimes the motor mounting screws are too long and they can penetrate the motor windings and cause intermittent failures just like this. We have experienced this ourselves several times when long screws were used and were touching the motor windings. Alternatively, something could be touching the ESC itself and intermittently affecting operation of the ESC
• once I see the px4 log with the FPV ESC, i can check if one of the motors was operating abnormally before it stopped spinning. There is some additional information in the log that can tell us if a particular motor requires more power to spin (compare to the others).

Questions:

• do you have px4 logs of the VOXL ESC with one of the motors shutting down in flight?
• is it possible that the aluminum enclosure is making contact with the ESC components, causing some kind of short? especially during crash, perhaps the ESC is making contact with the enclosure?
• did you check if the motor temperature was OK after successful flights?
• what was the total current draw during flight when the failure occurred? (should see that in the px4 log)

Alex

@Hector-Gutierrez Well, I'm happy that worked, but you shouldn't have to disable voxl-vision-hub like that. I only suggested that as a debug step. We'll take a look and see if we can reproduce the error and come up with a better solution.

@Hector-Gutierrez What calibration step is not working? Are you running any custom software, like ROS or MAVSDK based applications? If you disable voxl-vision-hub does the error go away?

@bschulzhf PX4 will only correct incoming messages. Something external will have to also run TIMESYNC with PX4 to estimate the offset and correct and incoming PX4 message timestamps that it wants to align. voxl-mavlink-server does this for a few messages.

@nickanick That was what I responded with previously. Did that documentation not address your question?