ModalAI Forum

boga

@Alex-Kushleyev thanks for clarifying this! I tracked the voxl-qvio-server source code and the mvVISLAM.h library documentation. It is clear that only fused data is available and now I also know why thanks to your explanation @Alex-Kushleyev. I am working with an engineer who is dealing with stereo visual odometry so in this case it is possible to get the visual odometry without the IMU fusion also. @Eric-Katzfey it seems that you did not understand my question, but Alex's answer was perfect, thanks anyway.

boga

Is it possible to get the visual odometry data before IMU fusion in other words just the visual odometry?

boga

@Eric-Katzfey, yes. I came up with the same solution in parallel. I am already able to read data. FTDI based converter appears as ttyUSBX (X=0...n).
Also, while we're at it... If I would use the VOXL Flight port J10 to read in UART data, what is the port number from the software side?
Is there a documentation about that?

boga

How do I add support for CP210x USB to Serial converter on my m500 drone?
I know that it is a driver / kernel module issue, but I can't figure it out. Connecting Arduino UNO creates a ttyACM0 (virtual serial) right away. But the device that I need to connect has CP210x serial converter chip onboard.

I found the tutorials below, but as I am new to this, I am unable to figure it out. Help please!
https://gitlab.com/voxl-public/system-image-build/voxl-build/-/blob/master/README.md#building-the-kernel-and-wlan-module
https://gitlab.com/voxl-public/system-image-build/meta-voxl/-/tree/master?ref_type=heads
https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers?tab=downloads
https://askubuntu.com/questions/941594/installing-cp210x-driver

boga

@Moderator, repeating some of my questions, because I did not get the answers.

We are logging the data on the 32GB (UFS 2.0) /data/voxl-logger/ location. That, probably means, that the speed and bandwidth are better than most SD cards, right? Which would also mean we are not running out of bandwidth, when recording from a camera or cameras. Is that so?
I posted the list of services that are running. Do you see any that are unnecessary and could be interfering with the logging process? I mean, which ones we could turn off?
Have you recorded the data using voxl-logger and gone through the logs to see if you have any data missing? Did you look at the files that I attached?
Could this be a voxl-logger version issue? We have an older version of the code (0.3.5), could this be an issue? Maybe there were some fixes introduced
Could you propose other methods for recording and checking the camera feed?
Maybe we should update the whole SDK?

boga

Short Answer
We are not logging to the SD card. We are logging to the default location: /data/voxl-logger/ (which is in soldered flash I believe).
Test results from logging one camera:

Tracking - no droputs
Stereo - shorter dropouts than logging all cameras

We can try logging to high-end SD Card.

@Moderator said in Huge number of camera frames skipped using voxl-logger:

top is not helpful as it does not account for CPU scaling

Ok, clear. Then we have to use voxl-inspect-cpu (or a modified version of it which will write the data into columns, so we can log it and correlate with the droputs).

Longer Answer
@Moderator, like I already said, we already tried logging just one camera at a time. Although, I just went through the gathered data again and I can see no dropouts when we are logging only the tracking cam. This was during 1 minute 36 second flight. When we are logging only the stereo camera, we see dropouts, although they have shorter period of 10 seconds, compared to the ones that happens when we are logging all the cameras + visual odometry stuff, in that case the dropouts are 30 seconds long.

We are logging to the default location which is /data/voxl-logger/ . I believe that this is located on a soldered flash? Do you want to tell me that SD card communication could be faster?

We need to log all of the cameras + visual odometry data.

So I have the logger with the following command: voxl-logger --preset_odometry --cam stereo --cam hires_large_color

When I was logging only the stereo or tracking then I executed the command like this (of course):
voxl-logger --cam stereo
voxl-logger --cam tracking

boga

@Moderator we have default setup, so it shouldn't be the case. Haven't modified the drone in any way and we are not running anything special. We tried to log only stereo camera data using voxl-logger, still had 10 second and about 300 frame "blackouts". I recommend you to look at the data that I uploaded, you can see the behavior over there.

I also logged the voxl-inspect-cam tool at the same time to see if all frames are there and according to the voxl-inspect-cam that is true. Meaning that in the published camera feed all of the frames are there, which would lead me to think that something happens with the voxl-logger (side of things). As hi-resolution camera logs frames do not drop as suddenly and at the same time, I would think that something is wrong with logging the tracking and stereo cameras?

I haven't had the chance to run the voxl-inspect-cpu while flying and it's a shame that there is no reasonable option to log the voxl-inspect-cpu data. I guess the voxl-portal is not needed, but we were not connected to it anyway. QGroundControl is connected and receiving all of the default stuff + hires camera feed (also default stuff).
voxl-logger is at version 0.3.5

Here is the output from voxl-instepct-services. Which services could I turn off?

Service Name | Version | Enabled | Running | CPU Usage

I am thinking of verifying if I am able to capture the video feed with another tool, to see if there are any blackouts at the same time that we have missing frames in the voxl-logger data. I am thinking of using voxl-ffmeg would you recommend something else? The setup should be as simple as possible, just to verify that we are able to capture all of the data from a camera or cameras.
I will also do the voxl-inspect-cpu test during flying and also I am planning to monitor the process load using top.

boga

We are gathering data, from m500 drone, to apply our own visual odometry algorithms. We discovered yesterday that a huge number of frames are suddenly skipped in stereo and tracking camera data. Also the hires camera frames are skipped, but in smaller amounts (usually 2 - 4 frames). I have added a link to a zip folder where there is stereo camera images, data.csv and calculations file (camera_calc_data.xls). You can see from the calculations file that there are 965 frames lost before sample/picture number 290 and that the image collection has been "frozen" for 33 seconds. Another place where we can see the same is at sample 567.
https://drive.google.com/file/d/1bh0zA9yBO6Si-yWjcdOsxNplCR-9AVLk/view?usp=sharing

Is this caused by the voxl-logger?
Or is this a camera stream issue?
System overload? Probably not, because the hires camera is still streaming at the same time.

We also did a test where we logged only the stereo camera, but we are still seeing similar behaviour. Although, in that case, there was a 10 second gap instead of 30 seconds.

boga

I found the following piece of code that seems to be dealing with the time_sync between VOXL and PX4: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-mavlink-server/-/blob/master/src/mavlink_services.c?ref_type=heads

But I do not fully understand it. Maybe someone can explain? I believe that this means that the logs are also synced or not?

boga

VOXL Flight has 5 IMUs - 2 connected to Snapdragon and 3 connected to Flight Core. The board is basically 2 products in one. I want to collect and synchronize the data from VOXL and Flight Core side. What is the best way for doing that?
On the VOXL side I can use voxl-logger to collect camera and IMU information and on the Flight Core (PX4) there are ulogs containing all the needed information.
If I understand correctly then VOXL side has timestamps in UNIX format, but (allegedly) the ulogs use microseconds since bootup? How do I synchoronize the voxl-logger and ulog data?

boga

@Eric-Katzfey That works, thanks!

boga

@Eric-Katzfey thanks for the answer. Are you sure about that?
I have m500 drone, where I have VOXL 1 Flight Deck, not VOXL 2.

You said:"Starting with SDK 1.2.0 px4 shifts to using the MONOTONIC clock on the DSP (Where most of PX4 runs)"

This sounds to me like you are talking about VOXL 2 Flight, or not?
I thought that on VOXL 1 Flight the PX4 applications and firmware are running on the ARM M7 STM32F765II - isn't that so?

Or do the PX4 appliations really run on DSP, also on VOXL 1 Flight controller?

boga

First, about the context. I want to log (VOXL Flight's) camera feed, IMUs and GPS for development.
I am able to log the camera feed and IMUs (on board the VOXL part) using voxl-logger. And I am able to log the IMUs and GPS on board of the Flight Core part using the default PX4 logging system, specifically Ulog.

To make sense of the data it should be synchronized, but I am not sure if it is? Does anybody know?

If I look at the voxl-logger files the timestamp is a 12-digit timestamp in nanoseconds and in the ulogs the timestamp is an 8-digit timestamp in milliseconds.

voxl-log timestamp example: 293857036778
px4 ulog timestamp exmaple: 29438654

Can someone tell me if these timestamps are taken from the same clock source?
Do I understand correctly that both timestamps are in Unix Timestamp format?
Is there documentation about the clock system on board of the VOXL Flight?

The Linux timestamp resets to 1st of January 1970, every time I disconnect the battery. I know that there is no RTC onboard.
Does anybody have experience adding RTC to the VOXL Flight Deck? Does it make sense to do it?
If I investigate the ulog files they seem to have correct date in the title. Where is this taken? GPS lock? QGroundControl?

boga

I need to log PX4 sensors and GPS data as fast as possible. Right now the logging intervals are too slow. I found this forum post: https://discuss.px4.io/t/high-speed-logging/25242/4 .

According to this post I should change the rates in this file:
https://github.com/PX4/PX4-Autopilot/blob/main/src/modules/logger/logged_topics.cpp

But this means that have to recompile the whole PX4 firmware (right?) and then upload it. Isn't there an easier way? Isn't there a way to change the logging intervals from a configuration file or through parameters or something like that?

boga

I want to record data from all the cameras at once (or at least from tracking and stereo at the same time). Is it possible?
Also, what is the frame rate when logging the camera images? Searched everywhere, but I could not find that information.

Thanks

ModalAI Forum

boga

Posts