Topics created by Rowan Dempster

@Moderator Hi Modal,

Does this mean you can create a custom data file partition that you can flash using fastboot?

Yes that is correct.

We have not explored this before but it has been asked a few times. This could be very helpful for other developers.

I'm certainly hopeful that it will be helpful here at Cleo Robotics! So far with my prototyping it works as expected and cuts flashing time of some large docker images we have here at Cleo down by a noticeable fraction (no file overhead via fastboot like with ADB).

I think the snippet I posted covers the baseline functionality of getting a custom "payload" into the data partition. However if there is more I can elaborate on in terms of the toolchain / what's in the payload, and if that elaboration will be helpful to other VOXL2 developers, I would be happy to elaborate 🙂 Just let me know!

Other similar discussion points I tackled recently that I'm happy to talk about lessons of:

Flashing the system image and VOXL/CLEO SDK through a Windows Machine (journeys in USB device drivers) Building Flutter applications for uniform flashing process across all operating systems Building release bundles (i.e. a collection of partition binaries) in CI

@Rowan-Dempster , Thanks for the update!

On my end, I did not get a chance to enable MISP pipeline (normalization in particular) to run offline. Let me double check something with you - would you want to load a log with raw10 and resize + generate the normalized image (of a different size) or just keep the same size as input?

Since you have the basic rolling shutter QVIO working, it would be interesting to see how that compares to the AR0144 QVIO from the same data set. Any details you would like to share? (only if you want to) 😄

Regarding playback results from QVIO output not being exactly repeatable -- " no deterministic mechanism to ensure that voxl-qvio-server initializes using the same camera frame" -- what do you mean by that? if you start QVIO and start feeding the frames + imu data from voxl-replay, the data should arrive into QVIO repeatably and QVIO should initialize on the same frame - after the initialization conditions have been satisfied.

If I remember correctly the way QVIO app works is that it may hold the frame until all the IMU data for that frame has arrived and then pushes the frame into the QVIO algorithm. If you push the frame before pushing all the IMU data, then the algorithm will still process the frame but it wont have the IMU data for the whole duration of the frame capture.

The frame capture duration for global shutter cameras is simple :

start = start of exposure end = end of exposure = start of exposure + exposure time
(i believe it is common to assign a timestamp which is equal to the center of exposure, which is between the start and end timestamps). center of exposure = start of exposure + (exposure time)/2

In order to find the center of exposure for rolling shutter camera, you need to use the following formula, as used in our EIS implementation in MISP:

int64_t center_of_exposure_ns = start_of_exposure_ns + (exposure_time_ns + readout_time_ns)/2;

In any case, this would be something to double check.. also i don't know how far your trajectories are from run to run.

Another tip: since you don't have ground truth in your data, it helps to start and end your data collection in exactly the same spot and orientation, so you can use the end position drift as a good metric (ground truth).

Alex

To clarify, our current platforms are shipped with voxl-open-vins-server as the default VINS solution; however, QVIO is still available (SDK 1.6.2).

Hi @Rowan-Dempster ,

We just merged the changes to dev : https://gitlab.com/voxl-public/voxl-sdk/core-libs/libmodal-pipe/-/merge_requests/39.

A new build of the libmodal-pipe library will be added after tonight's nightly build here : http://voxl-packages.modalai.com/dists/qrb5165/dev/binary-arm64/ .

Any SDK releases after this will include this support.

Alex

@Alex-Kushleyev The IMU is being configured with an 8K ODR so by increasing IMU_GYRO_RATEMAX you are reading the FIFO more often but reading less samples each time. So you are mainly increasing the overhead of context switching. Can you characterize how much the load increases just by increasing IMU_GYRO_RATEMAX and not doing any of the extra UART IO? Unfortunately we don't have a lot of control over the low level implementation of the IO drivers that are in the Qualcomm code. And there is no DMA that could help lower IO overhead. One idea would be to lower the ODR to 1K so that you are only reading one sample from the FIFO at each interrupt.

@Rowan-Dempster ,

Here is where the raw data from the TOF sensor comes into voxl-camera-server : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/hal3_camera_mgr.cpp?ref_type=heads#L1242

The data is in the MIPI12 packed format. The TOF library expects RAW16 format, so the data is converted from MIPI12 to RAW16 and fed into the TOF processing library. It is possible that dumping the RAW data is what RRF recording would be doing, so you could add that functionality to camera server to write the raw data to file. I could also help with publishing the RAW12 data as is to a mpa channel, so you can log it outside of the camera server (for example using voxl-record-raw-image utility. You could clarify with PMD if that would work (in other words, you would need to know the log format of the RRF recording)

We have not tried enabling the raw data logging. However, if you know what function needs to be called within PMD library (maybe you just enable logging, and it does everything for you), you should be able to add it somewhere in the TOF init code, maybe here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/tof_interface.cpp?ref_type=heads#L982

You can find the API header files for Royale / Spectre in /usr/include/royale either on VOXL2 directly on in /opt/sysroots/qrb5165_1/usr/include/royale the voxl-cross:V4.4 docker.

Hopefully that helps!

Alex

@Alex-Kushleyev Gotcha, thanks for the info Alex and your help resolving this issue!

@Rowan-Dempster No problem! Kind of primitive but effective 🙂

(top is Royale 4, bottom is 5)

@Rowan-Dempster ,

I myself tried to do this some time ago and ran into issues, which i could not resolve. I am sure also someone else at Modal also tried it before me.

If I remember correctly, my issue was that it could not get camera module in the second instance of the camera server, on this line, and i did not pursue this further (did not spend too much time on it though).

https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/master/src/hal3_helpers.cpp?ref_type=heads#L79

So in your case you are able to get the camera module?

To help debug, you should run logcat while you are trying this and see what messages might show up there.

And, finally, just wanted to mention that we don't know if what you are doing is possible, but if you want to keep pursuing this direction, I can help if i know the answer to your questions, but may not be able to spend time working on it right now.

EDIT: in order to see if separate instances of camera server are possible, you could try to see if you can run multiple instances of gstreamer plugin that uses qmmfsrc. I found a doc that should be helpful in testing this, take a look : https://thundercomm.s3-ap-northeast-1.amazonaws.com/shop/doc/1596593567074634/Thundercomm EB5_Multimedia SDK User Guide_V1.1_ie89e.pdf

Specifically, the following command should work, although i did not test it:

gst-launch-1.0 -e qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! omxh264enc control-rate=max-bitrate target-bitrate=6000000 interval-intraframes=29 periodicity-idr=1 ! queue ! filesink location="/data/vid.h264"

Also, i dont see how you sent the camera id in this example, but it should definitely be possible 🙂

See if you can run two independent instances of gstreamer using different cameras. I think it should work, but not sure.

Alex

@Rowan-Dempster I think it is safest to just not use data that reads 0,0,0 as the sensor cannot measure that close anyways

About 95C is when the temperature control loop will kick in and start reducing the maximum core frequencies (gradually). You can monitor the cpu usage and current core frequencies using voxl-inspect-cpu.

Here are the maximum core frequencies for all cores:

cpu0 1804.8 cpu1 1804.8 cpu2 1804.8 cpu3 1804.8 cpu4 2419.2 cpu5 2419.2 cpu6 2419.2 cpu7 2841.6

If you set the cpu governor mode to perf (voxl-set-cpu-mode perf), it will pin all the cores to max frequency and they will stay there unless they are being throttled due to temperature, which you can check using voxl-inspect-cpu.

@Sarika-Sharma You can get v1.0.10 here: http://voxl-packages.modalai.com/dists/qrb5165/dev/binary-arm64/libfc-sensor_1.0.10-202602041609_arm64.deb

@Rowan-Dempster No, it isn't

@Rowan-Dempster They've been released and are a part of the SDK but we just don't install them by default to save space.

The .debs are onboard already and should install with an apt install

In that context "blackisted" just means don't install by default.

@thomas I assume we don't want to be dealing with 4k raw arrays on the CPU for performance reasons. So yeah as early as possible in the image proc pipeline, and how early we can push it is what I'm asking for advice on. We'll also reach out to @Alex-Kushleyev thanks!!