Posts made by Alex Kushleyev

@ashwin , from software point of view, running multiple TOF V2 sensors on VOXL2 is not an issue, but as Vinny said, the power requirements would put a limit of a single TOF sensor per VOXL camera connector (J6, J7, J8), so that would mean at most 3 TOF sensors connected to J6L, J7L and J8L (camera slots 0, 2, 4 in order to avoid i2c slave address conflicts -- this is more of a note to self ).

However, assuming the SW can support these 3 TOF sensors, i am not sure we actually have the required interposers / flex cables to connect 3 TOF V2 sensors to the 3 camera ports. @Vinny , what do you think?

Also, @ashwin , would you need any other cameras connected to VOXL2 besides the multiple TOF sensors?

Alex

and also a simple example here : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/blob/master/tools/voxl-inspect-tof.c?ref_type=heads

@jameskuesel , thank you for letting me know about the focus issue. It seems we had a few cameras shipped out without being properly focused - we will make sure to take care of it in the future. Sorry about that.

Regarding the image quality, after adjusting the lens focus, are you satisfied with the image resolution / quality? Also i believe the color balance outdoors should look better under natural light.

Alex

I just made a fix on a branch, but did not test it:

https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commit/43aa51a43c339e008b86d637b87cb22d9c369588

If you are able to test the fix, please give it a try and let me know if it works!

Thanks!

Alex

@eric , thanks again for your input on this, i have posted a complete tutorial how to enable OpenCL in Docker on VOXL2 : https://docs.modalai.com/voxl-2-opencl-in-docker/

Alex

Hi @LucaVertiq , this is cool stuff. I am curious if you were able to do any direct comparisons of qvio performance using unfiltered vs low pass (rpm-based) vs notch (rpm based)?

In order to replicate the same conditions for each test, this would have to be done using mpa playback after collecting mpa logs of just imu data and images.

since the rpm data is not coming from mpa, it cannot be logged / played back using our logger.

However, one idea is to process the IMU data logs using RPM logs using a separate script and then run the playback + qvio to generate new results. The IMU data processing can have different options like the rpm-based notch filter and it would obviously also need to have a px4 log that contains the ESC status.

This also leads to a question, whether the IMU data should be filtered in the imu server or the application that uses the imu, since the requirements for the IMU data may differ for different applications, so it may be best to leave the IMU server publishing the original data. But also, there could be a separate output pipe from the imu sever with the notch filtered imu data..

Anyway, just wanted to check if you happened to get any quantitative results from your testing that show how the filtering helps with QVIO performance. My understanding from working with QVIO in the past was that the best thing to do was to feed unfiltered imu data into QVIO and let it integrate the noisy data. low pass filtering the IMU data too much would actually remove some of the actual components of motion that are important for motion tracking. So i am curious what exactly has improved in your case when you added notch filtering - is the drift vs distance improved or the number of VIO "blow-ups" has reduced?

Lots of questions, but I am happy to discuss this further and see how we can improve performance of our VIO stack

Alex

@Jetson-Nano , As far as I know, turtle mode should work the same when running PX4 on VOXL2 and FC V2. Have you tested it? The LED command should not interfere with the turtle mode.

I have not actually tested the turtle mode on FC myself, but the code that is running in voxl-esc px4 driver is the same, so as long as you are entering the turtle mode correctly, it should work.

Alex

@Jetson-Nano , there is probably more than one way of getting this to work, but I have been told that with the new approach that we have just implemented (being tested now), the mavlink tunnel should take care of routing the LED data from VOXL2 to FC V2 to ESC.

I asked my colleague to help describing how to test this.

Alex

@andrevs01 ,

The documentation links I provided describe how to get M0065 board running in PX4 as a standard actuator that supports 8 PWM outputs.

If you are familiar with PX4, GCS, Actuators, then look through those documentation pages and follow the setup.

However, i don't have the full information about your hardware, so i don't know if you need some customizations:

• are you using voxl2 on a flying drone? if so, are you using a modalai ESC or another ESC?
• Are you planning to use M0065 board to generate pwm output for your ESC (if you are using a standard ESC that accepts PWM as control input)
• do you need to use PX4 for your application? the reason why I ask is that you can use M0065 pwm outputs without PX4 if you don't need PX4 at all.

Alex

Hi @Cody , is it the case that the commanded RPM is relatively high, but motor just does not spin up? Did you confirm that from the logs?

We have a fork of px4-flight-review, which you can run on your linux pc locally https://github.com/modalai/px4-flight-review . There is a Quickstart Guide that should get you up and running in a few steps (Docker is required) - https://github.com/modalai/px4-flight-review?tab=readme-ov-file#quickstart-guide

This has support for our custom data logs, including commanded and actual motor RPMs, which is useful for debugging. Can you send a plot of RPMs from the flight review app?

You should have a plot like this, maybe zoomed in on areas of interest.

Alex

@Madelyne-Rossmann , sounds good! We will work on enabling larger resolutions for calibration and tag detection. It is a bit of a hassle due to very large frames, so some optimizations need to be made.

Alex

@Jetson-Nano ,

I checked with my colleague and they said that the current way will not work for forwarding the Neopixel LED packet to the FC V2. Here is roughly how the LED packet is created and sent to the ESC:

• voxl-send-neopixle-cmd creates a data packet (using the specific format for ESC) and sends it to a special process modal-io-bridge via mpa pipe that is also called modal-io-bridge.
• The modal-io-bridge is a PX4 module that is specific to VOXL2 and it runs on the CPU (not DSP). It receives the data from the mpa pipe and sends out the data using ORB_ID(modal_io_data) using data type modal_io_data_s.
• then voxl_esc driver that runs on the DSP gets this data and forwards it to the ESC directly, as shown here: https://github.com/modalai/px4-firmware/blob/70d986f08acbf90e3752ef832151d8f993e18697/src/drivers/actuators/voxl_esc/voxl_esc.cpp#L1346
• this approach has no option to forward the message to external FC.

However, actually this week we are making changes to use mavlink tunnel messages instead of this custom io bridge. I will find out how to get this to work for FC V2.

Alex

@黄充 , I think you found a small bug. We have two versions of TOF sensors:

• TOF V1 (which is EOL) with resolution 224x172 = 38528
• TOF V2 (which is what you probably have) with resolution 240x180 = 43200

As you can see in the source code, the json metadata for the pipe is initialized with w=224 and h=172, and it is never updated for the new TOF sensor.

https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/hal3_camera_mgr.cpp?ref_type=heads#L3576

voxl-streamer reads the information from pipe metadata and thinks the image size will be 224x172 and when receives the actual data, detects a mismatch.

If you are able to re-build the camera server with changing the resolution in UpdatePipeJson, i think this will work for you. We will fix this as well.

Thank you

Alex

@valvarez , the usage for the test tool is as follows:

voxl2:/$ qrb5165io-uart-test -h
Description: runs hardware loopback test on specified bus

Usage: qrb5165io-uart-test

    -d <device ID> (eg 3 for /dev/ttyHS3)

So you should run your test as:

voxl2:/$ qrb5165io-uart-test -d 2
[INFO]: success!

@voxl2_user , i will ask around..

Also, there may be some sysv compatibility mode, instead of fully utilizing sysv for all IPC. I think i saw something like that when grepping the kernel build source for sysv supprt. I will check a bit later.

Alex

@valvarez , i believe i have previously tried VOXL_ESC + VOXL2 IO drivers running at the same time and it worked. You just have two output interfaces that you can configure and test via GCS.

As Eric mentioned, you just need to make sure you configure the two drivers to start together and to connect to the correct ports (and baud rates).

Regarding outputting PWM from VOXL_ESC, there is a way to make it work. We do not have the PWM ports on the ESC exposed as normal PWM outputs in PX4, but you can use a special tool / command on voxl2, which will send message to the ESC (via PX4) and control the PWM port:

https://gitlab.com/voxl-public/voxl-sdk/services/voxl-io-server/-/blob/master/tools/voxl-send-esc-pwm-cmd.c (which is also part of our SDK) .

voxl2:/$ voxl-send-esc-pwm-cmd -h

Tool to send command a VOXL ESC to make it output a PWM signal
If timeout is disabled (default) then the esc will keep sending the same
pwm command even after this program exits. If enabled, the signal will stop
after 0.5s. Oneshot mode will automatically exit this program after one message
is sent to the esc, otherwise it will keep going untill you hit ctrl-C

ESC_PIN_M0129_AUX0           2
ESC_PIN_M0129_AUX1         258
ESC_PIN_M0129_AUX2         514
ESC_PIN_M0129_AUX3         770
ESC_PIN_M0129_AUX_REG_EN     6
ESC_PIN_M0129_AUX_REG_ADJ   22

ESC_PIN_M0138_VTX_REG       80
ESC_PIN_M0138_RGB_OUT       24
ESC_PIN_M0138_AUX0           2
ESC_PIN_M0138_AUX1         770
ESC_PIN_M0138_AUX_REG_EN    12
ESC_PIN_M0138_AUX_REG_ADJ   11

D0008_V5_LIGHT_BAR_PIN       2
D0008_V5_PAYLOAD_PIN_0     770
D0008_V5_PAYLOAD_PIN_1      24

-h, --help                  print this help message
-o, --one-shot              only send one command then quit,
                              not to be confused with short duration 125us oneshot mode
-p, --pin {pin}             output pin
-t, --timeout               enable 0.5s timeout, otherwise pwm will continue indefinitely
-w, --width {val}           pulse width microseconds (default 1500)

typical use: voxl-send-esc-pwm-cmd -p 2 -t -w 1500

You will need to provide the pin number. See the usage helps, which provides the pin mapping (look at pins for M0129 ESC, but pwm pins are the same on M0117, M0134 ESCs as well). For a simple test, you can use pin=2 (meaning this will control Pin A2 on ESC ID 0 ), which should control the PWM0 output on your ESC. The 4 PWM outputs on the ESC are each connected to A2 port of the corresponding microcontroller, responsible for each ESC ID. The pin mapping is also documented here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-io-server/-/blob/master/lib/modal_io.h?ref_type=heads#L66

Alex

@Gary-Holmgren , we have not tried the mask with 1280x800 resolution.

Can you please try using the mask of size 160x120 and make it half white / half black for sanity testing. It is possible that the mask size has to be of these specific dimensions (and it would get resized to match the aspect ratio of the actual image, not sure).

Also, you should enable the debug prints in QVIO, as mentioned here https://forum.modalai.com/topic/3459/masking-qvio/6 and there is another potential "gotcha", the mask path is relative to working directory where qvio server starts.. but after you enable the qvio server debug prints, you will see the path from which the mask file is being loaded, as you can see from screen shots from the other post.

Unfortunately, the underlying QVIO source code is not available, so I cant check the exact behavior of the masking feature.

Alex

@andrevs01 ,

The documentation is available here:

https://docs.modalai.com/voxl2-io-datasheet/
https://docs.modalai.com/voxl2-io-user-guide/
https://docs.modalai.com/voxl2-io-firmware/

We do not provide a way to flash the board with your own code (although you could do it if you REALLY wanted to, since the board uses a standard STM32 processor).

For your use case, are you using PX4 or not? We normally control M0065 PWM outputs from PX4, but it is possible to do it without PX4.

Alex

@voxl2_user , yes, I can also confirm that my usb to ethernet adapter (eth0) is no longer working on my voxl2 after enabling this..

It is possible that enabling SYSVIPC disables some other IPC mechanism used by the usb ethernet driver.. or something else is going on.. not sure..

Alex

I just tried it and it seems to be working

voxl2:/$ ls /proc/sys/kernel/sem*
/proc/sys/kernel/sem

voxl2:/$ ipcs -l                             

------ Messages Limits --------
max queues system wide = 32000
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 18014398509465599
max total shared memory (kbytes) = 18014398509481980
min seg size (bytes) = 1

------ Semaphore Limits --------
max number of arrays = 32000
max semaphores per array = 32000
max semaphores system wide = 1024000000
max ops per semop call = 500
semaphore max value = 32767