Posts made by Alex Kushleyev

@mkriesel , just to confirm, does this test work (run) on other voxl2 boards for you?

@mkriesel , our M0153 / M0201 adapter does have an option to connect Boson to the common sync line that is used for synchronizing AR0144 tracking cameras (GPIO 109). This line is typically driven at 30Hz, but the frequency is adjustable.

There is a resistor pad on M0153 / M0201 adapter which is DNI by default, so a 0-ohm resistor would need to be installed to enable this functionality. However, we have not verified that it works.

It is on our to-do list to verify the Boson sync. Once we confirm that it works, I can provide instructions for installing the resistor if you wanted to enable that functionality. Boson's configuration will also need to be updated to enable the sync input, but it's not a big change.

Let me get back to you next week..

Alex

@Aaron-Porter , perhaps systemd-timesyncd works together with systemd to correctly interpret the clock adjustments, so that run-time of systemd services is properly calculated.

Alex

Also, a quick update.. after some additional testing, we are actually able to communicate with Boson via the CCI connection directly from VOXL2. Hopefully we will be able to enable (partial) Boson configuration from VOXL2 directly, avoiding the use of the USB connection for simple configuration change.

Here is a sample script that can be used to test triggering FFC. The script uses cci-direct library which allows to communicate to cameras while they are running (in camera server).

Before running this script, make sure the camera server is running and it is configured to enable Boson camera (and take note of the camera id (sw id, not slot)), which is needed by cci-direct and the script. This is also abit of a hack since we are writing part of the payload as the the address in addressed write (as it is common in i2c transactions), but Boson's i2c interface does not support addressing.

#!/bin/bash

if [ "$#" -eq 0 ]; then
	echo "Error: No arguments provided."
	echo "Usage: $0 <boson camera id>"
	exit 1
fi

cam_id=$1

#command to trigger FFC
#(extra two 00's at the end to make it a multiple of 3, since we are sending 3 bytes at a time in this test)

data="8E A1 00 0C 00 00 00 FC 00 05 00 07 FF FF FF FF 00 00"

IFS=' ' read -r -a words <<< "$data"

for (( i=0; i<${#words[@]}; i+=3 )); do
	echo "${words[i]} ${words[i+1]} ${words[i+2]}"
	voxl-cci-direct -c ${cam_id} -w 0x${words[i]}${words[i+1]} 0x${words[i+2]}
done

Hello @mkriesel ,

Please see below instructions for test 14 bit Boson data stream on VOXL2:

• voxl-camera-server : use branch https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/tree/add-boson-14bit-support
• latest Boson drivers (8 bit unchanged, added 14 bit drivers) : link
• python scripts to read Boson config, set to 8 bit, set to 14 bit :
  • read config
  • set 8 bit
  • set 14 bit

Boson 14 bit release notes

• 14 bit data is published at RAW16 format (unpacked from 14 bit).
• The boson_bayer image stream can be viewed with voxl-portal, but will appear dark grey because the data is not scaled to full 16 bit range. voxl-portal will just convert 16->8 bit with a bit shift.
• 8-bit sensormodule has to be used to receive 8 bit data : com.qti.sensormodule.boson_X.bin (replace X with camera slot id)
• 14-bit sensormodule has to be used to receive 14 bit data : com.qti.sensormodule.boson_14bit_X.bin (replace X with camera slot id)

Configuring Boson to 14 bit:

• use SDK4.0_Boson_Plus_SDK
• connect Boson to Linux PC via USB
• run python3 boson_read.py to check the current settings
• run python3 boson_set_14bit.py to set to 14 bit output (check FPS setting in the script)
• run python3 boson_set_8bit.py to set to 14 bit output (check FPS setting in the script)
• using incorrect sensormodule (which does not match the bit resolution actual Boson output),
  will result in the following message in camera server and no data will be received from Boson:

ERROR:   Received "Buffer" error from camera: boson

• 30 or 60 FPS works with 8 and 14 bit. The fps setting in camera server config does not change FPS, but you should make the value in the voxl camera server config file consistent with the actual setting (otherwise video encoder bandwidth may be incorrect, if you use that)
• using Boson configured for 14 bit with version of camera server which does not support 14 bit will result in VOXL2 board crash due to insufficient buffer size allocated for the incoming frames (buffer overflow)
• use the camera server config shared above (with misp enabled for Boson camera)

voxl-camera-server will detect the frame size and output either:

camera boson640 detected frame size: 640x512 327680 bytes (819200 alloc).. raw bpp: 8

or

camera boson640 detected frame size: 640x512 573440 bytes (819200 alloc).. raw bpp: 14

Please give it a try. We will merge this to the dev branch of camera server soon.

Alex

@Aaron-Porter , I believe the time sync services are interchangeable, such as systemd-timesyncd, ntpd, chronyd. I don't think we have tried using anything other than the default systemd-timesyncd on voxl2.

If you try those out, please let us know if you find any issues! Thanks

Alex

@Allister-Lim ,

If you are not monitoring the battery voltage while working with your battery-powered VOXL, it is possible that you are over-discharging the batteries to the point of failure. Especially with LiPo batteries, it is unsafe to over-discharge them, so please be careful. (It seems that you are also using LiPo batteries, in addition to ModalAI Li-Ion batteries also shown (green)). There is nothing wrong with using LiPo batteries, but you have to be more careful with those. Over-discharged batteries lose capacity and Lipo Batteries may become dangerous.

It is a good idea to use an appropriate DC power supply when working with VOXL2 while it is powered on continuously. 12V 3-5A Power supply, connected to the APM (which will provide 5V for VOXL) should be sufficient. You should have no problem finding one, but in case you need, here is a link to the AC adapter that we sell : https://www.modalai.com/collections/accessories/products/ps-xt60

Alex

@Rowan-Dempster ,

The SPI Read is a blocking call, so the DSP will spend time in that call while the data is being transferred, but it may be sleeping while waiting for the data to come into the SPI Hardware.

The UART Write is, actually, also a blocking call, so the DSP will write the data to the UART Hardware and sleep until the data is finished sending.

If you are measuring the times spent in those functions, it can add up. If you increase your UART baud rate, that time should go down. While the DSP is waiting inside those functions, it can process other threads.

The DSP utilization measurement is not very accurate, so it should not be trusted completely. How much utilization increase are you seeing from increasing the processing rate from 800Hz to 1000Hz ?

Maybe @Eric-Katzfey can comment as well - thanks!

Alex

@mkriesel , I would suggest doing a UART loop-back test of the ESC connector on your VOXL2 board. You would need to make a connector which connects UART TX and RX on VOXL2 side (ESC disconnected). We have a SW tool for doing a loopback test of DSP UARTs : qrb5165io-slpi-uart-test

Source code: https://gitlab.com/voxl-public/voxl-sdk/core-libs/libqrb5165-io/-/blob/master/apps/examples/qrb5165io-slpi-uart-test.c

You can test the ESC UART on J18 and also two additional UARTS (also mapped to the DSP) on J19. If the test fails only on J18 UART, then this may be some HW issue. If none of the UARTS on J18 and J19 pass the loopback test, then issue is probably in SW.

Please try and we can go from there.

Alex

Hi @isaac,

We only have two variants of the kernel for VOXL2. The default (original) kernel and one that supports M0173. (mach. var 1.0.0 and 1.0.1 respectively, which you can see in output of voxl-version).

If you are using M0135 camera interposer board, you would have to use the first kernel. However, this kernel is set up for a stereo setup in J8, so J8U is not available for regular cameras. If you are using J6 and J7 for cameras, then using M0135 in J8 to enable J8U, is not really possible, even with a kernel modification (due to some shared reset lines that will affect several cameras, if enabled). I would need to know exactly what cameras are connected where in order to provide more details.

Now, if you use the second kernel in combination with M0173, you can also use other adapters such as M0181, M0172. Enabling a hi-res camera in J8U has been discussed in the following thread, with images and instructions : https://forum.modalai.com/topic/4757/connecting-2-boson-cameras-and-2-mipi-cameras-4-cameras-total . Please note that you can use the smaller boards such as M0181 and M0172 instead of M0173 with the second kernel.

I would strongly suggest going the second route, but if you have an issue with this, let me know. I don't have the full understanding based on your brief question.

Alex

@mkriesel , we were able to get the 14 bit data from Boson after performing a firmware update on our old Hadron unit.

Enabling this will require updated voxl-camera-server, a new Boson sensormodule file as well as a change in the Boson configuration.

We were not able to use the FLIR GUI to switch Boson to 14bit MIPI, but using their python API, we were able to do it over USB (while connected to a linux PC).

While outputting 14 bit data, which is RAW output from the sensor, you cannot simultaneously get the post AGC 8 bit data, so you would need to do your own processing on the 14 bit data.

Our plan is to enable the RAW data publishing first along with perhaps a simple post-processed version of the data. The data will be published in uint16 format, so that you won't have to do the 14->16 bit unpacking.

Here are the Boson versions that worked / did not work to enable 14 bit MIPI data.

Hadron (640x512)
Before update: Software Version 3 0 26416 (not able to switch to 14 bit mode via gui/ python api)
After update: Software Version: 4 1 27660 (able to switch to 14 bit mode via python api and get data on voxl2)

Small Boson (320x256):
Software Version: 4 0 27141 (able to switch to 14 bit mode via python api and get data on voxl2)

Update is coming probably next week.

Regarding the 60Hz, actually we made some improvements in camera server a while ago and 60Hz is working now. You can just update your Boson FPS using the GUI and try it. You should update the FPS in voxl-camera-server.conf to 60 as well, so that if you are using the encoded output, the encoder knows what FPS to expect (remember, the camera server cannot actually change the FPS of the Boson).

Finally, we were previously using the ISP to process the Boson data, which made the image look a bit worse. Please make sure to use MISP when testing boson and disable the small / large encoded streams.

Here is a sample config:

{
			"type":	"boson",
			"name":	"boson640",
			"enabled":	true,
			"camera_id":	2,
			"fps":	30,
			"en_preview":	true,
			"en_raw_preview":	true,
			"en_misp":	true,
			"preview_width":	640,
			"preview_height":	512,
			"en_snapshot":	false,
			"ae_mode":	"off",
			"en_rotate":	false,			
			"misp_width":	640,
			"misp_height":	512,
			"misp_venc_enable":	true,
			"misp_venc_mode":	"h264",
			"misp_venc_br_ctrl":	"cqp",
			"misp_venc_Qfixed":	30,
			"misp_venc_Qmin":	15,
			"misp_venc_Qmax":	50,
			"misp_venc_nPframes":	29,
			"misp_venc_mbps":	2,
			"misp_venc_osd":	false
		}

In order to use MISP for boson, you need to have the camera server which includes the following commits: -- see here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commits/master?ref_type=heads

By the way, there is now also a Boson color stream (heat map) which is generated from the original 8 bit post AGC image. You can see an example in this forum thread : https://forum.modalai.com/topic/4757/connecting-2-boson-cameras-and-2-mipi-cameras-4-cameras-total (where there are two bosons, both running at 60fps, by the way)

Alex

@roy-crosthwaite , please submit an RMA form for a replacement : https://www.modalai.com/rma

Thank you

Alex

@monkescripts said in Be careful building voxl-cross on older SDKS (Other than V1.50.0):

RUN wget http://voxl-packages.modalai.com/dists/qrb5165/sdk-1.5/binary-arm64/voxl-opencv_4.5.5-3_arm64.deb

This line installs only the opencv package into the voxl-cross docker image, it does not install the SDK.

The mismatch you are referring to is related to to applications being built against one version of header files, while being deployed on a system that has a different version of the corresponding library. In this case, the library is libmodal-pipe, which has had a change affecting the structure of some data types.

Most likely, what happened was that you hand an older version of SDK and other libraries installed on the VOXL2, but when you built your application using voxl-cross, the step which installs the dependecies (./install_build_deps.sh qrb5165 <....>) pulled in a newer version of the libmodal-pipe library and headers. When the application was deployed to voxl2, at run-time, it linked to the old libmodal-pipe shared library, causing the mismatch.

There are many updates to libmodal-pipe and most of them do not cause any issues because there are added features, which do not break existing functionality. However, sometimes, a breaking change like this happens..

Alex

@Martin-Lukac ,

The diagram for this variant of the ESC is correct : https://docs.modalai.com/modal-esc-datasheet/#connector-information-and-pin-outs. This applies to M0049, M0117 and M0134 ESCs.

If you are using the M0129 mini ESC, it has a different connector : https://docs.modalai.com/voxl-mini-esc-datasheet/#connector-information-and-pin-outs

Regarding the baud rate.. Your ESC probably came already set up to use 2Mbit baud rate in the ESC params. You have to make sure that the PX4 uses the baud rate that the ESC params are set to, otherwise there will be a baud rate mismatch. It looks like everything is working as expected.

Alex

Here are the links to the test firmware:

M0129-3
M0129-6

After you update, you should see the following version after running voxl-esc-scan.py (except yours will be M0129-6). The firmware version will be 39.23 and this version will be reserved only for this test firmware:

INFO: ESC Information:
INFO: ---------------------
	ID         : 0
	Board      : version 40: ModalAi 4-in-1 ESC (M0129-3)
	UID        : 0x2039333557555304003F0029
	Firmware   : version  39.23, hash 30c088dc
	Bootloader : version    184, hash 10bf24c8
...

In case you need help updating ESC firmware: https://docs.modalai.com/voxl-escs/faq/#q-how-do-i-update-esc-firmware

We won't officially release this firmware version in order to avoid the aforementioned risk of turning on 5.0V by just installing a firmware update, however we will work on enabling this in the ESC params.

Also, just as a reminder, please be careful soldering to the AUX test points, use high quality flexible cables, and use strain relief to avoid lifting the pads.

Alex

@Martin-Lukac ,

Thanks for the feedback. We can work on adding a param. Meanwhile, for your testing, I will send firmware (based off the latest version), which will just set the VAUX to 5.0V all the time.

Alex

@Martin-Lukac , the ModalAI ESCs are not (typically) controlled via PWM signal, so the calibration of the PWM range is not supported for these ESCs.

Instead, a one-time calibration (for a new motor/ prop/ voltage combination) needs to be done and a new ESC config file uploaded to the ESCs, as described here :https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-tools/calibration.md . We recommend that you perform this calibration while your ESC is connected to a Linux PC, so you can verify the results using the data plots that our tools (voxl-esc) generate.

Regarding arming / disarming.. The ModalAI ESC, while PX4 is running, is always armed and ready to spin. PX4 sends commands to the ESC and tells the ESC to not spin (stop). Whenever you are ready to fly, you send the takeoff command (or arm command) and the motors start spinning (at idle rpm if you are in manual control and your thrust is at minimum).

Please note that after calibrating the ESC configuration using the procedure above, you also need to tell PX4 the new RPM range of your motor, so that the motor control is applied correctly. Please see the following doc with more information : https://docs.modalai.com/modal-esc-px4-user-guide/#px4-114-and-newer-eg-voxl2-using-voxl-esc

You can think of armed stated is the state then the vehicle is ready to fly. When the propellers are not spinning, the vehicle is not ready to fly (disarmed).

Please let me know if you have any other questions.

Alex

@Martin-Lukac ,

Yes, we have the capability on the M0129 ESC to enable 5V AUX output instead of 3.3V. The reason why this has not been exposed to the users is to avoid a situation where a user has a 3.3V device plugged in, but the ESC outputs 5.0V.

Just to clarify your needs - do you need the output to be always 5V and for it to turn on as soon as the ESC boots up? or do you want to turn it on using GPIO (off or on 5V)?

We have two options, which we have been going back and forth internally:

• add a new ESC config param to specify the VAUX voltage (off, 3.3V, 5.0V) and whether it should be on after ESC starts up (the defaults would be 3.3V and ON after power up).
• enable gpio control of the pin that actually controls the VAUX setting (3.3V or 5.0V), so that users can change the output voltage via UART commands
  • this seems not practical, since if you have a device that's plugged in, it could get 3.3V first until you enable 5.0V output.

Since it is unlikely that someone would need to switch between 3.3V and 5.0V while the ESC is ON, we will probably go with the first option - add a param for voltage after start-up and still allow to tun the regulator on/off.

Please let me know your thoughts.

Alex

@Jskim , sounds good!

Just FYI, i noticed that the config file i provided has some outdated settings for IMX412 camera - I have some better suggestions and will follow up soon. You can still test with the above config - the hardware functionality will not be affected.

Alex

@luisgb , all of ModalAI ESCs can be controlled via UART using two modes

• power : command 0-100% power, which controls the average duty cycle for that motor
• RPM : (requires calibration for the motor + propeller type): controls rpm directly

Commands to the ESC are sent via UART and normally that UART port is connected to VOXL2 UART, which is mapped directly to the DSP. The DSP is where most of PX4 runs.

If you are looking for the easiest integration / testing, you can connect the ESC directly the VOXL2 CPU (using non-DSP UARTs) and use either python or c++ code to interface to the ESC (or similarly from a Linux Desktop / laptop).

It is also possible to communicate with the ESC while the ESC is connected to the DSP UART. This is how the voxl-esc tools work on VOXL2 (while PX4 is not running). This works by bridging the connection between the DSP and CPU and accessing the DSP UART from the CPU. However, this path has extra delay and is not recommended for any real-time feedback control applications.

If you provide more information about your use case, I can suggest the best path.

Alex