Boson 640 MIPI M0153 16-bit

mkriesel

Re: Boson 640 MIPI M0153: 16-bit Pre-AGC

Hello, I saw this post and I'm in a similar boat where I'm looking at using a boson over mipi. Is there any update that has been done or is planned with supporting 16 bit mipi and possibly 60 hz update (less important but would still be nice)?

Alex Kushleyev

@mkriesel , we were able to get the 14 bit data from Boson after performing a firmware update on our old Hadron unit.

Enabling this will require updated voxl-camera-server, a new Boson sensormodule file as well as a change in the Boson configuration.

We were not able to use the FLIR GUI to switch Boson to 14bit MIPI, but using their python API, we were able to do it over USB (while connected to a linux PC).

While outputting 14 bit data, which is RAW output from the sensor, you cannot simultaneously get the post AGC 8 bit data, so you would need to do your own processing on the 14 bit data.

Our plan is to enable the RAW data publishing first along with perhaps a simple post-processed version of the data. The data will be published in uint16 format, so that you won't have to do the 14->16 bit unpacking.

Here are the Boson versions that worked / did not work to enable 14 bit MIPI data.

Hadron (640x512)
Before update: Software Version 3 0 26416 (not able to switch to 14 bit mode via gui/ python api)
After update: Software Version: 4 1 27660 (able to switch to 14 bit mode via python api and get data on voxl2)

Small Boson (320x256):
Software Version: 4 0 27141 (able to switch to 14 bit mode via python api and get data on voxl2)

Update is coming probably next week.

Regarding the 60Hz, actually we made some improvements in camera server a while ago and 60Hz is working now. You can just update your Boson FPS using the GUI and try it. You should update the FPS in voxl-camera-server.conf to 60 as well, so that if you are using the encoded output, the encoder knows what FPS to expect (remember, the camera server cannot actually change the FPS of the Boson).

Finally, we were previously using the ISP to process the Boson data, which made the image look a bit worse. Please make sure to use MISP when testing boson and disable the small / large encoded streams.

Here is a sample config:

{
			"type":	"boson",
			"name":	"boson640",
			"enabled":	true,
			"camera_id":	2,
			"fps":	30,
			"en_preview":	true,
			"en_raw_preview":	true,
			"en_misp":	true,
			"preview_width":	640,
			"preview_height":	512,
			"en_snapshot":	false,
			"ae_mode":	"off",
			"en_rotate":	false,			
			"misp_width":	640,
			"misp_height":	512,
			"misp_venc_enable":	true,
			"misp_venc_mode":	"h264",
			"misp_venc_br_ctrl":	"cqp",
			"misp_venc_Qfixed":	30,
			"misp_venc_Qmin":	15,
			"misp_venc_Qmax":	50,
			"misp_venc_nPframes":	29,
			"misp_venc_mbps":	2,
			"misp_venc_osd":	false
		}

In order to use MISP for boson, you need to have the camera server which includes the following commits: -- see here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commits/master?ref_type=heads

By the way, there is now also a Boson color stream (heat map) which is generated from the original 8 bit post AGC image. You can see an example in this forum thread : https://forum.modalai.com/topic/4757/connecting-2-boson-cameras-and-2-mipi-cameras-4-cameras-total (where there are two bosons, both running at 60fps, by the way)

Alex

mkriesel

@Alex-Kushleyev I only need the RAW output I plan to do all the post processing myself so this is exactly what I need, Thanks!

Alex Kushleyev

Hello @mkriesel ,

Please see below instructions for test 14 bit Boson data stream on VOXL2:

voxl-camera-server : use branch https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/tree/add-boson-14bit-support
latest Boson drivers (8 bit unchanged, added 14 bit drivers) : link
python scripts to read Boson config, set to 8 bit, set to 14 bit :

Boson 14 bit release notes

14 bit data is published at RAW16 format (unpacked from 14 bit).
The boson_bayer image stream can be viewed with voxl-portal, but will appear dark grey because the data is not scaled to full 16 bit range. voxl-portal will just convert 16->8 bit with a bit shift.
8-bit sensormodule has to be used to receive 8 bit data : com.qti.sensormodule.boson_X.bin (replace X with camera slot id)
14-bit sensormodule has to be used to receive 14 bit data : com.qti.sensormodule.boson_14bit_X.bin (replace X with camera slot id)

Configuring Boson to 14 bit:

use SDK4.0_Boson_Plus_SDK
connect Boson to Linux PC via USB
run python3 boson_read.py to check the current settings
run python3 boson_set_14bit.py to set to 14 bit output (check FPS setting in the script)
run python3 boson_set_8bit.py to set to 14 bit output (check FPS setting in the script)
using incorrect sensormodule (which does not match the bit resolution actual Boson output),
will result in the following message in camera server and no data will be received from Boson:

ERROR:   Received "Buffer" error from camera: boson

30 or 60 FPS works with 8 and 14 bit. The fps setting in camera server config does not change FPS, but you should make the value in the voxl camera server config file consistent with the actual setting (otherwise video encoder bandwidth may be incorrect, if you use that)
using Boson configured for 14 bit with version of camera server which does not support 14 bit will result in VOXL2 board crash due to insufficient buffer size allocated for the incoming frames (buffer overflow)
use the camera server config shared above (with misp enabled for Boson camera)

voxl-camera-server will detect the frame size and output either:

camera boson640 detected frame size: 640x512 327680 bytes (819200 alloc).. raw bpp: 8

or

camera boson640 detected frame size: 640x512 573440 bytes (819200 alloc).. raw bpp: 14

Please give it a try. We will merge this to the dev branch of camera server soon.

Alex

Alex Kushleyev

Also, a quick update.. after some additional testing, we are actually able to communicate with Boson via the CCI connection directly from VOXL2. Hopefully we will be able to enable (partial) Boson configuration from VOXL2 directly, avoiding the use of the USB connection for simple configuration change.

Here is a sample script that can be used to test triggering FFC. The script uses cci-direct library which allows to communicate to cameras while they are running (in camera server).

Before running this script, make sure the camera server is running and it is configured to enable Boson camera (and take note of the camera id (sw id, not slot)), which is needed by cci-direct and the script. This is also abit of a hack since we are writing part of the payload as the the address in addressed write (as it is common in i2c transactions), but Boson's i2c interface does not support addressing.

#!/bin/bash

if [ "$#" -eq 0 ]; then
	echo "Error: No arguments provided."
	echo "Usage: $0 <boson camera id>"
	exit 1
fi

cam_id=$1

#command to trigger FFC
#(extra two 00's at the end to make it a multiple of 3, since we are sending 3 bytes at a time in this test)

data="8E A1 00 0C 00 00 00 FC 00 05 00 07 FF FF FF FF 00 00"

IFS=' ' read -r -a words <<< "$data"

for (( i=0; i<${#words[@]}; i+=3 )); do
	echo "${words[i]} ${words[i+1]} ${words[i+2]}"
	voxl-cci-direct -c ${cam_id} -w 0x${words[i]}${words[i+1]} 0x${words[i+2]}
done

mkriesel

@Alex-Kushleyev Cool, follow up then the Boson has an external sync pin, does the mipi board support this still?

Alex Kushleyev

@mkriesel , our M0153 / M0201 adapter does have an option to connect Boson to the common sync line that is used for synchronizing AR0144 tracking cameras (GPIO 109). This line is typically driven at 30Hz, but the frequency is adjustable.

There is a resistor pad on M0153 / M0201 adapter which is DNI by default, so a 0-ohm resistor would need to be installed to enable this functionality. However, we have not verified that it works.

It is on our to-do list to verify the Boson sync. Once we confirm that it works, I can provide instructions for installing the resistor if you wanted to enable that functionality. Boson's configuration will also need to be updated to enable the sync input, but it's not a big change.

Let me get back to you next week..

Alex