OV64B Configuration

jcai

@Alex-Kushleyev I appreciate the detailed response!

We'd be open to experimenting with some of this hardware. Is a picture of the M0159 (not schematics) something we could get?

As for an initial use case:

OV64B: 30fps, 720p/1080p
Boson: 30fps, 640x512, MIPI
M0161 IMX412: This is what's currently interfaced with the M0173. What would it take to include this in the setup?

Alex Kushleyev

@jcai ,

Yes, i will provide the photos of the boards.

And yes, it should work with M0173 + M0161 but only on voxl2. M0173 takes up two camera connectors J6 and J7, so the Hadron would connect to J8.

Another option is to use M0161 with just M0155 coax adapter, since connecting a single camera to M0173 is an overkill. M0155 + M0161 would only use one camera connector, allowing you to use M0161 and Hadron on voxl2 mini.

I will double check both options.

Alex

Alex Kushleyev

@jcai

Please see the following images of the Hadron 640R adapters. Please note that the hardware is subject to change, but this is our current version.

M0159 : Plugs into the back of Hadron. it has two micro coax cable coming out and an optional USB plug for connecting to the "Boson" inside Hadron

As I mentioned before, you can access the Boson camera via USB or MIPI.

M0181 is the adapter for connecting M0159 to VOXL2 / VOXL2 mini:

It is also possible to split up the two coaxes and instead of using a single M0181 for dual camera connection, M0155 for OV64B (https://www.modalai.com/products/m0155) and M0162 for Boson (or use USB for Boson).

jcai

@Alex-Kushleyev This looks pretty good, thanks!

If I'm understanding correctly, with one M0159, one M0181, and one M0155, I'd be able to connect all three sensors (OV64, boson, M0161) on a VOXL2/VOXL2 mini?

If that's the case, I'd be interested in experimenting with that setup. What would the process for getting these boards look like?

For the connection between the boards, what's the longest length coax I could get or use reliably?

Alex Kushleyev

@jcai, yes you are correct regarding the connections that will work on Voxl2 and Voxl2 mini.

Please reach out to https://modalai.com/contact and we will create a custom order for you. You should add a link to this thread. I will work with our team to get the right hardware for you.

You will need

M0159
M0181
two specific coax cables between those teo boards (18cm length?).
M0155 (assuming you already have M0161 with a coax, but is your M0161 connected to M0173 camera front end?

You should also get a tool for removing coax connectors - check at the bottom of this guide: https://docs.modalai.com/micro-coax-user-guide/ . I dont think we sell the tool, but it looks like the tool is out of stock (Mouser) - you will need it to disconnect your M0161 from M0173.. i will check if we can provide it.

Regarding length, it looks like the longest cable we sell is 18cm : https://www.modalai.com/products/mcbl-00084?variant=48956821733680 , and this is what i am using for testing Hadron.

Alex

Alex Kushleyev

@jcai , Here is a draft of instructions for using the Hadron adapter (M0159 + M0181) for beta testing. I will put these up on our docs page but just wanted to provide these to you so you can test the hardware. Please let me know if you have any questions.

WARNING:

even though the coax cables are the same for both connections, the order cannot be swapped
HR connector has to conect to HR and 5L to 5L
M0159 J2 -> M0181 J1
M0159 J3 -> M0181 J2
shipped assemblies have been correctly connected and tested

Software Setup

use voxl2 mini (VOXL2 also supported)
install sdk 1.4.0 (tested) or later
change apt source to dev : /etc/apt/sources.list.d/modalai.list
- deb [trusted=yes] http://voxl-packages.modalai.com/ ./dists/qrb5165/dev/binary-arm64/

apt update
apt install libmodal-pipe voxl-portal voxl-camera-server

Confirm sensormodule drivers exist

in chi-cdk directory where all the camera drivers are kept (but not used)

ls /usr/share/modalai/chi-cdk/ov64b40 : com.qti.sensormodule.ov64b40_0.bin ... 5.bin
ls /usr/share/modalai/chi-cdk/boson:    com.qti.sensormodule.boson_0.bin ... 5.bin

Confirm these exist (also part of camera sernsor drivers):

/usr/lib/camera/com.qti.sensor.ov64b40.so
/usr/lib/camera/com.qti.sensor.boson.so

Connect M0181 to Voxl2 Mini J7

make sure pin alignment on connectors is correct. the mounting holes should align and the tab of M0181 will hover above the main SoC
see attached image
the tab can be removed from M0181 if not needed

Copy correct sensormodule drivers

cp /usr/share/modalai/chi-cdk/boson/com.qti.sensormodule.boson_2.bin /usr/lib/camera/
cp /usr/share/modalai/chi-cdk/ov64b40/com.qti.sensormodule.ov64b40_3.bin /usr/lib/camera/

Configure CCI Mux on M0159 via GPIO

Use gpio 6 on J7 of VOXL2 Mini to configure the i2c mux to connect J7U cci to Hadron's cam cci
this needs to be done each time after reboot, before running camera server

voxl-gpio -m 6 out && voxl-gpio -w 6 1

Detect cameras

voxl-camera-server -l
(should detect both cameras)

Supported resolutions / modes

OV64B currently supports the following modes
these are the RAW modes that camera can be configured to (by the camera pipeline)
- the camera pipeline picks the best mode based on the requested resolution and fps
- note that if you request 3840x2160 30FPS, the camera pipeline actually selects 4624x3472 because it matches the desired FPS (30)
- actual selected mode can be checked using logcat | grep -i selected before starting camera server, the selected resolution will be printed for each camera.

# (Mbps per csi lane)
Mode0_9248x6944_10fps_2500.8Mbps
Mode1_4624x3472_30fps_PD_1136x860_1502Mbps
Mode4_3840x2160_60fps_2500.8Mbps
Mode5_1920x1080_240fps_2500.8Mbps
Mode6_1920x1080_30fps_800Mbps

Max resolution (works! about 9fps)

		"preview_width":	9248,
		"preview_height":	6944,

Note that at this >8K resolution, the Qualcomm ISP runs in dual VFE mode, which means you cannot use any other non-RAW camera that uses the ISP.
MISP approach does not use Qualcomm ISP and will support more cameras (documentation coming soon)

Minimum config

paste into /etc/modalai/voxl-camera-server.conf:

{
	"version":	0.1,
	"fsync_en":	false,
	"fsync_gpio":	109,
	"cameras":	[
		{
			"type":	"boson",
			"name":	"boson",
			"enabled":	true,
			"camera_id":	0,
			"fps":	30,
			"en_preview":	true,
			"en_misp":	false,
			"preview_width":	640,
			"preview_height":	512,
			"en_raw_preview":	true,
			"en_small_video":	false,
			"en_large_video":	false,
			"ae_mode":	"off"
		},
		{
			"type":				"ov64b",
			"name":				"hires",
			"enabled":			true,
			"camera_id":		1,
			"fps":				30,
			"en_preview":		true,
			"en_misp":			false,
			"preview_width":	1920,
			"preview_height":	1080,
			"en_raw_preview":	false,
			"en_small_video":	false,
			"en_large_video":	false,
			"en_snapshot":	false,
			"ae_mode":	"isp",
			"gain_min":	54,
			"gain_max":	32000
		}
		]
}

Start Camera Server

run voxl-camera-server in foreground to make sure everything is going right and you can view streams using voxl-portal
note that very high resolution streams viewed as YUVs (which are transferred as MJPG by voxl-portal) will take a lot of CPU to encode to MJPG and also a lot of bandwidth to stream.

Image of M0181 Plugged into VOXL2 Mini J7

jcai

@Alex-Kushleyev Great, I'll be updating with how the tests go

Alex Kushleyev

@jcai , updated doc posted here : https://docs.modalai.com/voxl2-hadron/ . Mostly the same information with some clarifications.

I will also add information about the USB connector on M0159 adapter.

Alex

smiley

Just stumbled across this topic and quickly wanted to ask if it is already known when those boards will be available in the shop?
Else, would it be possible to order through email?

Thanks a lot for your feedback!

Alex Kushleyev

@smiley , we made a small adjustment to the M0159 board to make it fit into the Hadron better (board outline). That was a small board re-spin (same functionality). Checking on the status..

Alex