OV64B Configuration

jcai · 10 Jan 2025, 21:44

Hello, I'm seeing in the documentation and in the code that ov64b sensors are supported by SDK 1.4. I've confirmed that the bins exist in chi-cdk.

However, I'm not seeing any information/documentation on camera configs nor any default camera-server configs in the source code for this sensor.

How do I go about using it with voxl-camera-server.conf?

Alex Kushleyev · 13 Jan 2025, 17:34

Hi @jcai,

I can help you with this. Can you please let me know which OV64B camera module you are using and where / how you are connecting to? (voxl2 or voxl2 mini and which camera connector, interposer, etc).

Alex

jcai · 13 Jan 2025, 18:28

@Alex-Kushleyev

I'm using a Hadron 640R with a VOXL2 (for now, but we plan on switching to a VOXL2 mini eventually). I also have a M0173 adapter board.

So far, I've only done some calcs for 35mm focal length:
With f = 4.8mm and D = 8.15mm, I get 35FL = 25.48mm
To make it an int, f = 4.75 and D = 8.2 is an option.

Alex Kushleyev · 13 Jan 2025, 20:13

@jcai , can you please clarify the following:

Do you also have any other connectors for Hadron and cables (micro coax)?

M0159 adapter board is required, which plugs into the back of Hadron and exposes two separate micro coax connections, one for Boson and one for OV64B).

I just need to make sure that you have all the required hardware.

Alex

jcai · 13 Jan 2025, 21:27

@Alex-Kushleyev I don't have the hadron itself on hand at the moment. I'm just doing some preliminary reading and planning for when we do get it.

We have a G-Hadron gimbal and accompanying G-Hadron RB5 adapter, which together will expose the boson through USB and the OV64B through MIPI.

Do you have a link to the M0159? I haven't been able to find it. If possible, I'd rather not connect the boson over USB.

Alex Kushleyev · 13 Jan 2025, 22:29

Thanks for clarifying regarding RB5 adapter. Unfortunately, camera connectors on RB5 and VOXL2 are not compatible, so you will not be able to use the gimbal right now.

I just checked with the team, here are some more details:

Boson / Hadron support is in beta, not generally available yet but if you want to do beta testing, we can arrange it
If you have a stock Hadron, you will need custom adapters to interface Hadron to VOXL2 (we support both OV64B and Boson using MIPI)
So far we have tested the following resolutions / fps using OV64B:
- 1920x1080 (1080p) up to 240fps
- 4K60 (i believed this is a 2x2 binned version of 8K maximum resolution)
- 8000x6000 9fps (this is slightly smaller than absolute max resolution 9248x6944 which we will try to enable).
- we may be able to achieve 8K (7680x4320) at fps closer to 30, but this has not been implemented yet.
support on voxl2 mini has also been tested (but limited testing so far)
Boson should be able to do 640x512 60fps (but on some units we had to slow down to 30fps due to mipi timing issues, just FYI)

If you would like to get early access to the hardware adapters, let's make sure that we can support your use case, so please provide some more high-level information, such as:

desired OV64b resolution / fps
are you going to need Boson connection via MIPI as well (the boson that is part of Hadron). resolution / fps

Here is the configuration that we are currently testing

the only difference between the two options is whether on VOXL2 side you have two separate connectors (which will take up two Jx ports on VOXL2) or a single connector for a single Jx camera port.

EDIT: M0159 also has a connector for USB2.0 that connects to Boson (provides the Linux serial and video interface, so you could run Boson over usb competely). Connector is JST SH (not standard USB plug)

In order to install M0159 on the back of Hadron, you may need to remove the adapter that is currently attached to it. M0159 plugs in to Hadron back plate - ready to plug state shown here :

None of these boards (M0159, M0181, M0162) are generally available yet (except for M0155). Let's first make sure that you are willing to experiment with this and then go from there.

Let me know if you have more questions about how the Hadron to VOXL2 connections will work.

Alex

jcai · 14 Jan 2025, 19:26

@Alex-Kushleyev I appreciate the detailed response!

We'd be open to experimenting with some of this hardware. Is a picture of the M0159 (not schematics) something we could get?

As for an initial use case:

OV64B: 30fps, 720p/1080p
Boson: 30fps, 640x512, MIPI
M0161 IMX412: This is what's currently interfaced with the M0173. What would it take to include this in the setup?

Alex Kushleyev · 14 Jan 2025, 22:45

@jcai ,

Yes, i will provide the photos of the boards.

And yes, it should work with M0173 + M0161 but only on voxl2. M0173 takes up two camera connectors J6 and J7, so the Hadron would connect to J8.

Another option is to use M0161 with just M0155 coax adapter, since connecting a single camera to M0173 is an overkill. M0155 + M0161 would only use one camera connector, allowing you to use M0161 and Hadron on voxl2 mini.

I will double check both options.

Alex

Alex Kushleyev · 16 Jan 2025, 17:30

@jcai

Please see the following images of the Hadron 640R adapters. Please note that the hardware is subject to change, but this is our current version.

M0159 : Plugs into the back of Hadron. it has two micro coax cable coming out and an optional USB plug for connecting to the "Boson" inside Hadron

As I mentioned before, you can access the Boson camera via USB or MIPI.

M0181 is the adapter for connecting M0159 to VOXL2 / VOXL2 mini:

It is also possible to split up the two coaxes and instead of using a single M0181 for dual camera connection, M0155 for OV64B (https://www.modalai.com/products/m0155) and M0162 for Boson (or use USB for Boson).

jcai · 16 Jan 2025, 20:52

@Alex-Kushleyev This looks pretty good, thanks!

If I'm understanding correctly, with one M0159, one M0181, and one M0155, I'd be able to connect all three sensors (OV64, boson, M0161) on a VOXL2/VOXL2 mini?

If that's the case, I'd be interested in experimenting with that setup. What would the process for getting these boards look like?

For the connection between the boards, what's the longest length coax I could get or use reliably?

Alex Kushleyev · 16 Jan 2025, 22:21

@jcai, yes you are correct regarding the connections that will work on Voxl2 and Voxl2 mini.

Please reach out to https://modalai.com/contact and we will create a custom order for you. You should add a link to this thread. I will work with our team to get the right hardware for you.

You will need

M0159
M0181
two specific coax cables between those teo boards (18cm length?).
M0155 (assuming you already have M0161 with a coax, but is your M0161 connected to M0173 camera front end?

You should also get a tool for removing coax connectors - check at the bottom of this guide: https://docs.modalai.com/micro-coax-user-guide/ . I dont think we sell the tool, but it looks like the tool is out of stock (Mouser) - you will need it to disconnect your M0161 from M0173.. i will check if we can provide it.

Regarding length, it looks like the longest cable we sell is 18cm : https://www.modalai.com/products/mcbl-00084?variant=48956821733680 , and this is what i am using for testing Hadron.

Alex

Alex Kushleyev · 26 Feb 2025, 05:00

@jcai , Here is a draft of instructions for using the Hadron adapter (M0159 + M0181) for beta testing. I will put these up on our docs page but just wanted to provide these to you so you can test the hardware. Please let me know if you have any questions.

WARNING:

even though the coax cables are the same for both connections, the order cannot be swapped
HR connector has to conect to HR and 5L to 5L
M0159 J2 -> M0181 J1
M0159 J3 -> M0181 J2
shipped assemblies have been correctly connected and tested

Software Setup

use voxl2 mini (VOXL2 also supported)
install sdk 1.4.0 (tested) or later
change apt source to dev : /etc/apt/sources.list.d/modalai.list
- deb [trusted=yes] http://voxl-packages.modalai.com/ ./dists/qrb5165/dev/binary-arm64/

apt update
apt install libmodal-pipe voxl-portal voxl-camera-server

Confirm sensormodule drivers exist

in chi-cdk directory where all the camera drivers are kept (but not used)

ls /usr/share/modalai/chi-cdk/ov64b40 : com.qti.sensormodule.ov64b40_0.bin ... 5.bin
ls /usr/share/modalai/chi-cdk/boson:    com.qti.sensormodule.boson_0.bin ... 5.bin

Confirm these exist (also part of camera sernsor drivers):

/usr/lib/camera/com.qti.sensor.ov64b40.so
/usr/lib/camera/com.qti.sensor.boson.so

Connect M0181 to Voxl2 Mini J7

make sure pin alignment on connectors is correct. the mounting holes should align and the tab of M0181 will hover above the main SoC
see attached image
the tab can be removed from M0181 if not needed

Copy correct sensormodule drivers

cp /usr/share/modalai/chi-cdk/boson/com.qti.sensormodule.boson_2.bin /usr/lib/camera/
cp /usr/share/modalai/chi-cdk/ov64b40/com.qti.sensormodule.ov64b40_3.bin /usr/lib/camera/

Configure CCI Mux on M0159 via GPIO

Use gpio 6 on J7 of VOXL2 Mini to configure the i2c mux to connect J7U cci to Hadron's cam cci
this needs to be done each time after reboot, before running camera server

voxl-gpio -m 6 out && voxl-gpio -w 6 1

Detect cameras

voxl-camera-server -l
(should detect both cameras)

Supported resolutions / modes

OV64B currently supports the following modes
these are the RAW modes that camera can be configured to (by the camera pipeline)
- the camera pipeline picks the best mode based on the requested resolution and fps
- note that if you request 3840x2160 30FPS, the camera pipeline actually selects 4624x3472 because it matches the desired FPS (30)
- actual selected mode can be checked using logcat | grep -i selected before starting camera server, the selected resolution will be printed for each camera.

# (Mbps per csi lane)
Mode0_9248x6944_10fps_2500.8Mbps
Mode1_4624x3472_30fps_PD_1136x860_1502Mbps
Mode4_3840x2160_60fps_2500.8Mbps
Mode5_1920x1080_240fps_2500.8Mbps
Mode6_1920x1080_30fps_800Mbps

Max resolution (works! about 9fps)

		"preview_width":	9248,
		"preview_height":	6944,

Note that at this >8K resolution, the Qualcomm ISP runs in dual VFE mode, which means you cannot use any other non-RAW camera that uses the ISP.
MISP approach does not use Qualcomm ISP and will support more cameras (documentation coming soon)

Minimum config

paste into /etc/modalai/voxl-camera-server.conf:

{
	"version":	0.1,
	"fsync_en":	false,
	"fsync_gpio":	109,
	"cameras":	[
		{
			"type":	"boson",
			"name":	"boson",
			"enabled":	true,
			"camera_id":	0,
			"fps":	30,
			"en_preview":	true,
			"en_misp":	false,
			"preview_width":	640,
			"preview_height":	512,
			"en_raw_preview":	true,
			"en_small_video":	false,
			"en_large_video":	false,
			"ae_mode":	"off"
		},
		{
			"type":				"ov64b",
			"name":				"hires",
			"enabled":			true,
			"camera_id":		1,
			"fps":				30,
			"en_preview":		true,
			"en_misp":			false,
			"preview_width":	1920,
			"preview_height":	1080,
			"en_raw_preview":	false,
			"en_small_video":	false,
			"en_large_video":	false,
			"en_snapshot":	false,
			"ae_mode":	"isp",
			"gain_min":	54,
			"gain_max":	32000
		}
		]
}

Start Camera Server

run voxl-camera-server in foreground to make sure everything is going right and you can view streams using voxl-portal
note that very high resolution streams viewed as YUVs (which are transferred as MJPG by voxl-portal) will take a lot of CPU to encode to MJPG and also a lot of bandwidth to stream.

Image of M0181 Plugged into VOXL2 Mini J7

jcai · 6 Mar 2025, 14:04

@Alex-Kushleyev Great, I'll be updating with how the tests go

Alex Kushleyev · 6 Mar 2025, 14:05

@jcai , updated doc posted here : https://docs.modalai.com/voxl2-hadron/ . Mostly the same information with some clarifications.

I will also add information about the USB connector on M0159 adapter.

Alex