Integration of Hadron 640R w/ VOXL2 Flight Deck on Sentinel

[shawn_ricardo]

Hello,

On a separate VOXL2, I've got the FLIR Hadron 640R up and running using the hardware/instructions found in https://docs.modalai.com/voxl2-hadron/#flir-hadron-640r-support-on-voxl2-and-voxl2-mini

I want to integrate this camera on the Modal AI Sentinel platform, but see that J7 is occupied by what appears to be the tracking camera and possibly a hi-res sensor.

Currently, I don't require an occupancy grid, so its possible to disconnect both pairs of stereo-cameras on the Sentinel and free up those ports.

Can y'all provide assistance in either:

1. Disconnecting the stereo cameras (both hardware and software instructions) and configuring the hadron adapter on these newly available ports
2. Disconnecting the stereo cameras (both hardware and software instructions), reconfiguring the existing tracking camera / hi-res camera on J7 to a newly freed port? (so I can connect the Hadron to J7)
3. Some other method that you recommend?

Thanks!

[shawn_ricardo]

Actually, can y'all provide the easiest path forward that supports just the tracking camera + hi-res front facing camera + FLIR Hadron?

That setup would be ideal

[shawn_ricardo]

@Eric-Katzfey

Would you be able to point me in the right direction? Maybe a list of technical docs or something of the sort?

Thanks!

[Eric Katzfey]

@shawn_ricardo I'll have to tag @Alex-Kushleyev on this one

[Alex Kushleyev]

@shawn_ricardo ,

The hardware configuration for your application should be as follows:

• J6 : tracking camera with either M0076 or M0135 interposer. Use J6L (lower camera port), which is the only port available on M0076 or JL port on M0135. This camera will be in HW Slot 0.
• J7 : Hadron, as described in the guide you already found. IR in HW slot 2, OV64B in slot 3
• J8 : hires (any) camera. Use either M0076 or M0135. Use J8L (lower camea port). HW slot 4

• note that J6U and J8U camera slots can only be used by the second stereo camera (in a stereo pair) due to custom I/O configuration that is default for VOXL2 varian 0 (which is what you have on Sentinel).

If you want to swap tracking and hires camera locations, you can just swap all the instructions as well

When connecting interposer and cameras, pay attention to pin1 orientaion, which is labeled on every camera connector, voxl2 connectors and interposers.

In terms of software setup:

• back up (move) all sensormodule files from /usr/lib/camera/
• copy the following sensormodule files that you will find in /usr/share/modalai/chi-cdk (i will assume that you are using imx214 hires camera):

cp /usr/share/modalai/chi-cdk/ov7251/com.qti.sensormodule.ov7251_0.bin /usr/lib/camera/
cp /usr/share/modalai/chi-cdk/boson/com.qti.sensormodule.boson_2.bin /usr/lib/camera/
cp /usr/share/modalai/chi-cdk/ov64b40/com.qti.sensormodule.ov64b40_3.bin /usr/lib/camera/
cp /usr/share/modalai/chi-cdk/imx214/com.qti.sensormodule.imx214_4.bin /usr/lib/camera/

confirm the following are present in /usr/lib/camera/"

com.qti.sensor.ov7251.so
com.qti.sensor.ov64b40.so
com.qti.sensor.boson.so
com.qti.sensor.imx214.so

Then, before updating /etc/modalai/voxl-camera-server.conf you can see if the cameras are detected:

voxl-camera-server -l

See output example here :

• https://docs.modalai.com/debugging-camera-issues/#checking-which-cameras-are-detected-by-camera-server
• https://docs.modalai.com/debugging-camera-issues/#sensor-id-to-sensor-type-mapping

For camera server config, use the current config that you already have, wipe any cameras from it that are not tracking or hires. Then add the two camera entries for Hadron into your config, as shown here : https://docs.modalai.com/voxl2-hadron/#minimum-config .

You will need to make sure the camera IDs specified in voxl-camera-server.conf are correct:
0: ov7251
1: boson
2: ov64b
3: imx214

At this point everything should work, once you get here, I can help you customize the video streams as you need.

Alex