Two-camera VIO non-functional in voxl-open-vins-server 0.6.0 (SDK 1.6.3)
-
System Info
Hardware: VOXL2 Mini (M0104)
SDK: 1.6.3
voxl-open-vins-server: 0.6.0
libmodal-flow: 1.0.3
libmodal-cv: 0.6.0
Cameras: 4x AR0144 (ON Semi) via M0188 expansion boardBackground
Building a custom GPS-denied navigation system targeting 4 AR0144 IR cameras in VIO. All 4 cameras are individually confirmed working as cam0 with good feature counts (80–160 features each, clean init). Single-camera VIO works but two-camera VIO is non-functional.Issue 1: cam1 always returns 0 features (internal tracker)
With two cameras in vio_cams.conf, cam1 returns 0 features and NaN disparity every frame regardless of which physical camera occupies the slot. Bug follows the VIO slot, not the hardware. Confirmed not a pipe, cal file, GPU/CPU, or config issue.
Enabling en_ext_feature_tracker: true causes cam1 to detect features (58–75), confirming the internal tracker's cam1 slot is broken.Issue 2: Camera fusion timeouts — two cameras never synchronized
With two cameras enabled, VIO never fuses both streams. The ready mask never reaches 0x3:
Camera fusion timeout (ready mask: 0x1, expected: 0x3)
Camera fusion timeout (ready mask: 0x2, expected: 0x3)
Camera fusion timeout (ready mask: 0x0, expected: 0x3)
This occurs with both internal and external tracker. Increasing fusion_rate_dt_ms to 40ms did not help.Questions
Is there a known fix or newer package available? apt list --upgradable shows nothing.
Is there a separate package feed for SDK 1.6.3 we should be pulling from?
Is fusion_rate_dt_ms the right parameter to fix the synchronization issue, and if so what value is recommended for two unsynchronized AR0144 cameras?
Our goal is 4-camera simultaneous VIO — is this supported on SDK 1.6.3? -
System Info
Hardware: VOXL2 Mini (M0104)
SDK: 1.6.3
voxl-open-vins-server: 0.6.0
libmodal-flow: 1.0.3
libmodal-cv: 0.6.0
Cameras: 4x AR0144 (ON Semi) via M0188 expansion boardBackground
Building a custom GPS-denied navigation system targeting 4 AR0144 IR cameras in VIO. All 4 cameras are individually confirmed working as cam0 with good feature counts (80–160 features each, clean init). Single-camera VIO works but two-camera VIO is non-functional.Issue 1: cam1 always returns 0 features (internal tracker)
With two cameras in vio_cams.conf, cam1 returns 0 features and NaN disparity every frame regardless of which physical camera occupies the slot. Bug follows the VIO slot, not the hardware. Confirmed not a pipe, cal file, GPU/CPU, or config issue.
Enabling en_ext_feature_tracker: true causes cam1 to detect features (58–75), confirming the internal tracker's cam1 slot is broken.Issue 2: Camera fusion timeouts — two cameras never synchronized
With two cameras enabled, VIO never fuses both streams. The ready mask never reaches 0x3:
Camera fusion timeout (ready mask: 0x1, expected: 0x3)
Camera fusion timeout (ready mask: 0x2, expected: 0x3)
Camera fusion timeout (ready mask: 0x0, expected: 0x3)
This occurs with both internal and external tracker. Increasing fusion_rate_dt_ms to 40ms did not help.Questions
Is there a known fix or newer package available? apt list --upgradable shows nothing.
Is there a separate package feed for SDK 1.6.3 we should be pulling from?
Is fusion_rate_dt_ms the right parameter to fix the synchronization issue, and if so what value is recommended for two unsynchronized AR0144 cameras?
Our goal is 4-camera simultaneous VIO — is this supported on SDK 1.6.3?@cbay , based on discussion from the other thread (https://forum.modalai.com/topic/5130/running-4-ar0144s-on-m0188), you are running two AR0144 cameras that are synchronized and two unsynchronized.
Can you please clarify which cameras you are using when testing the two-camera open vins setup? If you only have the 4 AR0144 plugged in, then ID0 and ID3 will be synchronized (camera slots 0 and 6) and the other two cameras (cam id 1, 2 - slots 2,3).
Alex
-
@cbay , based on discussion from the other thread (https://forum.modalai.com/topic/5130/running-4-ar0144s-on-m0188), you are running two AR0144 cameras that are synchronized and two unsynchronized.
Can you please clarify which cameras you are using when testing the two-camera open vins setup? If you only have the 4 AR0144 plugged in, then ID0 and ID3 will be synchronized (camera slots 0 and 6) and the other two cameras (cam id 1, 2 - slots 2,3).
Alex
@Alex-Kushleyev said in Two-camera VIO non-functional in voxl-open-vins-server 0.6.0 (SDK 1.6.3):
Can you please clarify which cameras you are using when testing the two-camera open vins setup? If you only have the 4 AR0144 plugged in, then ID0 and ID3 will be synchronized (camera slots 0 and 6) and the other two cameras (cam id 1, 2 - slots 2,3).
I think there may be some confusion, I am running the 4 camera's on J0, J1, J2, and J3 not J6. I have none of the camera;s setup with fsin drivers, just the standard ones. I am happy to swap these to fsin but that was where my problems were for getting frames.
If it is necessary to fsin for OpenVins please let me know, and then we can go through the process of syncing all of them.
-
@Alex-Kushleyev said in Two-camera VIO non-functional in voxl-open-vins-server 0.6.0 (SDK 1.6.3):
Can you please clarify which cameras you are using when testing the two-camera open vins setup? If you only have the 4 AR0144 plugged in, then ID0 and ID3 will be synchronized (camera slots 0 and 6) and the other two cameras (cam id 1, 2 - slots 2,3).
I think there may be some confusion, I am running the 4 camera's on J0, J1, J2, and J3 not J6. I have none of the camera;s setup with fsin drivers, just the standard ones. I am happy to swap these to fsin but that was where my problems were for getting frames.
If it is necessary to fsin for OpenVins please let me know, and then we can go through the process of syncing all of them.
@cbay , thanks for clarifying
On M0188, J1 and J3 do not have the sync signal connected. We will release a diagram shortly showing where to install 0-ohm resistors on M0188 to connect the sync signal to the corresponding camera ports.
And you will need fsin drivers for J0, J1, J2, J3 (will share them shortly as well)
By default, the AR0144 tracking cameras are connected to J0 and J6 (camera slots 0, 6), which are actually connected to a single CSI interface and are used in a combo mode (two 1-lane (mipi lane) cameras merged into a single 4-lane mipi interface).
Note that when the
fsinorcombo(combo mode drivers also use input trigger) are used, but there is no actual sync / trigger signal getting to the camera, then there will be no frames produced. This is specific to the AR0144 camera, as other cameras may operate differently and use the sync signal to align the readout trigger (so in absence of trigger signal, those cameras would simply be not sync'ed, but AR0144 just does not send any frames)Alex
-
@cbay , thanks for clarifying
On M0188, J1 and J3 do not have the sync signal connected. We will release a diagram shortly showing where to install 0-ohm resistors on M0188 to connect the sync signal to the corresponding camera ports.
And you will need fsin drivers for J0, J1, J2, J3 (will share them shortly as well)
By default, the AR0144 tracking cameras are connected to J0 and J6 (camera slots 0, 6), which are actually connected to a single CSI interface and are used in a combo mode (two 1-lane (mipi lane) cameras merged into a single 4-lane mipi interface).
Note that when the
fsinorcombo(combo mode drivers also use input trigger) are used, but there is no actual sync / trigger signal getting to the camera, then there will be no frames produced. This is specific to the AR0144 camera, as other cameras may operate differently and use the sync signal to align the readout trigger (so in absence of trigger signal, those cameras would simply be not sync'ed, but AR0144 just does not send any frames)Alex
I have uploaded the latest ar0144 drivers with
fsinversions for all camera slots here : https://storage.googleapis.com/modalai_public/temp/ar0144/ar0144_drivers_20260402.zipThere are two additional files (inside the zip), which you should copy to
/usr/lib/camerato make sure you have the latest updates:com.qti.sensor.ar0144.so-- contains functions for exposure / gain control (we recently made some improvements to make gain control smoother)com.qti.tuned.default.bin-- fixed gain scaling so that min gain (1.0x analog gain) is equal to 100 in the HAL3 gain units, not 54 (and the max gain will be 29.6 = 29600 for AR0144) -- you can also double check this usingvoxl-camera-server -land update the min/max gain settings in yourvoxl-camera-server.confto make sure you are using the full range.
My colleague will follow up with a diagram for locations of the DNI resistors that need to be installed to enable sync signal for camera slots 1 and 3 (0402 0-ohm resistor)
Alex
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login