Non standard camera configuration on VOXL running 4.0.0-0.9.4

[czarsimon]

Hi, I'm trying to get the following camera setup working with the latest 4.0.0 system image and voxl-sdk v0.9.4:

J2 - IMX214 hires
J3 - OV7251 stereo
J4 - TOF

I am using the following voxl-camera-server.conf:

{
	"version":	0.1,
	"cameras":	[{
			"name":	"hires",
			"enabled":	true,
			"frame_rate":	30,
			"type":	"imx214",
			"camera_id":	0,
			"preview_width":	640,
			"preview_height":	480,
			"snapshot_width":	3840,
			"snapshot_height":	2160
		},{
			"name":	"tof",
			"enabled":	true,
			"frame_rate":	5,
			"type":	"pmd-tof",
			"camera_id":	1
		}, {
			"name":	"stereo",
			"enabled":	true,
			"frame_rate":	30,
			"type":	"ov7251",
			"camera_id":	2,
			"camera_id_second":	3,
			"independent_exposure":	false,
			"ae_desired_msv":	60,
			"ae_filter_alpha":	0.6,
			"ae_ignore_fraction":	0.2,
			"ae_slope":	0.05,
			"ae_exposure_period":	1,
			"ae_gain_period":	1
		}]
}

And this is the output that I get when running the server:

voxl:/$ voxl-camera-server
WARNING: Camera stereo recieved much newer child than master (41), discarding master and trying again
ERROR:   Recieved Fatal error from camera: hires
ERROR:   Sending request 12, ErrorCode: -ENODEV
WARNING: Thread: hires result thread recieved ESTOP
WARNING: Thread: hires request thread recieved ESTOP
WARNING: Thread: tof result thread recieved ESTOP
WARNING: Thread: stereo_child result thread recieved ESTOP
WARNING: Thread: stereo result thread recieved ESTOP
WARNING: Thread: stereo request thread recieved ESTOP
WARNING: Thread: tof request thread recieved ESTOP
WARNING: Thread: stereo_child request thread recieved ESTOP

------ voxl-camera-server INFO: Camera server is now stopping
WARNING: Deleting buffers: 4 of 16 still in use
WARNING: Deleting buffers: 3 of 16 still in use
WARNING: Deleting buffers: 983049 of 983061 still in use

Segmentation fault:
Fault thread: voxl-camera-ser(tid: 3493)
Fault address: 0x1
Address not mapped.
Segmentation fault
voxl:/$

All cameras seem to work fine independently on these ports when only one is enabled in the configuration just not all together for some reason. How can I get all three of these cameras working at the same time? Thank you

Edit: it seems that the hires camera is causing issues, by itself its fine but together with the other two it causes a crash. Stereo and TOF work fine together though.

[Chad Sweet]

Hey Simon,

Each new configuration requires some work and rebuilding the kernel to enable.

Have you tried the standard TOF configuration here?

[czarsimon]

Hi @Chad-Sweet, the link you sent is for the VOXL 2, I'm trying to get this working on the VOXL. It feels like this was easier with the previous system image (3.8.0) which required only a customized voxl-camera-server.conf and a modified /system/etc/camera/camera_config.xml to enable the sensors on different ports (apart from the fact that stereo only works on J3). We have the hires and stereo working on J2 and J3 respectively on 3.8.0, the only thing missing was the TOF which is why we upgraded to 4.0.0 but it seems to have broken the existing setup that worked before. Any ideas as to why this might be the case?

Edit: it seems like there is no configuration which supports the camera combination we are trying to use, can you confirm if it is possible to use hires, stereo, and TOF on the latest VOXL image?

Edit2: even a stock configuration doesn't seem to be working, I have hires on J2 and stereo on J3 and using voxl-configure-cameras value of 13 which is 13 Hires + Stereo(OV7251) and I get the same errors:

voxl:/$ voxl-camera-server

------ voxl-camera-server: Camera server is now running
WARNING: Camera stereo recieved much newer child than master (49), discarding master and trying again
ERROR:   Recieved Fatal error from camera: hires
ERROR:   Sending request 4, ErrorCode: -ENODEV
WARNING: Thread: hires result thread recieved ESTOP
WARNING: Thread: hires request thread recieved ESTOP
WARNING: Thread: stereo result thread recieved ESTOP
WARNING: Thread: stereo_child result thread recieved ESTOP
WARNING: Thread: stereo_child request thread recieved ESTOP
WARNING: Thread: stereo request thread recieved ESTOP

------ voxl-camera-server INFO: Camera server is now stopping
WARNING: Deleting buffers: 4 of 16 still in use
WARNING: Deleting buffers: 983048 of 983061 still in use

Segmentation fault:
Fault thread: voxl-camera-ser(tid: 3179)
Fault address: 0x1
Address not mapped.
Segmentation fault
voxl:/$

Edit 3: full log of voxl-camera-server -d 0 with hires and stereo: https://pastebin.com/kFm6mar6. Key line seems to be ERROR: Recieved Fatal error from camera: hires which falls in line with my observations of the hires camera being the one causing issues as stereo and TOF work fine without it.

Edit 4: it's definitely something with the system image, I downgraded voxl-camera-server all the way down to 1.1.2 and the hires+stereo config is still not working.

[Chad Sweet]

We have not done any work to enable this configuration on VOXL 1. The more recent system image harmonized some things between VOXL 1 and VOXL 2 in regards to TOF, but it did not add any new TOF features.

[A Former User]

@czarsimon

The ENODEV issue means that there's a hardware issue with the sensor. Often this is caused by a torn ribbon cable or damage to the connector on the board. The HAL layer maintains a queue of 3 requests past the one actively being processed, so the error popping up on frame 4 is usually an indicator that the connection to the sensor is good enough to identify that it exists but there's an issue actually requesting the frames. You mentioned in your other post that the issue was seen on both port J2 and J4 indicating that it's probably an issue with the camera's cable. Can you look at the ribbon cable and make sure that it hasn't been torn or bent past it's spec (diameter 20x the thicknesss of the cable)

[czarsimon]

Thanks for the hint @Alex-Gardner, I tried two separate modules on 3.3.0, 3.6.0, and 3.8.0 and they worked just fine, it is only with system image 4.0.0 and voxl-sdk 0.9 that neither one works which makes me conclude that it's not the sensor but something with the software. Also both modules works just fine when run by themselves, it is only in conjunction with other sensors like stereo/tof that the hires is causing issues.