Using Qualcomm's camera subsystem on VOXL2

modaltb

We aren't supporting gst* stuff necessarily as our focus is via our camera-server as an attempt make general development on top of the QC stack easier over time. That said it is how I tested cameras initially and the framework is there and "works" as far as my use cases went, but it's a heavy lift sometimes going through the QC docs to find what you need.

For example, here's what I used to capture frames to disk to validate a ov7251 sensor:

gst-launch-1.0 -e qtiqmmfsrc camera=0 name=tracker ! video/x-h264,format=NV12,width=640,height=480,framerate=30/1 ! queue ! h264parse ! avdec_h264 ! videoconvert ! pngenc ! multifilesink enable-last-sample=false location="/data/misc/camera/cam0_img_%d_.png"

Now the gst- command line stuff is totally out of my wheel house, I generally copy paste commands here just to get my stuff done, but if you are familiar with gst- stuff then you should be able to find what you need on target.

AnthoJack

Great, thank you. I wasn't searching for the right name for the src module ("qmmfsrc" instead of "qtiqmmfsrc"). I'm not exactly knowledgeable about Gstreamer myself, I was just looking for a way to recover and process the images that allowed for a bit more control and configurability to do some tests.

You do seem to support gst stuff though since voxl-streamer uses it

paynej

I've been working on another RB5 platform and am migrating to VOXL. I used a command similar to above:

GST_DEBUG=3 gst-launch-1.0 -e qtiqmmfsrc camera=0 name=tracker ! video/x-h264,format=NV12,width=640,height=480,framerate=30/1 ! queue ! h264parse ! avdec_h264 ! videoconvert ! pngenc ! multifilesink enable-last-sample=false location="/data/misc/camera/cam0_img_%d_.png"

I get images, yet they are greyscale instead of color. Any help would be great!

Jeff

Chad Sweet

Which camera config are you using? https://docs.modalai.com/voxl2-camera-configs/

Are you sure it's not a monochrome image sensor you are using?

paynej

Should be hires and tracking only. I'm trying to use the color hires one. I noticed when using the voxl-streamer, that the tracking one is probably monochrome.
An older /etc/modalai/voxl-camera-server.conf.save shows all 4 cameras, tracking, hires and stereo.

Thanks for your reply. I hope to have this figured out soon.

paynej

C11. I'm trying to use the color hires one. I noticed when using the voxl-streamer, that the tracking ones and stereo is monochrome.

So now

GST_DEBUG=3 gst-launch-1.0 -e qtiqmmfsrc camera=3 ! video/x-raw,format=NV12,width=640,height=480,framerate=2/1 ! queue  ! videoconvert ! jpegenc ! multifilesink enable-last-sample=false location="/data/misc/camera/cam0_img_%d_.jpg"

is giving me no image.

All other cameras (0,1,2,4,5) are working correctly with B/W

If I enable voxl-camera-server and voxl-streamer-hires, then I get a RTSP color stream, so the HW seems good.

Thanks for your reply. I hope to have this figured out soon.

paynej

I think I have the snapshot hires color working! I was grabbing the first picture instead of the last. The first one, the sensor must not be ready.

paynej

Back again...

I have my ML program running now with the qtiqmmfscrc pipeline and disabling voxl-camera-server. I'd like to run voxl-camera-server with just the tracking and stereo cameras so voxl-qvio-server will get data. I tried to remove just the hires camera from /etc/modalai/voxl-camera-server.conf but running voxl-camera-server aborts:

rb5:/$ /usr/bin/voxl-camera-server -d 0
No secondary id found for camera: tracking, assuming mono
Secondary id found for camera: stereo_front, assuming stereo
Secondary id found for camera: stereo_rear, assuming stereo
------ voxl-camera-server: Starting camera server
Attempting to open the hal module
SUCCESS: Camera module opened on attempt 0
Aborted

rb5:/$ more /etc/modalai/voxl-camera-server.conf
{
	"version":	0.1,
	"cameras":	[{
			"name":	"tracking",
			"enabled":	true,
			"frame_rate":	30,
			"type":	"ov7251",
			"camera_id":	2,
			"ae_desired_msv":	60,
			"ae_filter_alpha":	0.600000023841858,
			"ae_ignore_fraction":	0.20000000298023224,
			"ae_slope":	0.05000000074505806,
			"ae_exposure_period":	1,
			"ae_gain_period":	1
		}, {
			"name":	"stereo_front",
			"enabled":	true,
			"frame_rate":	30,
			"type":	"ov7251",
			"camera_id":	0,
			"camera_id_second":	1,
			"ae_desired_msv":	60,
			"ae_filter_alpha":	0.600000023841858,
			"ae_ignore_fraction":	0.20000000298023224,
			"ae_slope":	0.05000000074505806,
			"ae_exposure_period":	1,
			"ae_gain_period":	1
		}, {
			"name":	"stereo_rear",
			"enabled":	true,
			"frame_rate":	30,
			"type":	"ov7251",
			"camera_id":	5,
			"camera_id_second":	4,
			"ae_desired_msv":	60,
			"ae_filter_alpha":	0.600000023841858,
			"ae_ignore_fraction":	0.20000000298023224,
			"ae_slope":	0.05000000074505806,
			"ae_exposure_period":	1,
			"ae_gain_period":	1
		}]
}

Any advice? Thank you for your time.

Chad Sweet

The Qualcomm HAL3 camera framework only allows one application to connect, which is why we created voxl-camera-server. What you want to do is not possible due to this limitation from Qualcomm

paynej

Thank you for the reply. That makes sense.

What are the available outputs of voxl-camera-server? It looks like voxl-qvio-server uses a pipe. Wondering if I have another option to qtiqmmfscrc, rtsp and udp.

Thanks you, I appreciate you taking the time to reply.

Chad Sweet

that is documented here: https://docs.modalai.com/voxl-camera-server/#output-of-camera-server

paynej

The Pipe API worked great! Thanks.

Can the hires camera output RBG instead of NV12 and can voxl-camera-server support that?