IMX412 Colour

[jcai]

Hi, I have a single IMX412 connected to M0173 J5 on a VOXL2.

com.qti.sensormodule.imx412_fpv_2.bin is the only sensormodule file I have in /usr/lib/camera.

This is my camera server config file:

/**
 * voxl-camera-server Configuration File
 *
 * Each camera has configurations for up to 4 HAL3 streams:
 *    - `preview` stream for raw unprocessed images from CV cameras
 *    - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming
 *    - `large_video` 4k (ish) h264/h265 for onboard video recording to disk
 *    - `snapshot` ISP-processed JPG snapshots that get saved to disk
 *
 * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled
 *
 * This file is generated from default values by voxl-configure-cameras.
 * Do not expect arbitrary resolutions to work, the ISP and video compression
 * pipelines only support very specific resolutions.
 *
 * The default video compression mode is cqp or Constant Quantization Parameter
 *
 *
 *
 */
{
	"version":	0.1,
	"fsync_en":	true,
	"fsync_gpio":	109,
	"cameras":	[{
			"type":	"imx412",
			"name":	"hires_front",
			"enabled":	true,
			"camera_id":	0,
			"fps":	30,
			"en_preview":	false,
			"preview_width":	640,
			"preview_height":	480,
			"en_raw_preview":	false,
			"en_small_video":	true,
			"small_video_width":	640,
			"small_video_height":	480,
			"small_venc_mode":	"h265",
			"small_venc_br_ctrl":	"cqp",
			"small_venc_Qfixed":	30,
			"small_venc_Qmin":	15,
			"small_venc_Qmax":	40,
			"small_venc_nPframes":	9,
			"small_venc_mbps":	2,
			"en_large_video":	false,
			"large_video_width":	4056,
			"large_video_height":	3040,
			"large_venc_mode":	"h265",
			"large_venc_br_ctrl":	"cqp",
			"large_venc_Qfixed":	38,
			"large_venc_Qmin":	15,
			"large_venc_Qmax":	50,
			"large_venc_nPframes":	29,
			"large_venc_mbps":	30,
			"en_snapshot":	false,
			"en_snapshot_width":	4056,
			"en_snapshot_height":	3040,
			"exif_focal_length":	3.0999999046325684,
			"exif_focal_length_in_35mm_format":	17,
			"exif_fnumber":	1.2400000095367432,
			"ae_mode":	"isp"
		}]
}

The issue I'm having has to do with the colour of the video. Indoors, the image looks fine but outdoors, there's a red tint that I can't seem to get rid of. I'm using isp currently, but I've also played around with lme_msv, to not much improvement. Is this behaviour normal?

[Alex Kushleyev]

@jcai , can you please try the following:

remove com.qti.sensormodule.imx412_fpv_2.bin (or rename to com.qti.sensormodule.imx412_fpv_2.bin.bak and copy the following to /usr/lib/camera:

cp /usr/share/modalai/chi-cdk/imx412/com.qti.sensormodule.imx412_2.bin /usr/lib/camera/

then restart camera server.

This other sensormodule driver will use another tuning file for image processing. Let me know if that works better for you, I can explain more details about what's going on.

Alex

[Alex Kushleyev]

Another thing to check.. does your IMX412 camera have IR filter installed or not? You should know this based on the part number of IMX412 that you purchased.

If there is no IR filter, it is possible that you will see more red tint in the image. If this is the case, we can discuss how to fix it.

Alex

[jcai]

@Alex-Kushleyev I've tried the other driver, and the result looks largely the same. It's a bit brighter. (Cropped for file size)
My sensor indeed doesn't have the IR filter.

[Alex Kushleyev]

@jcai , thank you for trying.

The issue here is that the tuning files we have for the Qualcomm ISP assume that the camera has a IR filter, so the AWB (auto white balance) algorithm in the camera pipeline is not able to compensate the significant difference in RGB levels with the IR filter removed.

The tuning file is pretty much a black box for the end user, deviating from the original camera parameters / lens specs (which were used for the tuning) can leave to image degradation or color variations.

We have been working on our own processing pipeline (MISP) which bypasses the ISP and performs debayering, color balance, etc. I will need to document a few steps so that you can try it out.

Does your use case require the full frame resolution, or what output resolution are you looking for?

Alex

[jcai]

@Alex-Kushleyev Thanks for the info. For now, we're going to swap the sensor for one with the filter to see how it performs. I'll be updating results here

[jcai]

Update: the filter was absolutely the difference maker