Camera specific tuning files and parsing binaries for fine-tuning

IlkkaN

Hi,
we know that VOXL 2 uses a QRB5165 chipset based on the UBUN1.0 release and that you have been working on some camera-specific tuning files for example to imx678.
Can you provide more details
1.how did you do that, by Chromatix tool from QC or something else?
2.Can you provide parameterparser.exe and chromatix api files from your ubun1.0 build (probably form chi-cdk\tools\buildbins and chi-cdk\api\chormatix) that can used to parse tuning binaries?
Or should we contact QC directly?

IlkkaN

@IlkkaN
I was able to find to ubun1.0 & ubun2.0 packages (qrb5165-ubun-1-0_hlos_dev_cam-sdk-master etc). Chromatix api & parser were found under lu.um.1.2.1 -folder (apps_proc/src/vendor/qcom/proprietary/chi-cdk).
With those files I was able to parse imx412 ready-made binary (com.qti.tuned.imx412_fpv.bin) and create my own project based on that which actually work in our device (according to QC QCS8250/Spectra480 can be used to tune QBR5165).

However there is no Chromatix project under \chi-cdk\oem\qcom\tuning for neither imx412 nor imx678. Also no tuning binary for imx678 at all.
Do you have Chromatix projects for those modules and/or can you provide imx678 tuning binary?

Alex Kushleyev

Hi @IlkkaN ,

The com.qti.tuned.imx412_fpv.bin tuning file was created in attempt to remove some of processing in order to reduce delay and tweak / reduce de-noise / smoothing. I believe the starting point was com.qti.tuned.cmk_imx577.bin which was provided by Qualcomm (you may be able to find that one). IMX577 and IMX412 are nearly identical cameras.

We do not have a tuning file for IMX678 and currently the tuning file name is not specified in the IMX678 sensormodules, which means that com.qti.tuned.default.bin will be used. The default tuning file is the fallback in case the tuning file name specified in the sensormodule file is not found. The default tuning file is not tuned for any specific camera and should be used with caution :).

If you want to confirm which tuning file is used by each camera / sensormodule, you can check this the following way:

add to /vendor/etc/camera/camxoverridesettings.txt:
logInfoMask=0x20000

then run

logcat | grep -i tuning

and then start voxl-camera-server, you should see output like this:

03-30 01:57:45.256  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1762 CreateTuningDataManager() Enter : sensor index: 0, sensor name: cmk_imx577, chromatix name: cmk_imx577
03-30 01:57:45.256  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1798 CreateTuningDataManager() Searching regular tuning file
03-30 01:57:45.262  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1839 CreateTuningDataManager() Loaded tuning file: /usr/lib/camera/com.qti.tuned.cmk_imx577.bin for sensor index: 0
03-30 01:57:45.301  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1762 CreateTuningDataManager() Enter : sensor index: 1, sensor name: ar0144, chromatix name: onsemi_ar0144
03-30 01:57:45.301  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1798 CreateTuningDataManager() Searching regular tuning file
03-30 01:57:45.310  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1839 CreateTuningDataManager() Loaded tuning file: /usr/lib/camera/com.qti.tuned.onsemi_ar0144.bin for sensor index: 1

Our approach is currently not to heavily rely on the ISP tuning, instead we are building our own light-weight image processing pipeline, called MISP, which does processing on the GPU (from RAW bayer frames). MISP also allows us to create up to 6 high resolution color video pipelines, as opposed to the limit of 2 cameras (in video mode) supported by Spectra480 ISP, since it has two separate full VFEs.

By using MISP, we have full control over the algorithms that are running and we can easily share details with the customers. However, MISP definitely lacks many features that are present in the hardware ISP.

Please let me know if you have any other questions.

Alex