Rotate imx412 stream

TomP

@TomP Any ideas? There should be a rotation parameter somewhere.

Alex Kushleyev

Are you using MISP in your config file or the Qualcomm ISP?

there are typically two ways to rotate the image stream:

read out the image backwards (typically a one or two register change in the camera configuration)
- reverse readout will also typically reverse the bayer pattern (RGGB vs BGGR), so whatever is doing the debayering also needs to be aware of the bayer pattern change
rotate image post readout in ISP or GPU, although i believe even the Qualcomm pipeline does the rotation in the GPU (however we have never used the rotation in the QC pipeline within voxl-camera-server).

For some cameras like OV7251, OV9782, AR0144, the en_rotate flag from voxl-camera-server.conf causes the appropriate registers to be updated to set the reverse readout and then the same flag is also used for correctly debayering (in ov9782 case). https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/cci_direct_helpers.cpp?ref_type=heads#L73

For the IMX412 we have not made that option available yet, but i know it is trivial to do, so i can make that change.

If you do use the QC ISP pipeline (using small_encoded or large_encoded streams), in order to make this work, we actually generate another sensormodule driver that sets the reverse read-out registers and informs the isp pipeline that the Bayer pattern is reversed. This is not ideal because we need to maintain another set of sensormodule drivers.. Ideally, there would be a flag in ISP pipeline to cause the rotation, but the following does not seem to work: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/hal3_camera_mgr.cpp?ref_type=heads#L87 (the flag used to set rotation in stream requests below in the same file)

Please let me know which streams you are actually using for IMX412 and I can help you.

Alex

TomP

@Alex-Kushleyev We use the ISP mode (“ae_mode”: “isp”) and the hires_small_encoded pipeline.
Thank you very much for your support.

Alex Kushleyev

@TomP , ok no problem. I can add rotated sensormodules to our release and send you a copy. One more question - can you please provide me exact sensormodule name for imx412 that you have in /usr/lib/camera ?

TomP

@Alex-Kushleyev The most current file name in the list in the directory is com.qti.sensormodule.imx412_flip_2.bin.

We have ordered and installed this camera:

Image Sensor 4k High-resolution, Low-light Sensor for VOXL (Starvis IMX412 w/ micro-coax & M12-style Lens) (MSU-M0161)

Alex Kushleyev

@TomP , actually there is already non-flip version in /etc/modalai/chi-cdk/imx412 . You can just use the sensormodule from that location with the same slot id (index at the end of sensormodule). Back up the current sensormodule (move it out of /usr/lib/camera/ or rename to .bak) and put the new one in the same location.

Alex

TomP

@Alex-Kushleyev the chi-cdk directory does not exist. here is the content of the /etc/modalai/ directory:

voxl2-mini:/etc/modalai$ ls
extrinsics.conf voxl-camera-server.conf voxl-px4-hitl-set-default-parameters.config
extrinsics.conf.thermal voxl-cpu-monitor.conf voxl-px4.conf
lost+found voxl-dfs-server.conf voxl-streamer.conf
network_interface_preferences.txt voxl-elrs.conf voxl-tag-detector.conf
tag_locations.conf voxl-imu-server.conf voxl-tflite-server.conf
vio_cams.conf voxl-mavlink-server.conf voxl-uvc-server.conf
vio_cams.conf.boson voxl-modem.conf voxl-vision-hub.conf
vio_cams.conf.seek voxl-px4-fake-imu-calibration.config voxl-vtx.conf

TomP

@Alex-Kushleyev I have now found the correct file in the /usr/share/modalai/chi-cdk directory. Now everything works as it should. Many thanks for your support!

Alex Kushleyev

@TomP , oh yeah, my bad.. you found the right one

Jetson Nano

@Alex-Kushleyev hey ,I had the same issue, I tried following the steps mentioned above, still the video stream rotation was not successful.
When i tried to replace the file in usr/lib/camera with the file in chi-cdk, it gave the error of hires not starting and no video. The default file in usr/lib/camera was for fpv. I tried to change it with normal imx412 file, it gave error i mentioned above.
Also the hires ID is 1.