ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Rotate imx412 stream

    Video and Image Sensors
    3
    16
    776
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Alex KushleyevA
      Alex Kushleyev ModalAI Team @TomP
      last edited by Alex Kushleyev

      @TomP ,

      Are you using MISP in your config file or the Qualcomm ISP?

      there are typically two ways to rotate the image stream:

      • read out the image backwards (typically a one or two register change in the camera configuration)
        • reverse readout will also typically reverse the bayer pattern (RGGB vs BGGR), so whatever is doing the debayering also needs to be aware of the bayer pattern change
      • rotate image post readout in ISP or GPU, although i believe even the Qualcomm pipeline does the rotation in the GPU (however we have never used the rotation in the QC pipeline within voxl-camera-server).

      For some cameras like OV7251, OV9782, AR0144, the en_rotate flag from voxl-camera-server.conf causes the appropriate registers to be updated to set the reverse readout and then the same flag is also used for correctly debayering (in ov9782 case). https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/cci_direct_helpers.cpp?ref_type=heads#L73

      For the IMX412 we have not made that option available yet, but i know it is trivial to do, so i can make that change.

      If you do use the QC ISP pipeline (using small_encoded or large_encoded streams), in order to make this work, we actually generate another sensormodule driver that sets the reverse read-out registers and informs the isp pipeline that the Bayer pattern is reversed. This is not ideal because we need to maintain another set of sensormodule drivers.. Ideally, there would be a flag in ISP pipeline to cause the rotation, but the following does not seem to work: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/hal3_camera_mgr.cpp?ref_type=heads#L87 (the flag used to set rotation in stream requests below in the same file)

      Please let me know which streams you are actually using for IMX412 and I can help you.

      Alex

      T 1 Reply Last reply Reply Quote 0
      • T
        TomP @Alex Kushleyev
        last edited by

        @Alex-Kushleyev We use the ISP mode (“ae_mode”: “isp”) and the hires_small_encoded pipeline.
        Thank you very much for your support. 🙂

        Alex KushleyevA 1 Reply Last reply Reply Quote 0
        • Alex KushleyevA
          Alex Kushleyev ModalAI Team @TomP
          last edited by

          @TomP , ok no problem. I can add rotated sensormodules to our release and send you a copy. One more question - can you please provide me exact sensormodule name for imx412 that you have in /usr/lib/camera ?

          T 1 Reply Last reply Reply Quote 0
          • T
            TomP @Alex Kushleyev
            last edited by

            @Alex-Kushleyev The most current file name in the list in the directory is com.qti.sensormodule.imx412_flip_2.bin.

            We have ordered and installed this camera:

            Image Sensor 4k High-resolution, Low-light Sensor for VOXL® (Starvis IMX412 w/ micro-coax & M12-style Lens) (MSU-M0161)

            Alex KushleyevA 1 Reply Last reply Reply Quote 0
            • Alex KushleyevA
              Alex Kushleyev ModalAI Team @TomP
              last edited by

              @TomP , actually there is already non-flip version in /etc/modalai/chi-cdk/imx412 . You can just use the sensormodule from that location with the same slot id (index at the end of sensormodule). Back up the current sensormodule (move it out of /usr/lib/camera/ or rename to .bak) and put the new one in the same location.

              Alex

              T 1 Reply Last reply Reply Quote 0
              • T
                TomP @Alex Kushleyev
                last edited by TomP

                @Alex-Kushleyev the chi-cdk directory does not exist. here is the content of the /etc/modalai/ directory:

                voxl2-mini:/etc/modalai$ ls
                extrinsics.conf voxl-camera-server.conf voxl-px4-hitl-set-default-parameters.config
                extrinsics.conf.thermal voxl-cpu-monitor.conf voxl-px4.conf
                lost+found voxl-dfs-server.conf voxl-streamer.conf
                network_interface_preferences.txt voxl-elrs.conf voxl-tag-detector.conf
                tag_locations.conf voxl-imu-server.conf voxl-tflite-server.conf
                vio_cams.conf voxl-mavlink-server.conf voxl-uvc-server.conf
                vio_cams.conf.boson voxl-modem.conf voxl-vision-hub.conf
                vio_cams.conf.seek voxl-px4-fake-imu-calibration.config voxl-vtx.conf

                T 1 Reply Last reply Reply Quote 0
                • T
                  TomP @TomP
                  last edited by TomP

                  @Alex-Kushleyev I have now found the correct file in the /usr/share/modalai/chi-cdk directory. Now everything works as it should. Many thanks for your support!

                  Alex KushleyevA 1 Reply Last reply Reply Quote 0
                  • Alex KushleyevA
                    Alex Kushleyev ModalAI Team @TomP
                    last edited by

                    @TomP , oh yeah, my bad.. you found the right one 🙂

                    Jetson NanoJ 1 Reply Last reply Reply Quote 1
                    • Jetson NanoJ
                      Jetson Nano @Alex Kushleyev
                      last edited by Jetson Nano

                      @Alex-Kushleyev hey ,I had the same issue, I tried following the steps mentioned above, still the video stream rotation was not successful.
                      When i tried to replace the file in usr/lib/camera with the file in chi-cdk, it gave the error of hires not starting and no video. The default file in usr/lib/camera was for fpv. I tried to change it with normal imx412 file, it gave error i mentioned above.
                      Also the hires ID is 1.

                      Alex KushleyevA 1 Reply Last reply Reply Quote 0
                      • Alex KushleyevA
                        Alex Kushleyev ModalAI Team @Jetson Nano
                        last edited by

                        @Jetson-Nano ,

                        What was the error that you saw when you tried the rotated non-FPV driver for IMX412?

                        Alex

                        Jetson NanoJ 1 Reply Last reply Reply Quote 0
                        • Jetson NanoJ
                          Jetson Nano @Alex Kushleyev
                          last edited by

                          @Alex-Kushleyev
                          These are the files available
                          Screenshot from 2025-09-22 10-18-41.png

                          IMX412 has four options IMX412 , IMX412-flip , IMX412-fpv, IMX412-fpv-flip

                          IMX412-fpv is the only folder which contained option for camera 1 , other folders there was no option for camera number 1, so i created it by copy a file from that folder and renaming it for camera 1, I would like to know whether it was the right approach.
                          The highlighted option in the following images are the files I created.

                          IMX412
                          Screenshot from 2025-09-22 10-24-26.png

                          IMX412-flip
                          Screenshot from 2025-09-22 10-24-31.png

                          IMX412-fpv-misp
                          Screenshot from 2025-09-22 10-24-41.png

                          While running with with file form any of the three other options other than the imx412-fpv the camera was not detected .

                          Screenshot from 2025-09-22 10-25-55.png

                          Screenshot from 2025-09-22 10-26-45.png

                          I copied the file from the /usr/share/chi-cdk/imx412 *** to the /usr/lib/camera/ folder, after while I try to run the camera server, only the tracking and TOF is recognised, I don't know what is the error.
                          I am using the configuration 27 for camera setup, but I am only using 1 tracking, 1 hires and 1 TOF.
                          Hires is the camera 1

                          Screenshot from 2025-09-22 10-29-07.png

                          Alex KushleyevA 1 Reply Last reply Reply Quote 0
                          • Alex KushleyevA
                            Alex Kushleyev ModalAI Team @Jetson Nano
                            last edited by

                            @Jetson-Nano , I just posted the latest drivers for IMX412 in the following location, which includes the normal and rotated drivers : https://storage.googleapis.com/modalai_public/temp/imx412_test_bins/20250919/imx412_fpv_eis_20250919_drivers.zip

                            Please note that if you are using MISP, the rotated drivers will have the reversed bayer order (BGGR and not RGGB), so you will need to let MISP know about that by using the following parameter in voxl-camera-server.conf (otherwise your red and blue colors will be flipped):

                            "bayer_type" : "bggr"
                            

                            If the bayer type parameter does not fix the color order, then you will need to update voxl-camera-server, so that it includes the following commit : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commit/a58128d3da61c3aea19d6db6c22e7176a04ae1db

                            If you are using the ISP output, then it will know the bayer pattern type directly from the sensormodule file, so you do not need to add the bayer_type parameter.

                            To answer your question about renaming sensormodules - changing the name does not change its functionality. The slot IDs are part of the sensormodule, not the name, so you cannot create a new sensormodule by just renaming the file to switch the camera slot ID.

                            Please try the new drivers and if you are still having issues, please do the following:

                            • disable all the cameras except for your imx412 (in camera config)
                            • share your camera config section for the IMX412 camera
                            • share the output of the camera server when you run it

                            Alex

                            Jetson NanoJ 1 Reply Last reply Reply Quote 0
                            • Jetson NanoJ
                              Jetson Nano @Alex Kushleyev
                              last edited by

                              @Alex-Kushleyev Thank you for reverting back, I will steps mentioned by you.

                              Jetson NanoJ 1 Reply Last reply Reply Quote 0
                              • Jetson NanoJ
                                Jetson Nano @Jetson Nano
                                last edited by

                                @Alex-Kushleyev Thank you for sharing the files and the support, I was able to flip the video by using the flip file for camera 1. Now it is working fine.

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post
                                Powered by NodeBB | Contributors