ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    4K Hires Camera crashes camera server

    Ask your questions right here!
    3
    21
    1255
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Alex KushleyevA
      Alex Kushleyev ModalAI Team @Alex Kushleyev
      last edited by

      If you just want 4K video, you should select "large_video_width" 3840 and height 2160, you can disable snapshot and small video if you are not using them.

      R 1 Reply Last reply Reply Quote 0
      • R
        riteshsharma @Alex Kushleyev
        last edited by

        @Alex-Kushleyev I followed your suggestion and selected "large_video_width" 3840 and height 2160. Now hires camera isn't working. But if I go back to 1920x1080, it works
        Capture_error.PNG

        Alex KushleyevA 1 Reply Last reply Reply Quote 0
        • Alex KushleyevA
          Alex Kushleyev ModalAI Team @riteshsharma
          last edited by Alex Kushleyev

          @riteshsharma that is interesting, it looks like it is failing to allocate memory for the video encoder. Can you please disable the small video stream in and disable the snapshot voxl-camera-server.conf and try again?

          R 1 Reply Last reply Reply Quote 0
          • R
            riteshsharma @Alex Kushleyev
            last edited by

            @Alex-Kushleyev Yes, I disabled small video stream before trying this. This is what my voxl-camera-server.conf looks like:

            voxl:/$ cat etc/modalai/voxl-camera-server.conf
            /**

            • voxl-camera-server Configuration File
            • Each camera has configurations for up to 4 HAL3 streams:
              • preview stream for raw unprocessed images from CV cameras
              • small_video 720p (ish) h264/h265 compressed for fpv video streaming
              • large_video 4k (ish) h264/h265 for onboard video recording to disk
              • snapshot ISP-processed JPG snapshots that get saved to disk
            • on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled
            • This file is generated from default values by voxl-configure-cameras.
            • Do not expect arbitrary resolutions to work, the ISP and video compression
            • pipelines only support very specific resolutions.
            • The default video compression mode is cqp or Constant Quantization Parameter

            */
            {
            "version": 0.1,
            "cameras": [{
            "type": "ov7251",
            "name": "tracking",
            "enabled": true,
            "camera_id": 1,
            "fps": 30,
            "en_rotate": false,
            "en_preview": true,
            "preview_width": 640,
            "preview_height": 480,
            "en_raw_preview": true,
            "ae_mode": "lme_msv",
            "ae_desired_msv": 60,
            "exposure_min_us": 20,
            "exposure_max_us": 33000,
            "gain_min": 54,
            "gain_max": 8000,
            "exposure_soft_min_us": 5000,
            "ae_filter_alpha": 0.600000023841858,
            "ae_ignore_fraction": 0.20000000298023224,
            "ae_slope": 0.05000000074505806,
            "ae_exposure_period": 1,
            "ae_gain_period": 1
            }, {
            "type": "imx412",
            "name": "hires",
            "enabled": true,
            "camera_id": 0,
            "fps": 30,
            "en_preview": false,
            "preview_width": 640,
            "preview_height": 480,
            "en_raw_preview": false,
            "en_small_video": false,
            "small_video_width": 1024,
            "small_video_height": 768,
            "small_venc_mode": "h264",
            "small_venc_br_ctrl": "cqp",
            "small_venc_Qfixed": 30,
            "small_venc_Qmin": 15,
            "small_venc_Qmax": 40,
            "small_venc_nPframes": 9,
            "small_venc_mbps": 2,
            "en_large_video": true,
            "large_video_width": 3840,
            "large_video_height": 2160,
            "large_venc_mode": "h264",
            "large_venc_br_ctrl": "cqp",
            "large_venc_Qfixed": 40,
            "large_venc_Qmin": 15,
            "large_venc_Qmax": 50,
            "large_venc_nPframes": 29,
            "large_venc_mbps": 40,
            "en_snapshot": false,
            "en_snapshot_width": 3840,
            "en_snapshot_height": 2160,
            "ae_mode": "isp"
            }]
            }

            Alex KushleyevA 1 Reply Last reply Reply Quote 0
            • Alex KushleyevA
              Alex Kushleyev ModalAI Team @riteshsharma
              last edited by Alex Kushleyev

              @riteshsharma,

              Are you using VOXL1 or VOXL2? it seems VOXL1 based on the terminal output.

              Which SDK do you have installed?

              I am going to assume this is VOXL1, it is possible that it does not have enough memory designated for camera pipeline for all the buffers.

              Are you able to change the number of NUM_RECORD_BUFFERS from 16 to 8 or 10? https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/master/src/hal3_camera_mgr.cpp?ref_type=heads#L86

              I believe we increased the number of buffers at some point and the 4K30 use was missed in testing on VOXL1.

              If you are able to experiment with number of buffers yourself, that would be great, maybe the easiest way to do this is to get the camera server repo on your VOXL1 and compile on target with modified number of buffers. If you cannot compile the camera server in any way, i can help you.

              Alex KushleyevA 1 Reply Last reply Reply Quote 0
              • Alex KushleyevA
                Alex Kushleyev ModalAI Team @Alex Kushleyev
                last edited by

                also in your earlier posts, you showed more prints that print out the buffer info, such as:

                    Count Min: 16
                    Count Actual: 16
                etc
                

                This is the buffer information for the two cameras, you can see if less than 16 were allocated for the hires camera (camera 0)

                Also, i see you are using three ov7251 cameras, which also require some additional "special" memory and perhaps we missed testing that use case with 16 buffers, so reducing number of recording buffers should solve the issue (it looks like it is able to allocate 11 of them, but you can use 8 to be safe)

                R 1 Reply Last reply Reply Quote 0
                • R
                  riteshsharma @Alex Kushleyev
                  last edited by

                  @Alex-Kushleyev Hi Alex, Thank you for your suggestion, I will try to change the buffers to 10. I have two stereo camera ("ov7251"), one tracking camera ("ov7251"), one flir boson camera ("uvc") and one Hires camera ("imx412"). I am trying to operate all cameras on VOXL1 together. I am not sure if 10 buffers will be enough.

                  Alex KushleyevA 1 Reply Last reply Reply Quote 0
                  • Alex KushleyevA
                    Alex Kushleyev ModalAI Team @riteshsharma
                    last edited by

                    @riteshsharma , the number of buffers is per camera 🙂 so 16 is definitely an overkill

                    R 1 Reply Last reply Reply Quote 0
                    • R
                      riteshsharma @Alex Kushleyev
                      last edited by riteshsharma

                      @Alex-Kushleyev Hi Alex, I tried changing buffer to 8. It is still failing to initialize encoder for hires camera.

                      =================================================================
                      Starting Camera: tracking (id #1)
                      Using gain limits min: 100 max: 6399
                      Starting Camera: hires (id #0)
                      Using gain limits min: 54 max: 8000
                      pthread_setaffinity_np: Success
                      WARNING: Port Def 0:
                      Count Min: 11
                      Count Actual: 11
                      Size: 0xbfd000
                      Buffers Contiguous: Yes
                      Buffer Alignment: 0
                      WARNING: Port Def 1:
                      Count Min: 16
                      Count Actual: 16
                      Size: 0xbdd800
                      Buffers Contiguous: No
                      Buffer Alignment: 0
                      WARNING: Encoder expecting(11) more buffers than module allocated(8)
                      ERROR: OMX Set config failed!
                      ERROR: Failed to initialize encoder for camera: hires
                      ERROR: Failed to start camera: hires
                      Starting Camera: stereo (id #2)
                      Using gain limits min: 100 max: 6399

                      with buffers set to 12, it is still failing to start:

                      =================================================================
                      Starting Camera: tracking (id #1)
                      Using gain limits min: 100 max: 6399
                      Starting Camera: hires (id #0)
                      Using gain limits min: 54 max: 8000
                      pthread_setaffinity_np: Success
                      ERROR: Failed to get correct number of buffers from OMX module, expected: 12 got: 11
                      ERROR: OMX SetPortParams of PortIndexIn failed!
                      ERROR: OMX Set config failed!
                      ERROR: Failed to initialize encoder for camera: hires
                      ERROR: Failed to start camera: hires
                      Starting Camera: stereo (id #2)
                      Using gain limits min: 100 max: 6399

                      ------ voxl-camera-server: Started 2 of 3 cameras

                      ------ voxl-camera-server: Camera server is now running
                      pthread_setaffinity_np: Success
                      pthread_setaffinity_np: Success

                      Alex KushleyevA 1 Reply Last reply Reply Quote 0
                      • Alex KushleyevA
                        Alex Kushleyev ModalAI Team @riteshsharma
                        last edited by

                        @riteshsharma

                        Can you try 11?

                        Alex KushleyevA 1 Reply Last reply Reply Quote 0
                        • Alex KushleyevA
                          Alex Kushleyev ModalAI Team @Alex Kushleyev
                          last edited by

                          Also, just for the test, you could try to disable the other cameras and see if the system is able to allocate more buffers for hires camera (12?)

                          I can test your setup sometime soon and provide better recommendations, I just dont have it in front of me right now.

                          R 1 Reply Last reply Reply Quote 0
                          • R
                            riteshsharma @Alex Kushleyev
                            last edited by

                            @Alex-Kushleyev Hi Alex, buffer when set to 11 works. I am able to see all my cameras. But now I am not seeing hires_large_encoded and hires_small_encoded stream on my voxl-portal.

                            Capture.PNG

                            R 1 Reply Last reply Reply Quote 0
                            • R
                              riteshsharma @riteshsharma
                              last edited by riteshsharma

                              @Alex-Kushleyev I have one more question about the length of the cable for connecting Hires (imx412) 4k Camera. I tried using more than one extension and it does not work. Do you have any suggestion for longer wire or extension.

                              IMG_4914 (1).jpg

                              Alex KushleyevA 1 Reply Last reply Reply Quote 0
                              • Alex KushleyevA
                                Alex Kushleyev ModalAI Team @riteshsharma
                                last edited by

                                @riteshsharma , good progress.. i dont think voxl-portal is able to display h264 stream.

                                If you use “voxl-inspect-cam -a” , it should show stream stats. Also you can view the h264 stream using voxl-streamer, at least that is what we do on voxl2.

                                Regarding chaining the extension flex cables, our tests also showed that only one extension works for imx412 due to high data rate. Other cameras may be able to go longer. We dont have a longer solution for imx412

                                R 1 Reply Last reply Reply Quote 0
                                • R
                                  riteshsharma @Alex Kushleyev
                                  last edited by

                                  @Alex-Kushleyev Thanks Alex. You have been great help.

                                  Alex KushleyevA 1 Reply Last reply Reply Quote 0
                                  • Alex KushleyevA
                                    Alex Kushleyev ModalAI Team @riteshsharma
                                    last edited by

                                    @riteshsharma you are welcome. If the current approach initializes and works fine (is able to allocate all the buffers), the performance should be stable. I am a bit surprised so many buffers are needed for the encoder, there is probably some other settings in our software to change that... However since it is actually working with 11 buffers, maybe there is no need for me to test / fix that. If you do encounter any other issues, let me know..

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post
                                    Powered by NodeBB | Contributors