ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Minimizing voxl-camera-server CPU usage in SDK1.6

    Video and Image Sensors
    2
    24
    409
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Rowan DempsterR
      Rowan Dempster @Alex Kushleyev
      last edited by Rowan Dempster

      @Alex-Kushleyev Hi Alex, I compiled libmodal-pipe with ION client-only support in 32 bit builds. I then changed the voxl-mpa-tools toolchain to 32 bit and set the 32 bit toolchain to actually build the tool executables and removed the ones that didn't compile. After deploying those two modified packages to the VOXL2 and running the same repub command that I ran before, I am getting a runtime error while parsing the incoming ION buffers on the client side

      voxl2:/$ voxl-image-repub top_misp_norm_ion -o top_misp_norm_ion_test                                                                                                                                                 
Subscribed to ION input pipe: top_misp_norm_ion                                                                                                                                                                       
invalid metadata, magic number=85, expected 1229934146                                                                                                                                                                
invalid metadata, magic number=85, expected 1229934146                                                                                                                                                                
invalid metadata, magic number=85, expected 1229934146

      Tracking it down, it looks like the call to pipe_client_bytes_in_pipe in the _ion_buf_cb is what's printing the error. If I remove that line (so not returning if pipe_client_bytes_in_pipe returns 0) it prints some garbage (incorrect) metadata and then segfaults:

      voxl2:/$ voxl-image-repub top_misp_norm_ion -o top_misp_norm_ion_test
Subscribed to ION input pipe: top_misp_norm_ion
CH0 Got input ion frame size: 10330x-2843, format: STEREO_NV12, stride: 50834, size: 396 bytes

Segmentation fault:
Fault thread: voxl-image-repu(tid: 7900)
Fault address: 0x4
Address not mapped.
Segmentation fault

      Note that I didn't recompile voxl-camera-server on-top of the modified libmodal-pipe, don't think that would change anything.

      Let me know if you want me to send you my diffs of libmodal-pipe (w.r.t. master) and voxl-mpa-tools (w.r.t. add-new-image-tools).

      P.S. I also tried voxl-inspect-ion-stream top_misp_norm_ion (that tool is also 32 bit now) and it's printing garbage too

      |         Pipe Name |  fd  |  bytes  | wide |  hgt |exp(ms)| gain | frame id |latency(ms)|  fps |  mbps  |t
| top_misp_norm_ion |    6 |     455 | 3237 |-29847 | 52.43 |-24576 |1448040524 |1955547.7  | 30.0 |    0.6
      Alex KushleyevA 1 Reply Last reply Reply Quote 0
      • Alex KushleyevA
        Alex Kushleyev ModalAI Team @Rowan Dempster
        last edited by

        @Rowan-Dempster ,

        I think you are close.. sure, if you want to share the diff or make a fork of the repos, I will try it out.

        Alex

        Rowan DempsterR 1 Reply Last reply Reply Quote 0
        • Rowan DempsterR
          Rowan Dempster @Alex Kushleyev
          last edited by

          @Alex-Kushleyev Here are the diffs:

          • libmodal-pipe: https://gitlab.com/rowan.dempster/libmodal-pipe/-/merge_requests/1/diffs
          • voxl-mpa-tools: https://gitlab.com/rowan.dempster/voxl-mpa-tools/-/merge_requests/1/diffs

          Thanks for your help!

          Alex KushleyevA 1 Reply Last reply Reply Quote 0
          • Alex KushleyevA
            Alex Kushleyev ModalAI Team @Rowan Dempster
            last edited by

            Hi @Rowan-Dempster, thank you. I will take a look today.

            Alex

            Alex KushleyevA 1 Reply Last reply Reply Quote 0
            • Alex KushleyevA
              Alex Kushleyev ModalAI Team @Alex Kushleyev
              last edited by Alex Kushleyev

              Hi @Rowan-Dempster ,

              I made a few changes (based on your branches) to libmodal-pipe and the 32-bit tools and now it seems to be working without any run time errors from the test tools or 64bit camera server:

              https://gitlab.com/voxl-public/voxl-sdk/core-libs/libmodal-pipe/-/tree/ion-32bit-test
              https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/tree/ion-32bit-test

              The issue in libmodal-pipe was with a struct size difference for 32 / 64 bit systems.

              Please note that when you build libmodal pipe, do not deploy the 64bit version to voxl2 as it will break things. Later we will need to fix this up so the struct padding is platform dependent.

              voxl-image-repub is working now . I had to disable the FD caching due to a strange issue (segfault) when inserting into std::map. Did not investigate why that happened. Without duplicating the FD (using the original FD in the callback, everything works). Also the 32-bit version of voxl-inspect-ion-stream works as expected.

              Feel free to test this out. If you don't find any more issues, we can try to mainline this change after cleanup. I am a bit concerned that std::map insert failed, but it is possible this is because i am using the latest voxl-cross to build the app, but my test board is running on old SDK (and there could have been a c++ runtime libs update sometime in the recent voxl-cross).

              Alex

              Rowan DempsterR 1 Reply Last reply Reply Quote 0
              • Rowan DempsterR
                Rowan Dempster @Alex Kushleyev
                last edited by

                @Alex-Kushleyev Hi Alex,

                Success! I was able to run QVIO using the ION pipe and saw the resultant reduction in CPU util both on the camera server process as well as the QVIO process (small but noticeable reductions).

                Note that I was not able to run the camera server with the 32-bit customized libmodal-pipe was installed. So the approach I took was run voxl-camera-server first with the unmodified libmodal-pipe, then install the 32-bit customized libmodal pipe, and run QVIO. Of course this would not be feasible in production. Here are the errors I'm getting when trying to run voxl-camera-server with the 32-bit customized libmodal-pipe installed:

                =================================================================
thread is locked to cores: 4 5 6 7
connected to mavlink pipe
Connected to cpu-monitor
Starting Camera: back (id #0)
gbm_create_device(156): Info: backend name is: msm_drm
MISP Initializing for camera back
 Detected 1 platform(s)
 Detected 1 GPU device(s)
Estimated imu dt = 0.000977s
WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (1200000 bps provided). Using FPS hack. scale = 2.500000
WARNING: Encoder expecting(16) more buffers than module allocated(1)
ERROR:   OMX Set config failed!
ERROR:   Failed to initialize MISP encoder for camera: back
ERROR:   Failed to start camera: back
Starting Camera: top (id #1)
MISP Initializing for camera top
WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (1200000 bps provided). Using FPS hack. scale = 2.500000
WARNING: Encoder expecting(16) more buffers than module allocated(1)
ERROR:   OMX Set config failed!
ERROR:   Failed to initialize MISP encoder for camera: top
ERROR:   Failed to start camera: top
Starting Camera: tof (id #2)
Starting Camera: hires (id #3)
WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (2000000 bps provided). Using FPS hack. scale = 1.500000
WARNING: Encoder expecting(16) more buffers than module allocated(1)
ERROR:   OMX Set config failed!
ERROR:   Failed to initialize encoder for camera: hires
ERROR:   Failed to start camera: hires
Starting Camera: bottom (id #4)
MISP Initializing for camera bottom
WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (1200000 bps provided). Using FPS hack. scale = 2.500000
WARNING: Encoder expecting(16) more buffers than module allocated(1)
ERROR:   OMX Set config failed!
ERROR:   Failed to initialize MISP encoder for camera: bottom
ERROR:   Failed to start camera: bottom

                What are the next steps towards mainlining this in a way that works "out of the box"? Anything else I should test or any way I can support that process?

                Thank you,
                Rowan

                Alex KushleyevA 1 Reply Last reply Reply Quote 0
                • Alex KushleyevA
                  Alex Kushleyev ModalAI Team @Rowan Dempster
                  last edited by

                  @Rowan-Dempster , nice! I am glad it worked.

                  i think the issue here is that we have changed the struct definition (of the ion buffer type), which should not be changed for the 64-bit library. So until we hopefully mainline this, you can just manually copy the 32-bit libmodal-pipe.so into /usr/lib/ (while the original 64-bit version will remain in /usr/lib64). Did you try that?

                  For mainlining this, i think we will need to do a #ifdef 32 bit and insert the allignment dummy bytes as well as bypassing the ion allocation functions, which require libgbm.so.. You could try that change as well, just need to figure out how to detect 64 vs 32 bit in the few source files that we changed.

                  Alex

                  Rowan DempsterR 1 Reply Last reply Reply Quote 0
                  • Rowan DempsterR
                    Rowan Dempster @Alex Kushleyev
                    last edited by

                    @Alex-Kushleyev Okay sounds like we're almost there in terms of readiness for mainline then. Would you like me to take that on and put up a gitlab MR from my fork into modalai's libmodal-pipe repo? I can schedule an hour or two to polish that up and get it ready sometime in the next few days.

                    Alex KushleyevA 1 Reply Last reply Reply Quote 0
                    • Alex KushleyevA
                      Alex Kushleyev ModalAI Team @Rowan Dempster
                      last edited by

                      @Rowan-Dempster , sure, that would be helpful. I don't see any issues with merging that in, since if we do it right, it will not affect 64 bit build at all and will only add limited support for 32 bit -- limited, meaning, for clients only, cannot allocate new buffers.

                      i will check with the team today to make sure we are all good, so perhaps you should wait for that.

                      Alex

                      Rowan DempsterR 1 Reply Last reply Reply Quote 0
                      • Rowan DempsterR
                        Rowan Dempster @Alex Kushleyev
                        last edited by

                        @Alex-Kushleyev

                        sure, that would be helpful.

                        i will check with the team today to make sure we are all good, so perhaps you should wait for that.

                        Will do!

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Powered by NodeBB | Contributors