ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login
    1. Home
    2. Rowan Dempster
    3. Posts
    • Profile
    • Following 0
    • Followers 0
    • Topics 32
    • Posts 105
    • Best 0
    • Controversial 0
    • Groups 0

    Posts made by Rowan Dempster

    • RE: Running QVIO on a hires camera

      Hi @Alex-Kushleyev,

      Resurrecting this old thread, we now have the IMX412 on a drone and we are now ready to give to VIO on the IMX412 our full attention and lots of testing effort. Where I'm at now is I have a prototype working and QVIO does run on the IMX412 camera and outputs estimates that seem reasonable, but I'm 100% sure it's not configured as good as it could be cause I made so many assumptions that I would like your input on:

      Which camera data / pipe to use
      Ideally, we would like the IMX412 VIO to perform close to (or of course better than!) a ar0144 tracking camera, in terms of the quality of the image for feature tracking, low CPU usage, low latency frames, etc etc etc. In this spirit, I've been looking into how to get the MISP normalized pipes coming from the IMX412 and also how to get the camera server producing ION data to get the same CPU usage gains we saw in https://forum.modalai.com/topic/4893/minimizing-voxl-camera-server-cpu-usage-in-sdk1-6.
      I saw that in the pipe setup, the normalized code for IMX412 was commented out
      4f2e7cff-3212-46c7-9929-988f0ce413e1-image.png
      After commenting it back in I was able to see in the portal a decent looking normalized stream. I also see the ION pipe pop up for that norm stream but I haven't tried that ION pipe on the QVIO server yet (I'm confident it would work though, just waiting for https://gitlab.com/voxl-public/voxl-sdk/core-libs/libmodal-pipe/-/commit/d18521776e3e88f396d85aa657769c47f29e9c9f to get tagged!).
      Do you see any issue with using the MISP norm pipe for IMX412 VIO, or is that actually what you would recommend?

      Which resolution to use
      I know you talked about some resolution advice above, but I'm a little bit confused on the specifics on where to put those numbers. You had suggested 1996x1520 for a 5.5ms readout time. Do these numbers go into the Preview Width config fields? Here is the entire diff of the config settings I have been using for my testing:
      b75275d6-9c35-48ac-a95c-a59b439e7f54-image.png
      The other values I have a question about in that diff is The MISP width / height fields, I chose 998x760 which is half of the Preview Width resolution you suggested. I did this because I wasn't sure of any compute bottle necks that would pop up if I fed a 1996x1520 image into QVIO. Do you think 998x760 is good or maybe I should pick a like 0.75 downsample so something like 1497x1140 for the MISP width/height.

      Camera Driver files to use and how to version control and deploy those
      Could you confirm that the binary files in https://storage.googleapis.com/modalai_public/temp/imx412_test_bins/20250919/imx412_fpv_eis_20250919_drivers.zip are still the latest and the recommended binaries to use? Could you also advise on how to version control these files and deploy them to the voxl2 when the camera server .deb is deployed? I want to keep all files related to bringing up the camera in the voxl-camera-server debian if possible. I see some binaries files being stored in this path:
      03191a03-455e-4e18-bd50-dd19b9d5c028-image.png

      So if I understand the process correctly, those files will end up in /usr/share/modalai/chi-cdk/imx412-fps-misp. Is that where the voxl-configure-cameras C looks for them? Also, do I have to do anything with the com.qti.sensor.imx412_fpv.so file in the zip link that you sent, or do I just ignore that file?

      My end desired behavior is that when I install the voxl camera server .deb, I don't have to worry about also copying binary files over to the voxl, or moving any files around on the voxl, or having to remember to run voxl-configure-cameras C. So maybe the path forward there is to have all files deployed into the right places by the .deb install and then in the postinst script auto run voxl-configure-cameras C? What do you think?

      Aspect ratio concerns and their affect on field of view and camera calibration
      Is the aspect ratio you suggested (1996x1520) the actual aspect ratio of the sensor? Or does the sensor support multiple aspect ratios, or is there something more complex going on I don't understand here? I just want to make sure that we're using as much FoV as the sensor supports. That should be the goal for VIO feature detection and for streaming right, maximize field of view?

      Also, how does changing the aspect ratio / resolution affect the camera calibration? We're using kalibr which asks for a focal length bootstrap, which we've been giving it 470 for the ar0144 camera, do you know of a way that we can figure out an accurate focal length bootstrap for the images that we end up using for the IMX412?

      How to not lose other IMX412 features like 4k recording and EIS streaming etc
      This is kinda my biggest concern about the feasibility of this whole thing: Will we still be able to get the other awesome IMX412 features like high quality streaming to the GCS with EIS, as well as high quality 4K recordings even in difficult low light environments, AND at the same time optimize the IMX412 for VIO which demands stuff like fast readout for less skew?

      Any advice you can give here on mapping out the tradeoffs? Are there any non-starters like not being able to get 4K recording if we opt to use the MISP norm pipe for VIO? Or are you confident that we can get the best of all 3 worlds 🙂

      Exposure time concerns
      I agree with your initial point that needing low exposure for low motion blur is important. As I mentioned in the intro to this message, I have a prototype working and I am now at a place where I can indeed tune the gain vs exposure / auto exposure params. Could you help me with that? I assume that this tradeoff also applies to the ar0144 camera, any lessons I can take from there?

      QVIO readoutTime param
      Great find and thanks for pointing that out!! To confirm the specific numbers here, if I use the 1996x1520 preview width/height which has a documented read out time of 5.5ms (I should confirm this using the -d mode), then I should put 0.0055 for that parameter?

      As always, thank you for your help in camera related matters, we would be no where close to where we are now with robotic perception without your guidance!

      posted in GPS-denied Navigation (VIO)
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      Hi @Alex-Kushleyev ,

      Has there been a tag created in https://gitlab.com/voxl-public/voxl-sdk/core-libs/libmodal-pipe that contains these changes? Please let me know when that happens, would would like to get these CPU gains moved into production ASAP!

      All the best,
      Rowan

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      Hi @Alex-Kushleyev,

      I confirmed that the https://gitlab.com/voxl-public/voxl-sdk/core-libs/libmodal-pipe/-/tree/ion-32bit-test branch does indeed allow voxl-qvio-server to use ION pipes (using DEN_ION_BUF and DEN_32BIT_ION_BUF). I have not detected any other breakages of the Cleo SDK or performance degradations, but that's something we will monitor as we test more.

      In CPU perf mode, average core util dropped from 31% to 22%, this is a huge help, thank you very much for your work.

      We will be eagerly awaiting the release of a libmodal-pipe gitlab tag with these changes in it!

      Cheers,
      Rowan

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      Hi @Alex-Kushleyev,

      I did a quick test. Can you please test to make sure the updated libmodal pipe works with your 32 bit application and also does not break the main 64-bit functionality? Then i think we can merge it to dev.

      Yes absolutely, I can get that done today.

      Thanks for your patience!

      Thank you for taking this on!

      Rowan

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Hi Alex, Happy New Years 🙂

      Is there an ETA on when the 32-bit ION buffer support in libmodal-pipe will be main-lined? For our internal time planning here at Cleo.

      Thank you,
      Rowan

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Toolchain for m0054-data-fs.ext4

      @Moderator Hi Modal,

      Does this mean you can create a custom data file partition that you can flash using fastboot?

      Yes that is correct.

      We have not explored this before but it has been asked a few times. This could be very helpful for other developers.

      I'm certainly hopeful that it will be helpful here at Cleo Robotics! So far with my prototyping it works as expected and cuts flashing time of some large docker images we have here at Cleo down by a noticeable fraction (no file overhead via fastboot like with ADB).

      I think the snippet I posted covers the baseline functionality of getting a custom "payload" into the data partition. However if there is more I can elaborate on in terms of the toolchain / what's in the payload, and if that elaboration will be helpful to other VOXL2 developers, I would be happy to elaborate 🙂 Just let me know!

      Other similar discussion points I tackled recently that I'm happy to talk about lessons of:

      • Flashing the system image and VOXL/CLEO SDK through a Windows Machine (journeys in USB device drivers)
      • Building Flutter applications for uniform flashing process across all operating systems
      • Building release bundles (i.e. a collection of partition binaries) in CI
      posted in VOXL SDK
      Rowan DempsterR
      Rowan Dempster
    • RE: Toolchain for m0054-data-fs.ext4

      For anyone reading this in the future this works:

      docker run --rm --privileged \
          -v "$SYSTEM_IMAGE_DIR":/system-image:ro \
          -v "$CLEO_IMAGE_DIR":/cleo-image \
          -v "$CLEO_PROVISION_DIR":/cleo-provision:ro \
          ubuntu:18.04 bash -c "
          set -e
          
          # Install tools
          apt-get update -qq
          apt-get install -y -qq android-tools-fsutils e2fsprogs >/dev/null
          
          # Extract stock data partition contents
          echo 'Extracting stock m0054-data-fs.ext4...'
          mkdir -p /tmp/data_root
          
          # Convert sparse to raw and mount
          simg2img /system-image/m0054-data-fs.ext4 /tmp/stock_raw.ext4
          mkdir -p /mnt/stock
          mount -o loop,ro /tmp/stock_raw.ext4 /mnt/stock
          
          # Copy all stock files
          cp -a /mnt/stock/* /tmp/data_root/
          umount /mnt/stock
          rm /tmp/stock_raw.ext4
          
          echo 'Stock contents copied.'
          
          # Add cleo-provision directory
          cp -a /cleo-provision /tmp/data_root/cleo-provision
          
          echo 'Added cleo-provision directory'
          
          # Create the ext4 image using same parameters as BitBake recipe:
          #   -s            : sparse output
          #   -l SIZE       : filesystem size in bytes
          #   -a /data      : android mount point
          #   -b 4096       : block size
          make_ext4fs -s -l $USERDATA_SIZE_EXT4 -a /data -b 4096 /cleo-image/m0054-data-fs.ext4 /tmp/data_root
          
          echo ''
          echo 'Created sparse ext4 image successfully'
      "
      

      You'll need to be on an arm64 architecture.

      posted in VOXL SDK
      Rowan DempsterR
      Rowan Dempster
    • RE: Toolchain for m0054-data-fs.ext4

      I found https://gitlab.com/voxl-public/system-image-build/meta-voxl2/-/blob/qrb5165-ubun1.0-14.1a/recipes-products/image/qti-ubuntu-robotics-image.bbappend#L148, which is in bitbake language (not familiar)?

      Is

      do_makeuserdata() {
          make_ext4fs -s -l ${USERDATA_SIZE_EXT4} ${IMAGE_EXT4_SELINUX_OPTIONS} \
              -a /data -b 4096 ${DEPLOY_DIR_IMAGE}/${OVERLAYIMAGE_TARGET} ${IMAGE_ROOTFS}/data
      }
      

      something I could do just on normal ubuntu in CI? I'm thinking to build a cleo-specific userdata in CI with all of our custom software.

      What are ${IMAGE_EXT4_SELINUX_OPTIONS}, is there some secret sauce there?

      posted in VOXL SDK
      Rowan DempsterR
      Rowan Dempster
    • Toolchain for m0054-data-fs.ext4

      Hi Modal,

      I'm looking into adding some data in the m0054-data-fs.ext file system. I'm having some trouble getting the voxl2 to work correctly when I try to add the data and then create a new Android sparse image that I flash instead of m0054-data-fs.ext. Would it be possible to provide me with the tools used to create that m0054-data-fs.ext file system at ModalAI so that I can use the exact same tools to create the sparse android image after adding my files to it?

      Thank you,
      Rowan

      posted in VOXL SDK
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Sounds good! I did not make any changes since the revisions you shared 5 days ago.

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev

      sure, that would be helpful.

      i will check with the team today to make sure we are all good, so perhaps you should wait for that.

      Will do!

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Okay sounds like we're almost there in terms of readiness for mainline then. Would you like me to take that on and put up a gitlab MR from my fork into modalai's libmodal-pipe repo? I can schedule an hour or two to polish that up and get it ready sometime in the next few days.

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Hi Alex,

      Success! I was able to run QVIO using the ION pipe and saw the resultant reduction in CPU util both on the camera server process as well as the QVIO process (small but noticeable reductions).

      Note that I was not able to run the camera server with the 32-bit customized libmodal-pipe was installed. So the approach I took was run voxl-camera-server first with the unmodified libmodal-pipe, then install the 32-bit customized libmodal pipe, and run QVIO. Of course this would not be feasible in production. Here are the errors I'm getting when trying to run voxl-camera-server with the 32-bit customized libmodal-pipe installed:

      =================================================================
      thread is locked to cores: 4 5 6 7
      connected to mavlink pipe
      Connected to cpu-monitor
      Starting Camera: back (id #0)
      gbm_create_device(156): Info: backend name is: msm_drm
      MISP Initializing for camera back
       Detected 1 platform(s)
       Detected 1 GPU device(s)
      Estimated imu dt = 0.000977s
      WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (1200000 bps provided). Using FPS hack. scale = 2.500000
      WARNING: Encoder expecting(16) more buffers than module allocated(1)
      ERROR:   OMX Set config failed!
      ERROR:   Failed to initialize MISP encoder for camera: back
      ERROR:   Failed to start camera: back
      Starting Camera: top (id #1)
      MISP Initializing for camera top
      WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (1200000 bps provided). Using FPS hack. scale = 2.500000
      WARNING: Encoder expecting(16) more buffers than module allocated(1)
      ERROR:   OMX Set config failed!
      ERROR:   Failed to initialize MISP encoder for camera: top
      ERROR:   Failed to start camera: top
      Starting Camera: tof (id #2)
      Starting Camera: hires (id #3)
      WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (2000000 bps provided). Using FPS hack. scale = 1.500000
      WARNING: Encoder expecting(16) more buffers than module allocated(1)
      ERROR:   OMX Set config failed!
      ERROR:   Failed to initialize encoder for camera: hires
      ERROR:   Failed to start camera: hires
      Starting Camera: bottom (id #4)
      MISP Initializing for camera bottom
      WARNING: OMX SetTargetBitrate: H265 CBR requires bps >= 3.0Mbit (1200000 bps provided). Using FPS hack. scale = 2.500000
      WARNING: Encoder expecting(16) more buffers than module allocated(1)
      ERROR:   OMX Set config failed!
      ERROR:   Failed to initialize MISP encoder for camera: bottom
      ERROR:   Failed to start camera: bottom
      

      What are the next steps towards mainlining this in a way that works "out of the box"? Anything else I should test or any way I can support that process?

      Thank you,
      Rowan

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Here are the diffs:

      • libmodal-pipe: https://gitlab.com/rowan.dempster/libmodal-pipe/-/merge_requests/1/diffs
      • voxl-mpa-tools: https://gitlab.com/rowan.dempster/voxl-mpa-tools/-/merge_requests/1/diffs

      Thanks for your help!

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Hi Alex, I compiled libmodal-pipe with ION client-only support in 32 bit builds. I then changed the voxl-mpa-tools toolchain to 32 bit and set the 32 bit toolchain to actually build the tool executables and removed the ones that didn't compile. After deploying those two modified packages to the VOXL2 and running the same repub command that I ran before, I am getting a runtime error while parsing the incoming ION buffers on the client side

      voxl2:/$ voxl-image-repub top_misp_norm_ion -o top_misp_norm_ion_test                                                                                                                                                 
      Subscribed to ION input pipe: top_misp_norm_ion                                                                                                                                                                       
      invalid metadata, magic number=85, expected 1229934146                                                                                                                                                                
      invalid metadata, magic number=85, expected 1229934146                                                                                                                                                                
      invalid metadata, magic number=85, expected 1229934146
      

      Tracking it down, it looks like the call to pipe_client_bytes_in_pipe in the _ion_buf_cb is what's printing the error. If I remove that line (so not returning if pipe_client_bytes_in_pipe returns 0) it prints some garbage (incorrect) metadata and then segfaults:

      voxl2:/$ voxl-image-repub top_misp_norm_ion -o top_misp_norm_ion_test
      Subscribed to ION input pipe: top_misp_norm_ion
      CH0 Got input ion frame size: 10330x-2843, format: STEREO_NV12, stride: 50834, size: 396 bytes
      
      Segmentation fault:
      Fault thread: voxl-image-repu(tid: 7900)
      Fault address: 0x4
      Address not mapped.
      Segmentation fault
      

      Note that I didn't recompile voxl-camera-server on-top of the modified libmodal-pipe, don't think that would change anything.

      Let me know if you want me to send you my diffs of libmodal-pipe (w.r.t. master) and voxl-mpa-tools (w.r.t. add-new-image-tools).

      P.S. I also tried voxl-inspect-ion-stream top_misp_norm_ion (that tool is also 32 bit now) and it's printing garbage too

      |         Pipe Name |  fd  |  bytes  | wide |  hgt |exp(ms)| gain | frame id |latency(ms)|  fps |  mbps  |t
      | top_misp_norm_ion |    6 |     455 | 3237 |-29847 | 52.43 |-24576 |1448040524 |1955547.7  | 30.0 |    0.6
      
      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Sounds good, I proceeded with resolving the basic typing issues related to 32 bit monotonic time. Next issue is a linker error that looks a bit more tricky:

      -- Build files have been written to: /home/root/build32
      [ 10%] Building CXX object library/CMakeFiles/modal_pipe.dir/src/buffers.cpp.o
      [ 40%] Building C object library/CMakeFiles/modal_pipe.dir/src/server.c.o
      [ 40%] Building C object library/CMakeFiles/modal_pipe.dir/src/common.c.o
      [ 40%] Building C object library/CMakeFiles/modal_pipe.dir/src/misc.c.o
      [ 50%] Building CXX object library/CMakeFiles/modal_pipe.dir/src/buffers/gbm.cpp.o
      [ 60%] Building C object library/CMakeFiles/modal_pipe.dir/src/interfaces.c.o
      [ 70%] Building C object library/CMakeFiles/modal_pipe.dir/src/client.c.o
      [ 80%] Building C object library/CMakeFiles/modal_pipe.dir/src/sink.c.o
      [ 90%] Building C object library/CMakeFiles/modal_pipe.dir/src/start_stop.c.o
      [100%] Linking CXX shared library libmodal_pipe.so
      /opt/sysroots/qrb5165_1/usr/lib/libgbm.so: file not recognized: file format not recognized
      collect2: error: ld returned 1 exit status
      make[2]: *** [library/CMakeFiles/modal_pipe.dir/build.make:228: library/libmodal_pipe.so] Error 1
      make[1]: *** [CMakeFiles/Makefile2:106: library/CMakeFiles/modal_pipe.dir/all] Error 2
      make: *** [Makefile:136: all] Error 2
      
      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Hi Alex I'm running into compilation errors when building libmodal-pipe after turning the EN_ION_BUF flag ON (builds fine with the flag OFF)

      Found voxl-cross version: 4.4
      -- ---------------------------------------------------------
      -- Using voxl-cross 32-bit toolchain for QRB5165
      -- C Compiler     : /usr/bin/arm-linux-gnueabi-gcc-7
      -- C++ Compiler   : /usr/bin/arm-linux-gnueabi-g++-7
      -- Sysroot        : /opt/sysroots/qrb5165_1
      -- C flags        : -isystem=/usr/arm-linux-gnueabi/include -isystem=/usr/include -idirafter /usr/include -fno-stack-protector -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv4
      -- CXX flags      : -isystem=/usr/arm-linux-gnueabi/include -isystem=/usr/include -idirafter /usr/include -fno-stack-protector -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv4
      -- EXE Link Flags : -L/opt/sysroots/qrb5165_1/usr/lib32 -L/opt/sysroots/qrb5165_1/lib -L/opt/sysroots/qrb5165_1/usr/lib -L/opt/sysroots/qrb5165_1/usr/arm-linux-gnueabi -L/opt/sysroots/qrb5165_1/usr/lib/gcc-cross/arm-linux-gnueabi/7 -L/usr/lib
      -- SO Link Flags  : -L/opt/sysroots/qrb5165_1/usr/lib32 -L/opt/sysroots/qrb5165_1/lib -L/opt/sysroots/qrb5165_1/usr/lib -L/opt/sysroots/qrb5165_1/usr/arm-linux-gnueabi -L/opt/sysroots/qrb5165_1/usr/lib/gcc-cross/arm-linux-gnueabi/7 /opt/sysroots/qrb5165_1/usr/lib/gcc-cross/arm-linux-gnueabi/7/libgcc.a /opt/sysroots/qrb5165_1/usr/lib/gcc-cross/arm-linux-gnueabi/7/libgcc_eh.a /opt/sysroots/qrb5165_1/usr/lib/gcc-cross/arm-linux-gnueabi/7/libssp_nonshared.a /opt/sysroots/qrb5165_1/usr/arm-linux-gnueabi/lib/libc_nonshared.a -L/usr/lib
      -- The C compiler identification is GNU 7.3.0
      -- The CXX compiler identification is GNU 7.3.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/arm-linux-gnueabi-gcc-7 - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/arm-linux-gnueabi-g++-7 - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- INFO: building with ion buf support
      -- INFO: building with mavlink support
      INFO: Skipping Building Python Bindings
      -- INFO: Skipping examples and tools
      -- Configuring done (1.8s)
      -- Generating done (0.0s)
      -- Build files have been written to: /home/root/build32
      [ 10%] Building C object library/CMakeFiles/modal_pipe.dir/src/start_stop.c.o
      [ 30%] Building C object library/CMakeFiles/modal_pipe.dir/src/client.c.o
      [ 30%] Building CXX object library/CMakeFiles/modal_pipe.dir/src/buffers.cpp.o
      [ 50%] Building C object library/CMakeFiles/modal_pipe.dir/src/interfaces.c.o
      [ 50%] Building C object library/CMakeFiles/modal_pipe.dir/src/sink.c.o
      [ 60%] Building C object library/CMakeFiles/modal_pipe.dir/src/common.c.o
      [ 70%] Building C object library/CMakeFiles/modal_pipe.dir/src/misc.c.o
      [ 80%] Building C object library/CMakeFiles/modal_pipe.dir/src/server.c.o
      [ 90%] Building CXX object library/CMakeFiles/modal_pipe.dir/src/buffers/gbm.cpp.o
      /home/root/library/src/server.c: In function 'pipe_server_write_ion_buffer':
      /home/root/library/src/server.c:1757:102: error: format '%ld' expects argument of type 'long int', but argument 5 has type 'int64_t {aka long long int}' [-Werror=format=]
               printf("server preparing to send ion buffer id %d (fd %d) to clients, n_clients: %d, time: %ld\n",
                                                                                                          ~~^
                                                                                                          %lld
                      ion_buf->buffer_id, fd, c[ch].n_clients, _time_monotonic_ns());
                                                               ~~~~~~~~~~~~~~~~~~~~                          
      /home/root/library/src/client.c: In function '_stop_helper_and_remove_pipe':
      /home/root/library/src/client.c:1353:56: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64_t {aka long long int}' [-Werror=format=]
               if(en_debug) printf("ch: %d, shutdown socket %ld ns\n", ch, _time_monotonic_ns());
                                                            ~~^            ~~~~~~~~~~~~~~~~~~~~
                                                            %lld
      /home/root/library/src/client.c: In function 'pipe_client_close':
      /home/root/library/src/client.c:1475:43: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64_t {aka long long int}' [-Werror=format=]
               printf("ch: %d, shutdown socket %ld ms\n", ch, _time_monotonic_ns());
                                               ~~^            ~~~~~~~~~~~~~~~~~~~~
                                               %lld
      cc1: all warnings being treated as errors
      make[2]: *** [library/CMakeFiles/modal_pipe.dir/build.make:93: library/CMakeFiles/modal_pipe.dir/src/client.c.o] Error 1
      make[2]: *** Waiting for unfinished jobs....
      cc1: all warnings being treated as errors
      make[2]: *** [library/CMakeFiles/modal_pipe.dir/build.make:149: library/CMakeFiles/modal_pipe.dir/src/server.c.o] Error 1
      make[1]: *** [CMakeFiles/Makefile2:106: library/CMakeFiles/modal_pipe.dir/all] Error 2
      make: *** [Makefile:136: all] Error 2
      

      Seems like I'm going down an uncharted path here. I'm okay with spending my time charting it out since getting QVIO to work with ION pipes will save us critical CPU util. Does ModalAI have plans to switch the official SDK QVIO server release (for qrb5165) to using ION pipes (I saw ModalAI switched to using the MISP pipes for QVIO a few months ago, taking a CPU hit)?

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Actually had some time today to do the profiling and explore the ION pipes.

      I see the same improvements that you quoted:

      On 2.2.17
      Normal pipes: 45%
      MISP pipes: 66%
      MISP pipes + 4x tracking pipes: 160%

      On perf improvements
      Normal pipes: 45%
      MISP pipes: 52%
      MISP pipes + 4x tracking pipes: 71%

      On perf improvements + ION pipes
      MISP pipes: 48%
      MISP pipes + 4x tracking pipes: 52%

      I started looking into using the ION pipe in the QVIO server code but saw this in the voxl-mpa-tools build script:

      	qrb5165)
      		check_docker "4.3"
      		mkdir -p build32
      		cd build32
      		cmake -DCMAKE_TOOLCHAIN_FILE=${TOOLCHAIN_QRB5165_1_32} \
      				-DBUILD_TOOLS=OFF \
      				-DEN_ION_BUF=OFF ../
      		make -j$(nproc)
      		cd ../
      		mkdir -p build64
      		cd build64
      		cmake -DCMAKE_TOOLCHAIN_FILE=${TOOLCHAIN_QRB5165_1_64} \
      				-DBUILD_TOOLS=ON \
      				-DEN_ION_BUF=ON ../
      		make -j$(nproc)
      		cd ../
      		;;
      
      	qrb5165-2)
      		check_docker "4.3"
      		mkdir -p build
      		cd build
      		cmake -DCMAKE_TOOLCHAIN_FILE=${TOOLCHAIN_QRB5165_2_64} \
      				-DBUILD_TOOLS=ON \
      				-DEN_ION_BUF=OFF ../
      		make -j$(nproc)
      		cd ../
      		;;
      

      Since we're using the qrb5165 (not the qrb5165-2) we have to use the 32bit toolchain to build QVIO (right?):

      	qrb5165)
      		check_docker "4.4"
      		mkdir -p build
      		cd build
      		cmake -DCMAKE_TOOLCHAIN_FILE=${TOOLCHAIN_QRB5165_1_32} ../
      		make -j$(nproc)
      		cd ../
      		;;
      

      Does this mean that on the qrb5165 (using the 32 bit toolchain), the QVIO server cannot be a client to the ION pipes?

      Thank you,
      Rowan

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Minimizing voxl-camera-server CPU usage in SDK1.6

      @Alex-Kushleyev Hi Alex, just acknowledging your messages, thanks for looking into this and providing us with a path forward. I will have time over the next 2-3 days to test the perf-optimizations branch and familiarize myself the ION based clients. I will update you then.

      posted in Video and Image Sensors
      Rowan DempsterR
      Rowan Dempster
    • RE: Migrating from QVIO to OpenVINS (SDK1.6)

      Hi Modal,

      Just following up here, I am going to put our migration to OpenVINS onto the backlog for the time being. Please let me know when the SDK is ready for QVIO -> OpenVINS migration, and when there is bandwidth to assist us with the migration. You can reach me at rowan.dempster@cleorobotics.com.

      posted in GPS-denied Navigation (VIO)
      Rowan DempsterR
      Rowan Dempster