ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Voxl2 Docker (Ubuntu 22) with OpenCL/Adreno

    Ask your questions right here!
    3
    16
    1480
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Eric KatzfeyE
      Eric Katzfey ModalAI Team @eric
      last edited by

      @eric Yes, I did put that together based on some Qualcomm example code for VOXL. Not really sure how to do something similar on VOXL 2 but I'll ask around the office to see if anyone has some ideas on how to get that going.

      E 1 Reply Last reply Reply Quote 1
      • E
        eric @Eric Katzfey
        last edited by

        @Eric-Katzfey Really appreciate it! Thanks so much

        Alex KushleyevA 1 Reply Last reply Reply Quote 0
        • Alex KushleyevA
          Alex Kushleyev ModalAI Team @eric
          last edited by

          @eric I will try it out, please give me a few days.

          Alex

          Alex KushleyevA 1 Reply Last reply Reply Quote 1
          • Alex KushleyevA
            Alex Kushleyev ModalAI Team @Alex Kushleyev
            last edited by

            Quick update, I did some testing and tried searching documentation and could not get it to work. There are posts online asking Qualcomm whether this is possible, but there is no response there.

            I know this would be a useful feature and I will try again next week. I want to see how this worked on VOXL1 and perhaps I am missing something.

            In my test, I am also just doing a simple device query and it works on the host VOXL2 but not inside docker, tried mapping various .so libraries and devices to the docker container and no luck yet.

            Alex

            E 1 Reply Last reply Reply Quote 0
            • E
              eric @Alex Kushleyev
              last edited by

              @Alex-Kushleyev That aligns pretty well with my own experience so far. Thanks again for looking into this!

              Alex KushleyevA 1 Reply Last reply Reply Quote 0
              • Alex KushleyevA
                Alex Kushleyev ModalAI Team @eric
                last edited by

                @eric I was able to get the GPU device query inside ubuntu 22.04 docker working using the following steps. It is possible that we can reduce the number of mapped devices and libraries to the docker container, but i am just going to give you this information right now so you can test. I will try to clean this up a bit later. I only tried the device query for now, but i figured i would let you know that there is progress..

                #run docker
                docker run -it --rm --privileged --device=/dev/kgsl-3d0 --device=/dev/ion -v /proc:/proc -v /firmware/image:/firmware/image -v /lib/firmware:/lib/firmware -v /sys/class:/sys/class -v /sys/bus:/sys/bus -v /sys/devices:/sys/devices -v /data:/data -v /usr/lib/liblog.so.0:/usr/lib/liblog.so.0 -v /usr/lib/libOpenCL.so:/usr/lib/libOpenCL.so -v /usr/lib/libcutils.so.0:/usr/lib/libcutils.so.0 -v /usr/lib/libllvm-qcom.so:/usr/lib/libllvm-qcom.so -v /usr/lib/libion.so.0.0.0:/usr/lib/libion.so.0.0.0 -v /usr/lib/libsync.so.0.0.0:/usr/lib/libsync.so.0.0.0 -v /usr/lib/libgsl.so:/usr/lib/libgsl.so -v /usr/lib/libCB.so:/usr/lib/libCB.so -v /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0.5600.4:/usr/lib/aarch64-linux-gnu/libglib-2.0.so.0.5600.4 -v `pwd`:/opt/code -w /opt/code arm64v8/ubuntu:22.04 bash
                
                apt-get update
                apt install --no-install-recommends -y pocl-opencl-icd
                

                then run your test app to query the device..

                E 1 Reply Last reply Reply Quote 1
                • E
                  eric @Alex Kushleyev
                  last edited by

                  @Alex-Kushleyev 🙇

                  Alex KushleyevA 1 Reply Last reply Reply Quote 0
                  • Alex KushleyevA
                    Alex Kushleyev ModalAI Team @eric
                    last edited by Alex Kushleyev

                    OK, a little more clean-up, it seems this is the minimal set of libraries /devices needed:

                    docker run -it --rm --privileged \
                    	-v /usr/lib/libOpenCL.so:/usr/lib/libOpenCL.so \
                    	-v /usr/lib/libCB.so:/usr/lib/libCB.so \
                    	-v /usr/lib/libgsl.so:/usr/lib/libgsl.so \
                    	-v /usr/lib/liblog.so.0:/usr/lib/liblog.so.0 \
                    	-v /usr/lib/libcutils.so.0:/usr/lib/libcutils.so.0 \
                    	-v /usr/lib/libsync.so.0.0.0:/usr/lib/libsync.so.0.0.0 \
                    	-v /usr/lib/libion.so.0.0.0:/usr/lib/libion.so.0.0.0 \
                    	-v /usr/lib/libllvm-qcom.so:/usr/lib/libllvm-qcom.so \
                    	-v /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0.5600.4:/usr/lib/aarch64-linux-gnu/libglib-2.0.so.0.5600.4 \
                    	-v `pwd`:/opt/code -w /opt/code \
                    	arm64v8/ubuntu:22.04 bash
                    

                    (--privileged mode maps all the needed devices to the docker container)

                    Then install some more packages (not sure if this can be reduced, not clear exactly what is missing):

                    apt-get update
                    apt install --no-install-recommends -y pocl-opencl-icd
                    

                    Maybe we can figure out what lib is still missing so that pocl-opencl-icd does not have to be installed.. At least the issue was the a missing library, not a mapped device

                    For testing, I used a device query script from here:

                    root@733a6d4d5fdb:/opt/code# ./simple_query 
                    1. Device: QUALCOMM Adreno(TM)
                     1.1 Hardware version: OpenCL 2.0 Adreno(TM) 650
                     1.2 Software version: OpenCL 2.0 QUALCOMM build: commit # changeid # Date: 11/10/21 Wed Local Branch:  Remote Branch:  Compiler E031.37.12.01
                     1.3 OpenCL C version: OpenCL C 2.0 Adreno(TM) 650
                     1.4 Parallel compute units: 3
                    

                    I also verified that a simple matrix multiplication app also worked (not provided here)

                    @eric , can you please let me know if this works for you?

                    Alex

                    E 1 Reply Last reply Reply Quote 0
                    • E
                      eric @Alex Kushleyev
                      last edited by

                      @Alex-Kushleyev

                      OMG IT WORKS!!

                      I was able to extract all these libraries from the host and directly install them inside the docker, and now the pcol-opencl-icd installation isn't needed.

                      This is really important for us, since it allows us to build external dependencies that rely on OpenCL in our pipeline directly without bind mounts (outside the host environment).

                      Really, really appreciate all your help!

                      FROM arm64v8/ubuntu:22.04
                      
                      # Install necessary dependencies
                      RUN apt-get update && \
                          apt-get install -y \
                          cmake \
                          build-essential \
                          libglib2.0-0
                      
                      # Copy Adreno GPU dependencies
                      # - libcutils0_0-r1_arm64.deb
                      # - libsync_1.0-r1_arm64.deb
                      # - qti-libion_0-r1_arm64.deb
                      # - liblog0_1.0-r1_arm64.deb
                      # - qti-adreno_1.0-r0_arm64.deb
                      COPY dep /root/dep
                      
                      # Create required directory for qti-adreno install
                      RUN mkdir /usr/include/KHR && dpkg -i /root/dep/*.deb 
                      
                      # Copy and build test script
                      COPY ./hellocl /root/hellocl
                      RUN cd /root/hellocl && mkdir build && cd build && cmake .. && make
                      
                      CMD ["bash"]
                      
                      voxl2:~/opencl$ docker run -it --rm --privileged opencl:latest ./root/hellocl/build/hellocl
                      Platform Information:
                      Platform Name: QUALCOMM Snapdragon(TM)
                      Platform Vendor: QUALCOMM
                      Platform Version: OpenCL 2.0 QUALCOMM build: commit # changeid # Date: 11/10/21 Wed Local Branch:  Remote Branch: 
                      Platform Profile: FULL_PROFILE
                      Platform Extensions:  
                      ------------------------------------
                      Device Information:
                      Device Name: QUALCOMM Adreno(TM)
                      Device Vendor: QUALCOMM
                      Driver Version: OpenCL 2.0 QUALCOMM build: commit # changeid # Date: 11/10/21 Wed Local Branch:  Remote Branch:  Compiler E031.37.12.01
                      Device Version: OpenCL 2.0 Adreno(TM) 650
                      Device OpenCL C Version: OpenCL C 2.0 Adreno(TM) 650
                      Device Max Compute Units: 3
                      This should be three: 3
                      
                      Alex KushleyevA 1 Reply Last reply Reply Quote 0
                      • Alex KushleyevA
                        Alex Kushleyev ModalAI Team @eric
                        last edited by

                        hi @eric ,

                        Nice! very clean.

                        Did you use dpkg-repack to create debs of installed packages, such as:

                        apt-get install dpkg-repack
                        dpkg-repack qti-adreno
                        

                        Cool trick!

                        I will test this out and add to our docs.

                        Alex

                        E 1 Reply Last reply Reply Quote 0
                        • E
                          eric @Alex Kushleyev
                          last edited by eric

                          @Alex-Kushleyev Yes, dpkg -S <file path> to figure out which debs installed which libraries (ie, dpkg -S /usr/lib/libOpenCL.so), apt-cache show to see the source (ubuntu ppa vs modalai), then dpkg-repack to repack the modalai debs.

                          Thanks again!

                          Alex KushleyevA 1 Reply Last reply Reply Quote 0
                          • Alex KushleyevA
                            Alex Kushleyev ModalAI Team @eric
                            last edited by

                            @eric , thanks again for your input on this, i have posted a complete tutorial how to enable OpenCL in Docker on VOXL2 : https://docs.modalai.com/voxl-2-opencl-in-docker/

                            Alex

                            E 1 Reply Last reply Reply Quote 0
                            • E
                              eric @Alex Kushleyev
                              last edited by

                              @Alex-Kushleyev Awesome! Thanks again for all your help with this!

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Powered by NodeBB | Contributors