ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Voxl2 Docker (Ubuntu 22) with OpenCL/Adreno

    Ask your questions right here!
    3
    16
    1480
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Alex KushleyevA
      Alex Kushleyev ModalAI Team @eric
      last edited by

      @eric I will try it out, please give me a few days.

      Alex

      Alex KushleyevA 1 Reply Last reply Reply Quote 1
      • Alex KushleyevA
        Alex Kushleyev ModalAI Team @Alex Kushleyev
        last edited by

        Quick update, I did some testing and tried searching documentation and could not get it to work. There are posts online asking Qualcomm whether this is possible, but there is no response there.

        I know this would be a useful feature and I will try again next week. I want to see how this worked on VOXL1 and perhaps I am missing something.

        In my test, I am also just doing a simple device query and it works on the host VOXL2 but not inside docker, tried mapping various .so libraries and devices to the docker container and no luck yet.

        Alex

        E 1 Reply Last reply Reply Quote 0
        • E
          eric @Alex Kushleyev
          last edited by

          @Alex-Kushleyev That aligns pretty well with my own experience so far. Thanks again for looking into this!

          Alex KushleyevA 1 Reply Last reply Reply Quote 0
          • Alex KushleyevA
            Alex Kushleyev ModalAI Team @eric
            last edited by

            @eric I was able to get the GPU device query inside ubuntu 22.04 docker working using the following steps. It is possible that we can reduce the number of mapped devices and libraries to the docker container, but i am just going to give you this information right now so you can test. I will try to clean this up a bit later. I only tried the device query for now, but i figured i would let you know that there is progress..

            #run docker
            docker run -it --rm --privileged --device=/dev/kgsl-3d0 --device=/dev/ion -v /proc:/proc -v /firmware/image:/firmware/image -v /lib/firmware:/lib/firmware -v /sys/class:/sys/class -v /sys/bus:/sys/bus -v /sys/devices:/sys/devices -v /data:/data -v /usr/lib/liblog.so.0:/usr/lib/liblog.so.0 -v /usr/lib/libOpenCL.so:/usr/lib/libOpenCL.so -v /usr/lib/libcutils.so.0:/usr/lib/libcutils.so.0 -v /usr/lib/libllvm-qcom.so:/usr/lib/libllvm-qcom.so -v /usr/lib/libion.so.0.0.0:/usr/lib/libion.so.0.0.0 -v /usr/lib/libsync.so.0.0.0:/usr/lib/libsync.so.0.0.0 -v /usr/lib/libgsl.so:/usr/lib/libgsl.so -v /usr/lib/libCB.so:/usr/lib/libCB.so -v /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0.5600.4:/usr/lib/aarch64-linux-gnu/libglib-2.0.so.0.5600.4 -v `pwd`:/opt/code -w /opt/code arm64v8/ubuntu:22.04 bash
            
            apt-get update
            apt install --no-install-recommends -y pocl-opencl-icd
            

            then run your test app to query the device..

            E 1 Reply Last reply Reply Quote 1
            • E
              eric @Alex Kushleyev
              last edited by

              @Alex-Kushleyev 🙇

              Alex KushleyevA 1 Reply Last reply Reply Quote 0
              • Alex KushleyevA
                Alex Kushleyev ModalAI Team @eric
                last edited by Alex Kushleyev

                OK, a little more clean-up, it seems this is the minimal set of libraries /devices needed:

                docker run -it --rm --privileged \
                	-v /usr/lib/libOpenCL.so:/usr/lib/libOpenCL.so \
                	-v /usr/lib/libCB.so:/usr/lib/libCB.so \
                	-v /usr/lib/libgsl.so:/usr/lib/libgsl.so \
                	-v /usr/lib/liblog.so.0:/usr/lib/liblog.so.0 \
                	-v /usr/lib/libcutils.so.0:/usr/lib/libcutils.so.0 \
                	-v /usr/lib/libsync.so.0.0.0:/usr/lib/libsync.so.0.0.0 \
                	-v /usr/lib/libion.so.0.0.0:/usr/lib/libion.so.0.0.0 \
                	-v /usr/lib/libllvm-qcom.so:/usr/lib/libllvm-qcom.so \
                	-v /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0.5600.4:/usr/lib/aarch64-linux-gnu/libglib-2.0.so.0.5600.4 \
                	-v `pwd`:/opt/code -w /opt/code \
                	arm64v8/ubuntu:22.04 bash
                

                (--privileged mode maps all the needed devices to the docker container)

                Then install some more packages (not sure if this can be reduced, not clear exactly what is missing):

                apt-get update
                apt install --no-install-recommends -y pocl-opencl-icd
                

                Maybe we can figure out what lib is still missing so that pocl-opencl-icd does not have to be installed.. At least the issue was the a missing library, not a mapped device

                For testing, I used a device query script from here:

                root@733a6d4d5fdb:/opt/code# ./simple_query 
                1. Device: QUALCOMM Adreno(TM)
                 1.1 Hardware version: OpenCL 2.0 Adreno(TM) 650
                 1.2 Software version: OpenCL 2.0 QUALCOMM build: commit # changeid # Date: 11/10/21 Wed Local Branch:  Remote Branch:  Compiler E031.37.12.01
                 1.3 OpenCL C version: OpenCL C 2.0 Adreno(TM) 650
                 1.4 Parallel compute units: 3
                

                I also verified that a simple matrix multiplication app also worked (not provided here)

                @eric , can you please let me know if this works for you?

                Alex

                E 1 Reply Last reply Reply Quote 0
                • E
                  eric @Alex Kushleyev
                  last edited by

                  @Alex-Kushleyev

                  OMG IT WORKS!!

                  I was able to extract all these libraries from the host and directly install them inside the docker, and now the pcol-opencl-icd installation isn't needed.

                  This is really important for us, since it allows us to build external dependencies that rely on OpenCL in our pipeline directly without bind mounts (outside the host environment).

                  Really, really appreciate all your help!

                  FROM arm64v8/ubuntu:22.04
                  
                  # Install necessary dependencies
                  RUN apt-get update && \
                      apt-get install -y \
                      cmake \
                      build-essential \
                      libglib2.0-0
                  
                  # Copy Adreno GPU dependencies
                  # - libcutils0_0-r1_arm64.deb
                  # - libsync_1.0-r1_arm64.deb
                  # - qti-libion_0-r1_arm64.deb
                  # - liblog0_1.0-r1_arm64.deb
                  # - qti-adreno_1.0-r0_arm64.deb
                  COPY dep /root/dep
                  
                  # Create required directory for qti-adreno install
                  RUN mkdir /usr/include/KHR && dpkg -i /root/dep/*.deb 
                  
                  # Copy and build test script
                  COPY ./hellocl /root/hellocl
                  RUN cd /root/hellocl && mkdir build && cd build && cmake .. && make
                  
                  CMD ["bash"]
                  
                  voxl2:~/opencl$ docker run -it --rm --privileged opencl:latest ./root/hellocl/build/hellocl
                  Platform Information:
                  Platform Name: QUALCOMM Snapdragon(TM)
                  Platform Vendor: QUALCOMM
                  Platform Version: OpenCL 2.0 QUALCOMM build: commit # changeid # Date: 11/10/21 Wed Local Branch:  Remote Branch: 
                  Platform Profile: FULL_PROFILE
                  Platform Extensions:  
                  ------------------------------------
                  Device Information:
                  Device Name: QUALCOMM Adreno(TM)
                  Device Vendor: QUALCOMM
                  Driver Version: OpenCL 2.0 QUALCOMM build: commit # changeid # Date: 11/10/21 Wed Local Branch:  Remote Branch:  Compiler E031.37.12.01
                  Device Version: OpenCL 2.0 Adreno(TM) 650
                  Device OpenCL C Version: OpenCL C 2.0 Adreno(TM) 650
                  Device Max Compute Units: 3
                  This should be three: 3
                  
                  Alex KushleyevA 1 Reply Last reply Reply Quote 0
                  • Alex KushleyevA
                    Alex Kushleyev ModalAI Team @eric
                    last edited by

                    hi @eric ,

                    Nice! very clean.

                    Did you use dpkg-repack to create debs of installed packages, such as:

                    apt-get install dpkg-repack
                    dpkg-repack qti-adreno
                    

                    Cool trick!

                    I will test this out and add to our docs.

                    Alex

                    E 1 Reply Last reply Reply Quote 0
                    • E
                      eric @Alex Kushleyev
                      last edited by eric

                      @Alex-Kushleyev Yes, dpkg -S <file path> to figure out which debs installed which libraries (ie, dpkg -S /usr/lib/libOpenCL.so), apt-cache show to see the source (ubuntu ppa vs modalai), then dpkg-repack to repack the modalai debs.

                      Thanks again!

                      Alex KushleyevA 1 Reply Last reply Reply Quote 0
                      • Alex KushleyevA
                        Alex Kushleyev ModalAI Team @eric
                        last edited by

                        @eric , thanks again for your input on this, i have posted a complete tutorial how to enable OpenCL in Docker on VOXL2 : https://docs.modalai.com/voxl-2-opencl-in-docker/

                        Alex

                        E 1 Reply Last reply Reply Quote 0
                        • E
                          eric @Alex Kushleyev
                          last edited by

                          @Alex-Kushleyev Awesome! Thanks again for all your help with this!

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Powered by NodeBB | Contributors