ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Qualcomm Flight RB5 5G Camera Setup

    Ask your questions right here!
    rb5 camera rb5-sdk
    2
    11
    270
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      doe43john77
      last edited by

      Hi,

      I'm attempting to setup the hi-res camera on the Qualcomm Flight RB5 5G (RB5).
      So far, I've been following the docs for the Qualcomm Flight RB5 SDK, available here.

      Note: I understand that as of July 2022, the RB5 has integrated the VOXL SDK
      into its firmware, replacing the old RB5 SDK. When I first began setting up the
      RB5 and trying to connect it to QGroundControl (QGC), I performed this firware
      update. In the process of troubleshooting the connection process, I was looking
      through some systemd error messages which said that I was missing some
      packages which happened to belong to the old RB5 SDK. I ended up reinstalling
      the old firmware. The whole process of connecting to QGC took 2 weeks, so I'd
      like to avoid a firmware update to VOXL if possible.

      My first step was to configure the rb5-camera-server service via the adb
      shell to run on startup using the rb5-camera-server-configure command. After
      running this, I checked to see if the services were running for cameras 0, 1, 2
      , 4 and 5. Here is the output:

      • Cameras 0, 1, 2 and 5 have similar outputs where the say there is an unknown
        "lvalue" in the systemd configuration file (which I never touched):
      ● rb5-camera-server@0.service - "rb5-camera-server-0"
   Loaded: loaded (/usr/bin/rb5-camera-server; indirect; vendor preset: enabled)
   Active: active (running) since Thu 2023-06-08 15:35:49 UTC; 34s ago
 Main PID: 2197 (rb5-camera-serv)
    Tasks: 4 (limit: 4915)
   CGroup: /system.slice/system-rb5\x2dcamera\x2dserver.slice/rb5-camera-server@0.service
           └─2197 /usr/bin/rb5-camera-server -c 0

Jun 08 15:35:48 m0052 systemd[1]: Starting "rb5-camera-server-0"...
Jun 08 15:35:49 m0052 systemd[1]: Started "rb5-camera-server-0".
Jun 08 15:35:49 m0052 rb5-camera-server[2197]: gbm_create_device(156): Info: backend name is: msm_drm
Jun 08 15:35:49 m0052 systemd[1]: /etc/systemd/system/rb5-camera-server@.service:12: Unknown lvalue 'After' in section 'Service'
Jun 08 15:35:49 m0052 systemd[1]: /etc/systemd/system/rb5-camera-server@.service:13: Unknown lvalue 'Requires' in section 'Service'
      • Camera 4, on the other hand, gives the following output which follows the
        example in the docs:
      ● rb5-camera-server@4.service - "rb5-camera-server-4"
   Loaded: loaded (/usr/bin/rb5-camera-server; indirect; vendor preset: enabled)
   Active: active (running) since Thu 2023-06-08 15:35:50 UTC; 15min ago
  Process: 2219 ExecStartPre=/bin/sleep 1 (code=exited, status=0/SUCCESS)
 Main PID: 2224 (rb5-camera-serv)
    Tasks: 4 (limit: 4915)
   CGroup: /system.slice/system-rb5\x2dcamera\x2dserver.slice/rb5-camera-server@4.service
           └─2224 /usr/bin/rb5-camera-server -c 4

Jun 08 15:35:49 m0052 systemd[1]: Starting "rb5-camera-server-4"...
Jun 08 15:35:50 m0052 systemd[1]: Started "rb5-camera-server-4".
Jun 08 15:35:52 m0052 rb5-camera-server[2224]: gbm_create_device(156): Info: backend name is: msm_drm

      So, my first question: Why is the camera server service saying that there is
      an unknown option in the startup configuration files when I never touched them
      to begin with?

      Next, I tested the cameras with the rb5-camera-client command. All cameras
      began pushing metadata through the pipe successfully. I confirmed this by using
      systemctl status rb5-camera-server@X.

      Frame Information:
Individual Pixel Value 500: 4 

Metadata Information:
Timestamp:1267899026668 
Frame ID:43
Frame Width:640
Frame Height:480
fps: 30.00125

      The frame width and FPS line up with the specs of the secondary cameras. An example of the systemd log is shown below:

      ● rb5-camera-server@0.service - "rb5-camera-server-0"
   Loaded: loaded (/usr/bin/rb5-camera-server; indirect; vendor preset: enabled)
   Active: active (running) since Thu 2023-06-08 15:35:49 UTC; 28min ago
 Main PID: 2197 (rb5-camera-serv)
    Tasks: 4 (limit: 4915)
   CGroup: /system.slice/system-rb5\x2dcamera\x2dserver.slice/rb5-camera-server@0.service
           └─2197 /usr/bin/rb5-camera-server -c 0

Jun 08 15:35:48 m0052 systemd[1]: Starting "rb5-camera-server-0"...
Jun 08 15:35:49 m0052 systemd[1]: Started "rb5-camera-server-0".
Jun 08 15:35:49 m0052 rb5-camera-server[2197]: gbm_create_device(156): Info: backend name is: msm_drm
Jun 08 15:35:49 m0052 systemd[1]: /etc/systemd/system/rb5-camera-server@.service:12: Unknown lvalue 'After' in section 'Service'
Jun 08 15:35:49 m0052 systemd[1]: /etc/systemd/system/rb5-camera-server@.service:13: Unknown lvalue 'Requires' in section 'Service'
Jun 08 15:55:26 m0052 rb5-camera-server[2197]: camera ID is set to 0
Jun 08 15:55:26 m0052 rb5-camera-server[2197]: [INFO] Publishing to the following pipe: /dev/rb5-camera-pipe-0
Jun 08 15:55:26 m0052 rb5-camera-server[2197]: [INFO] Started camera ID: 0

      Second step was to configure the rb5-streamer service (which controls
      camera 3, the hi-res camera) to run on startup using the
      rb5-streamer-configure factory_enable command. The service seems to start up
      normally based on the systemd log:

      ● rb5-streamer.service - rb5-streamer
   Loaded: loaded (/etc/systemd/system/rb5-streamer.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2023-06-08 16:09:28 UTC; 14s ago
 Main PID: 10448 (rb5-streamer)
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/rb5-streamer.service
           └─10448 /usr/bin/rb5-streamer

Jun 08 16:09:28 m0052 systemd[1]: Started rb5-streamer.
Jun 08 16:09:28 m0052 rb5-streamer[10448]: [INFO] RTSP stream ready at rtsp://VEHICLES_IP:8554/camera3

      For references, here is the example from the technical docs:

      ● rb5-streamer.service - "rb5-streamer"
   Loaded: loaded (/usr/bin/rb5-streamer; indirect; vendor preset: enabled)
   Active: active (running) since Mon 2021-07-26 16:28:00 UTC; 15s ago

Jul 26 16:28:00 qrb5165-rb5 systemd[1]: Started "rb5-streamer".
Jul 26 16:28:00 qrb5165-rb5 rb5-streamer[9522]: gbm_create_device(156)
Jul 26 16:28:00 qrb5165-rb5 rb5-streamer[9522]: stream ready at rtsp://127.0.0.1:8554/test

      My output is missing gbm_create_device(156) and the actual IP address. I
      should note that the drone is connected to Wi-Fi successfully and has an IP
      address that I can ping and SSH into.

      For the most part, everything starts up without any hiccups. However, when I
      try to open the stream in VLC the stream fails. QGC doesn't even show the video
      and I'm stumped. I followed the directions, checked I was putting the right
      RTSP addressed and I have no clue how to proceed.

      Help would be immensely appreciated.

      Thanks!

      tomT 1 Reply Last reply Reply Quote 0
      • tomT
        tom admin @doe43john77
        last edited by

        @doe43john77 I'd recommend loading VOXL SDK 0.9.5 for RB5 and going from there: https://developer.modalai.com/asset/view/124

        D 1 Reply Last reply Reply Quote 0
        • D
          doe43john77 @tom
          last edited by

          @tom Hi Tom. I updated the firmware of the RB5 to the VOXL 0.9.5 SDK. The update went smoothly. I'm trying to connect the drone to QGroundControl but I'm not having luck. I think I've covered all my bases: the host PC and drone are on the same Wi-Fi network, the drone's voxl-px4 service is running (as confirmed by using the voxl-inspect-services command). Suggestions?

          tomT 1 Reply Last reply Reply Quote 0
          • tomT
            tom admin @doe43john77
            last edited by

            @doe43john77 There are a few other services necessary in order to connect to QGC.

            • voxl-mavlink-server needs to be running
            • voxl-vision-px4 needs to be running

            You also need to modify the config file at /etc/modalai/voxl-vision-px4.conf and change the qgc_ip value to the IP address of your ground station and restart the service with:
            systemctl restart voxl-vision-px4

            D 1 Reply Last reply Reply Quote 0
            • D
              doe43john77 @tom
              last edited by

              @tom I have enabled and started running the voxl-mavlink-server service but only been able to enable (but not start) the voxl-vision-px4 service. I've included a more detailed output voxl-vision-px4 service by running systemctl status voxl-vision-px4 -l:

              rb5:/$ systemctl status voxl-vision-px4 -l
● voxl-vision-px4.service - voxl-vision-px4
   Loaded: loaded (/usr/bin/voxl-vision-px4; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2023-06-12 18:52:55 UTC; 33s ago
  Process: 2759 ExecStart=/usr/bin/voxl-vision-px4 (code=exited, status=255)
 Main PID: 2759 (code=exited, status=255)

Jun 12 18:52:55 m0052 voxl-vision-px4[2759]:     frame:              tof
Jun 12 18:52:55 m0052 systemd[1]: voxl-vision-px4.service: Main process exited, code=exited, status=255/n/a
Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: voa_input #4
Jun 12 18:52:55 m0052 voxl-vision-px4[2759]:     enabled:            1
Jun 12 18:52:55 m0052 voxl-vision-px4[2759]:     type:               rangefinder
Jun 12 18:52:55 m0052 voxl-vision-px4[2759]:     input_pipe:         rangefinders
Jun 12 18:52:55 m0052 voxl-vision-px4[2759]:     frame:              body
Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: =================================================================
Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: loading extrinsics config file
Jun 12 18:52:55 m0052 systemd[1]: voxl-vision-px4.service: Failed with result 'exit-code'.

              Also, I tried modifying the voxl-vision-px4.conf file with the IP address of my host machine with QGC in case the voxl-vision-px4 service needs the IP address for proper startup, but that didn't help.

              Additionally, I saw the word "extrinsics" in the systemctl output and I was curious if it was referring to the opencv files we had to backup before the upgrading the SDK (which I did btw).

              tomT 1 Reply Last reply Reply Quote 0
              • tomT
                tom admin @doe43john77
                last edited by

                @doe43john77 In order to get all of the services in a know good spot, you should be able to run voxl-configure-mpa and select RB5-Flight and it'll setup all the services all needed. Then you would have to go back and modify the qgc_ip again in the conf file.

                In regard to the opencv files, that's likely unrelated to this issue, but to restore them, you would just throw them into /data/modalai

                D 1 Reply Last reply Reply Quote 0
                • D
                  doe43john77 @tom
                  last edited by

                  @tom Got it! Its connecting to QGC now!

                  I still don't understand how to start up the cameras. The instructions in the user guide are in terms of the deprecated rb5-streamer package.

                  tomT 1 Reply Last reply Reply Quote 0
                  • tomT
                    tom admin @doe43john77
                    last edited by

                    @doe43john77 If you ran through voxl-configure-mpa then voxl-camera-server should be enabled and running, you can check this with voxl-inspect-services.

                    Your options for viewing a camera stream are either using voxl-portal (navigate to the IP address of your drone in your PC's browser and select the camera tab)

                    or using voxl-streamer in order to stream to programs like QGC or VLC. This can be setup using voxl-configure-streamer

                    D 1 Reply Last reply Reply Quote 0
                    • D
                      doe43john77 @tom
                      last edited by

                      @tom Got it! I can access the VOXL Portal page and view the camera feed. The hi-res camera is sending 480p video for some reason. Is there a configuration file I need to tweak to push out the 4K resolution?

                      tomT 1 Reply Last reply Reply Quote 0
                      • tomT
                        tom admin @doe43john77
                        last edited by

                        @doe43john77 You can find some info here: https://docs.modalai.com/voxl-camera-server/#changing-the-resolution-of-an-image-sensor

                        The default was set low as otherwise you're pumping a ton of data over the network which may not be necessary depending on the application

                        D 1 Reply Last reply Reply Quote 0
                        • D
                          doe43john77 @tom
                          last edited by

                          @tom Ah. This is a simple config file change. Thank you!

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Powered by NodeBB | Contributors