# Qualcomm Flight RB5 5G Camera Setup Source: https://forum.modalai.com/topic/2239/qualcomm-flight-rb5-5g-camera-setup Category: General Questions (https://forum.modalai.com/category/2/general-questions) Tags: rb5, camera, rb5-sdk Posted: 2023-06-08 17:01:04 UTC by doe43john77 Replies: 10 · Views: 3119 ## doe43john77 · 2023-06-08 17:01:04 UTC Hi, I'm attempting to setup the hi-res camera on the Qualcomm Flight RB5 5G (RB5). So far, I've been following the docs for the Qualcomm Flight RB5 SDK, available [here](https://docs.modalai.com/Qualcomm-Flight-RB5-sdk/). --- *Note: I understand that as of July 2022, the RB5 has integrated the VOXL SDK into its firmware, replacing the old RB5 SDK. When I first began setting up the RB5 and trying to connect it to QGroundControl (QGC), I performed this firware update. In the process of troubleshooting the connection process, I was looking through some `systemd` error messages which said that I was missing some packages which happened to belong to the old RB5 SDK. I ended up reinstalling the old firmware. The whole process of connecting to QGC took 2 weeks, so I'd like to avoid a firmware update to VOXL if possible.* --- My first step was to configure the `rb5-camera-server` service via the `adb` shell to run on startup using the `rb5-camera-server-configure` command. After running this, I checked to see if the services were running for cameras 0, 1, 2 , 4 and 5. Here is the output: - Cameras 0, 1, 2 and 5 have similar outputs where the say there is an unknown "lvalue" in the `systemd` configuration file (which I never touched): ``` ● rb5-camera-server@0.service - "rb5-camera-server-0" Loaded: loaded (/usr/bin/rb5-camera-server; indirect; vendor preset: enabled) Active: active (running) since Thu 2023-06-08 15:35:49 UTC; 34s ago Main PID: 2197 (rb5-camera-serv) Tasks: 4 (limit: 4915) CGroup: /system.slice/system-rb5\x2dcamera\x2dserver.slice/rb5-camera-server@0.service └─2197 /usr/bin/rb5-camera-server -c 0 Jun 08 15:35:48 m0052 systemd[1]: Starting "rb5-camera-server-0"... Jun 08 15:35:49 m0052 systemd[1]: Started "rb5-camera-server-0". Jun 08 15:35:49 m0052 rb5-camera-server[2197]: gbm_create_device(156): Info: backend name is: msm_drm Jun 08 15:35:49 m0052 systemd[1]: /etc/systemd/system/rb5-camera-server@.service:12: Unknown lvalue 'After' in section 'Service' Jun 08 15:35:49 m0052 systemd[1]: /etc/systemd/system/rb5-camera-server@.service:13: Unknown lvalue 'Requires' in section 'Service' ``` - Camera 4, on the other hand, gives the following output which follows the example in the docs: ``` ● rb5-camera-server@4.service - "rb5-camera-server-4" Loaded: loaded (/usr/bin/rb5-camera-server; indirect; vendor preset: enabled) Active: active (running) since Thu 2023-06-08 15:35:50 UTC; 15min ago Process: 2219 ExecStartPre=/bin/sleep 1 (code=exited, status=0/SUCCESS) Main PID: 2224 (rb5-camera-serv) Tasks: 4 (limit: 4915) CGroup: /system.slice/system-rb5\x2dcamera\x2dserver.slice/rb5-camera-server@4.service └─2224 /usr/bin/rb5-camera-server -c 4 Jun 08 15:35:49 m0052 systemd[1]: Starting "rb5-camera-server-4"... Jun 08 15:35:50 m0052 systemd[1]: Started "rb5-camera-server-4". Jun 08 15:35:52 m0052 rb5-camera-server[2224]: gbm_create_device(156): Info: backend name is: msm_drm ``` So, my first question: **Why is the camera server service saying that there is an unknown option in the startup configuration files when I never touched them to begin with?** Next, I tested the cameras with the `rb5-camera-client` command. All cameras began pushing metadata through the pipe successfully. I confirmed this by using `systemctl status rb5-camera-server@X`. ``` Frame Information: Individual Pixel Value 500: 4 Metadata Information: Timestamp:1267899026668 Frame ID:43 Frame Width:640 Frame Height:480 fps: 30.00125 ``` The frame width and FPS line up with the specs of the secondary cameras. An example of the `systemd` log is shown below: ``` ● rb5-camera-server@0.service - "rb5-camera-server-0" Loaded: loaded (/usr/bin/rb5-camera-server; indirect; vendor preset: enabled) Active: active (running) since Thu 2023-06-08 15:35:49 UTC; 28min ago Main PID: 2197 (rb5-camera-serv) Tasks: 4 (limit: 4915) CGroup: /system.slice/system-rb5\x2dcamera\x2dserver.slice/rb5-camera-server@0.service └─2197 /usr/bin/rb5-camera-server -c 0 Jun 08 15:35:48 m0052 systemd[1]: Starting "rb5-camera-server-0"... Jun 08 15:35:49 m0052 systemd[1]: Started "rb5-camera-server-0". Jun 08 15:35:49 m0052 rb5-camera-server[2197]: gbm_create_device(156): Info: backend name is: msm_drm Jun 08 15:35:49 m0052 systemd[1]: /etc/systemd/system/rb5-camera-server@.service:12: Unknown lvalue 'After' in section 'Service' Jun 08 15:35:49 m0052 systemd[1]: /etc/systemd/system/rb5-camera-server@.service:13: Unknown lvalue 'Requires' in section 'Service' Jun 08 15:55:26 m0052 rb5-camera-server[2197]: camera ID is set to 0 Jun 08 15:55:26 m0052 rb5-camera-server[2197]: [INFO] Publishing to the following pipe: /dev/rb5-camera-pipe-0 Jun 08 15:55:26 m0052 rb5-camera-server[2197]: [INFO] Started camera ID: 0 ``` Second step was to configure the `rb5-streamer` service (which controls camera 3, the hi-res camera) to run on startup using the `rb5-streamer-configure factory_enable` command. The service seems to start up normally based on the `systemd` log: ``` ● rb5-streamer.service - rb5-streamer Loaded: loaded (/etc/systemd/system/rb5-streamer.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2023-06-08 16:09:28 UTC; 14s ago Main PID: 10448 (rb5-streamer) Tasks: 2 (limit: 4915) CGroup: /system.slice/rb5-streamer.service └─10448 /usr/bin/rb5-streamer Jun 08 16:09:28 m0052 systemd[1]: Started rb5-streamer. Jun 08 16:09:28 m0052 rb5-streamer[10448]: [INFO] RTSP stream ready at rtsp://VEHICLES_IP:8554/camera3 ``` For references, here is the example from the technical docs: ``` ● rb5-streamer.service - "rb5-streamer" Loaded: loaded (/usr/bin/rb5-streamer; indirect; vendor preset: enabled) Active: active (running) since Mon 2021-07-26 16:28:00 UTC; 15s ago Jul 26 16:28:00 qrb5165-rb5 systemd[1]: Started "rb5-streamer". Jul 26 16:28:00 qrb5165-rb5 rb5-streamer[9522]: gbm_create_device(156) Jul 26 16:28:00 qrb5165-rb5 rb5-streamer[9522]: stream ready at rtsp://127.0.0.1:8554/test ``` My output is missing `gbm_create_device(156)` and the actual IP address. I should note that the drone is connected to Wi-Fi successfully and has an IP address that I can ping and SSH into. For the most part, everything starts up without any hiccups. However, when I try to open the stream in VLC the stream fails. QGC doesn't even show the video and I'm stumped. I followed the directions, checked I was putting the right RTSP addressed and I have no clue how to proceed. Help would be immensely appreciated. Thanks! ## Reply by tom (ModalAI staff) · 2023-06-08 17:05:07 UTC @doe43john77 I'd recommend loading VOXL SDK 0.9.5 for RB5 and going from there: https://developer.modalai.com/asset/view/124 ## Reply by doe43john77 · 2023-06-12 14:16:46 UTC (in reply to tom) @tom Hi Tom. I updated the firmware of the RB5 to the VOXL 0.9.5 SDK. The update went smoothly. I'm trying to connect the drone to QGroundControl but I'm not having luck. I think I've covered all my bases: the host PC and drone are on the same Wi-Fi network, the drone's `voxl-px4` service is running (as confirmed by using the `voxl-inspect-services` command). Suggestions? ## Reply by tom (ModalAI staff) · 2023-06-12 17:01:48 UTC (in reply to doe43john77) @doe43john77 There are a few other services necessary in order to connect to QGC. - `voxl-mavlink-server` needs to be running - `voxl-vision-px4` needs to be running You also need to modify the config file at `/etc/modalai/voxl-vision-px4.conf` and change the `qgc_ip` value to the IP address of your ground station and restart the service with: `systemctl restart voxl-vision-px4` ## Reply by doe43john77 · 2023-06-12 18:58:18 UTC (in reply to tom) @tom I have enabled and started running the `voxl-mavlink-server` service but only been able to *enable* (but not start) the `voxl-vision-px4` service. I've included a more detailed output `voxl-vision-px4` service by running `systemctl status voxl-vision-px4 -l`: ``` rb5:/$ systemctl status voxl-vision-px4 -l ● voxl-vision-px4.service - voxl-vision-px4 Loaded: loaded (/usr/bin/voxl-vision-px4; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Mon 2023-06-12 18:52:55 UTC; 33s ago Process: 2759 ExecStart=/usr/bin/voxl-vision-px4 (code=exited, status=255) Main PID: 2759 (code=exited, status=255) Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: frame: tof Jun 12 18:52:55 m0052 systemd[1]: voxl-vision-px4.service: Main process exited, code=exited, status=255/n/a Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: voa_input #4 Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: enabled: 1 Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: type: rangefinder Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: input_pipe: rangefinders Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: frame: body Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: ================================================================= Jun 12 18:52:55 m0052 voxl-vision-px4[2759]: loading extrinsics config file Jun 12 18:52:55 m0052 systemd[1]: voxl-vision-px4.service: Failed with result 'exit-code'. ``` Also, I tried modifying the `voxl-vision-px4.conf` file with the IP address of my host machine with QGC in case the `voxl-vision-px4` service needs the IP address for proper startup, but that didn't help. Additionally, I saw the word "extrinsics" in the `systemctl` output and I was curious if it was referring to the opencv files we had to backup before the upgrading the SDK (which I did btw). ## Reply by tom (ModalAI staff) · 2023-06-12 20:34:19 UTC (in reply to doe43john77) @doe43john77 In order to get all of the services in a know good spot, you should be able to run `voxl-configure-mpa` and select RB5-Flight and it'll setup all the services all needed. Then you would have to go back and modify the qgc_ip again in the conf file. In regard to the opencv files, that's likely unrelated to this issue, but to restore them, you would just throw them into `/data/modalai` ## Reply by doe43john77 · 2023-06-13 14:45:12 UTC (in reply to tom) @tom Got it! Its connecting to QGC now! I still don't understand how to start up the cameras. The instructions in the user guide are in terms of the deprecated `rb5-streamer` package. ## Reply by tom (ModalAI staff) · 2023-06-13 17:11:18 UTC (in reply to doe43john77) @doe43john77 If you ran through `voxl-configure-mpa` then `voxl-camera-server` should be enabled and running, you can check this with `voxl-inspect-services`. Your options for viewing a camera stream are either using `voxl-portal` (navigate to the IP address of your drone in your PC's browser and select the camera tab) or using `voxl-streamer` in order to stream to programs like QGC or VLC. This can be setup using `voxl-configure-streamer` ## Reply by doe43john77 · 2023-06-14 13:22:16 UTC (in reply to tom) @tom Got it! I can access the VOXL Portal page and view the camera feed. The hi-res camera is sending 480p video for some reason. Is there a configuration file I need to tweak to push out the 4K resolution? ## Reply by tom (ModalAI staff) · 2023-06-14 17:17:14 UTC (in reply to doe43john77) @doe43john77 You can find some info here: https://docs.modalai.com/voxl-camera-server/#changing-the-resolution-of-an-image-sensor The default was set low as otherwise you're pumping a ton of data over the network which may not be necessary depending on the application ## Reply by doe43john77 · 2023-06-15 14:23:41 UTC (in reply to tom) @tom Ah. This is a simple config file change. Thank you!