Using realsense camera for tflite-server inference
-
Hi need to use a realsense camera (L515 in particular but I tried also with a d455 and is the same) as a uvc device to be used with the voxl-tflite-server, on voxl2 with sdk 1.1.2 but also tried with a 0.9.5 and the behaviour was the same.
The camera is connected and detected as shown by
lsusb Bus 002 Device 004: ID 8086:0b64 Intel Corp. Bus 002 Device 003: ID 2c7c:0800 Bus 002 Device 002: ID 0424:5744 Standard Microsystems Corp. Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 005: ID 0424:2740 Standard Microsystems Corp. Bus 001 Device 004: ID 0b95:1790 ASIX Electronics Corp. AX88179 Gigabit Ethernet Bus 001 Device 002: ID 0424:2744 Standard Microsystems Corp. Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hubwith video streams from 2 to 9
voxl2:~$ ls /dev/video video0 video2 video32 video4 video6 video8 video1 video3 video33 video5 video7 video9```When I try to run
voxl-uvc-server -v 8086 -p 0b64 -dspecifying the vendor and the product, the camera is detected but when it tries to open the camera the server crash (I have also changed image dimension but nothing changed)
loading config file Enabling debug messages ================================================================= width: 640 height: 480 fps: 30 pipe_name: uvc ================================================================= voxl-uvc-server starting Image resolution 640x480, 30 fps chosen Vendor ID 0x8086 chosen Product ID 0x0b64 chosen UVC initialized Device 8086:0b64 found uvc_open failed UVC exited voxl-uvc-server endingI have tried to run the uvc-server with a standar logitech webcam and it works fine.
I tried also to simulate the behaviour of a webcam using v4l2loopback as suggested here but the kernel module is not supported and I am not able to install the package, I have then tried also the start the compilation of the kernel to understand if it was feasible to add it manually but also this process crashed at the sync phase within the container tutorial :
user@54bf0bc04ffb:~$ ./qrb5165-sync.sh [INFO] - Syncing CLO repo... File "/usr/bin/repo", line 51 def print(self, *args, **kwargs): ^ SyntaxError: invalid syntaxAlso the camera works properly when connected the voxl2 and with the realsense packages installed inside a seperate container.
The question is, how can I achieve to connect the camera to the tflite server? any suggestion or different solution is welcome
-
@Riccardo-Franceschini Hi Riccardo, we do not have any support for Realsense so you will need to write your own MPA server to connect to tflite-server. You could start with forking the uvc-server or trying to fix the crash you are currently encountering. we are happy to incorporate merge requests
-
Hi @Riccardo-Franceschini ,
sorry to reactivate this old topic but I'm stuck trying to get a real-sense D455 to work over the USB-port of the voxl2. Since apparently the real-sense products are not support by modalai I'd like to know how you got the camera connected and working.
Thanks in advance! -
Hi @K-Stute, sorry for the late reply .
Anyway, I connected the camera over the usb connection on the 5G modem, you need to be sure to have the proper cable (the standard one of the realsense is fine) and that the connection is over a usb 3.0 (only one of the two port of the modem works), otherwise everything works quite out of the box if you run everything on docker container (haven't tried without a container).
-
@Riccardo-Franceschini is there any chance that you could send me your docker image (or maybe the layers that are used to install the realsense "software")? I've been struggling to get the realsense in a ros2 docker working. The only think that worked for me is a docker image based on Alpine.
