# Should I upgrade? Source: https://forum.modalai.com/topic/1300/should-i-upgrade Category: General Questions (https://forum.modalai.com/category/2/general-questions) Posted: 2022-09-06 15:08:50 UTC by Ed Sutter Replies: 32 · Views: 8142 ## Ed Sutter · 2022-09-06 15:08:50 UTC Hi, I have a VOXL Flight Deck, and I've been doing things for several weeks but now I'm stumbling a bit trying to figure out how to stream a USB camera using voxl-streamer and voxl-uvc-server. Anyway, it makes me wonder if I should have done an upgrade when I first unwrapped this. The output of voxl-version is as follows: ``` / # voxl-version cat: /etc/modalai/voxl-software-bundle-version.txt: No such file or directory -------------------------------------------------------------------------------- system-image: ModalAI 3.3.0 BUILDER: ekatzfey BUILD_TIME: 2021-06-06_19:28 kernel: #1 SMP PREEMPT Sun Jun 6 19:41:01 UTC 2021 3.18.71-perf factory-bundle: 1.0.1 (Yocto installation) -------------------------------------------------------------------------------- architecture: aarch64 processor: apq8096 os: GNU/Linux -------------------------------------------------------------------------------- voxl-suite: Package: voxl-suite Version: 0.5.0 Depends: libjpeg_turbo (>= 9.0.4), libmodal_exposure (>= 0.0.2), libmodal_json (>= 0.3.6), libmodal_pipe (>= 2.1.1), librc_ma th (>= 1.1.5), libvoxl_cutils (>= 0.0.2), libvoxl_io (>= 0.5.4), mavlink-camera-manager (>= 0.0.2), mongoose (>= 7.3.0), open cv (>= 4.5.2-2), openmp (>= 10.0.2), voxl-camera-calibration (>= 0.1.1), voxl-camera-server (>= 0.8.1), voxl-cpu-monitor (>= 0.2.0), voxl-dfs-server (>= 0.2.2), voxl-docker-support (>= 1.1.3), voxl-gphoto2 (>= 0.0.5), voxl-imu-server (>= 0.9.1), voxl -mavlink (>= 0.0.2), voxl-modem (>= 0.12.0), voxl-mpa-tools (>= 0.3.6), voxl-nodes (>= 0.2.0), voxl-portal (>= 0.1.2), voxl-q vio-server (>= 0.3.4), voxl-streamer (>= 0.2.6), voxl-tag-detector (>= 0.0.2), voxl-tflite (>= 2.2.3), voxl-tflite-server (>= 0.1.5), voxl-utils (>= 0.8.4), voxl-vision-px4 (>= 0.9.5), voxl-vpn (>= 0.0.3) Status: install user installed Section: base Architecture: all Maintainer: james@modalai.com MD5Sum: 172dd2541a4b01ec415651b5ded3afab Size: 1956 Filename: voxl-suite_0.5.0_202111041852.ipk Description: meta-package for voxl-suite software collection Installed-Time: 144 -------------------------------------------------------------------------------- ``` Is it best to step back and flash a new system image with voxl_platform_3.8.0-0.7? Thanks in advance, Ed ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-06 15:24:05 UTC @Ed-Sutter It's definitely better to be on the latest release. What specific issue are you having with USB camera streaming? ## Reply by Ed Sutter · 2022-09-06 15:39:12 UTC (in reply to Eric Katzfey) @Eric-Katzfey Ok, I'll update now. Regarding the USB camera... I need to get an external USB camera streaming. Initially I just plugged one into the USB port on the LTE-addon module we have plugged into the expansion bus. Both *lsusb* and "*ls /dev/video**" show the new device, but after adding a configuration item to */etc/modalai/voxl-streamer.conf* and running "*voxl-streamer -c my-new-tag*" the streaming was about 1 frame every 30 seconds. Then I stumbled on the note further in the documentation about the use of **voxl-uvc-server**; but I don't quite understand how the two work together. Anyway, that's when I realized I was working with an image that is over a year old and posted this question... I'll upgrade, then try things again.. Update: The confusion I have is this... do I set up an entry in the voxl-streamer.conf file for the new camera? If yes, is the device mpa (since the uvc-server is running) or is it the /dev/videoX entry? ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-06 16:43:57 UTC (in reply to Ed Sutter) @Ed-Sutter Yes, use the ```uvc-generic``` record as a template. You can modify resolution, bit rate, etc. to match what you are putting into the pipe from voxl-uvc-server. ## Reply by Ed Sutter · 2022-09-06 17:37:29 UTC (in reply to Eric Katzfey) @Eric-Katzfey Ok, after running opkg update/opkg upgrade I see there now is a **uvc-generic** entry (that wasn't there prior to the upgrade). I'm still having trouble though. Here's what I'm doing sequentially: 1) Plug in my camera and get the VID/PID. 2) Run voxl-uvc-server -v MYVID -p MYPID -r640x480. 3) Run voxl-streamer -c uvc-generic. The first time I ran this voxl-streamer responded with: ``` Using configuration uvc-generic Using configuration file /etc/modalai/voxl-streamer.conf Camera server Connected ERROR: Frame size mismatch 613388 614400 ERROR: Timeout on input parameter initialization ``` and eventually voxl-uvc-server crashed (Segmentation fault). Then, strangely, I tried it a second time (changed nothing) and things were somewhat better. Voxl-streamer responded with: ``` Using configuration uvc-generic Using configuration file /etc/modalai/voxl-streamer.conf Camera server Connected Stream available at rtsp://127.0.0.1:8900/live ``` and with that I was able to run VLC on my host to see some [not-so-good] video. The frames were breaking up like they were out of sync. I used ctrl-c to terminate voxl-streamer, and shortly after that voxl-uvc-server crashed again. Any clue what might be going on? Are there other pages I should be reading besides [voxl-uvc-server](https://docs.modalai.com/voxl-uvc-server/) and [voxl-streamer](https://docs.modalai.com/voxl-streamer/)? ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-06 17:44:00 UTC (in reply to Ed Sutter) @Ed-Sutter Try adding the debug options when launching both voxl-uvc-server and voxl-streamer and then share the output. What camera are you using? ## Reply by Ed Sutter · 2022-09-06 18:24:07 UTC (in reply to Eric Katzfey) @Eric-Katzfey With -d on voxl-uvc-server, and voxl-streamer active, I see this output... ``` write to ch: 0 id: 0 result: 614400 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 40 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 613388 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 40 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 614400 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 40 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 613388 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 40 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 612364 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 40 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 611364 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 40 errno: 0 previous client state was 2 write to ch: 0 id: 0 result: 610328 errno: 0 ``` At 640x480 a consistent frame size would be 614400, so this is likely the cause of the frame breakup that I see visually. My camera is a Microsoft 1080P HD Sensor, that can emit YUY2, M420 or MJPG format. ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-06 18:29:14 UTC (in reply to Ed Sutter) @Ed-Sutter Can you share all of the output from both applications? We have never tried that camera. The supported format would be YUY2 but that should result in a fixed size frame. Not sure why it is moving around like that. ## Reply by Ed Sutter · 2022-09-06 18:38:16 UTC (in reply to Eric Katzfey) @Eric-Katzfey Sure... I just ran it again, and didn't even get to run VLC on my host and the server crashed. Here are the outputs... voxl-uvc_server: ``` / # voxl-uvc-server -v045e -p0772 -r640x480 -d Enabling debug messages voxl-uvc-server starting Vendor ID 0x045e chosen Product ID 0x0772 chosen Image resolution 640x480, 30 fps chosen UVC initialized Device 045e:0772 found Device opened uvc_get_stream_ctrl_format_size succeeded for format 0 Streaming starting Got frame callback! frame_format = 3, width = 640, height = 480, length = 613388, ptr = (nil) * got image 30 * got image 60 * got image 90 * got image 120 * got image 150 * got image 180 * got image 210 * got image 240 * got image 270 * got image 300 * got image 330 * got image 360 * got image 390 * got image 420 * got image 450 * got image 480 * got image 510 * got image 540 * got image 570 * got image 600 Segmentation fault / # ``` voxl-streamer: ``` / # voxl-streamer -c uvc-generic -d Using configuration uvc-generic Enabling debug messages Using configuration file /etc/modalai/voxl-streamer.conf MPA interface chosen Input pipe name /run/mpa/uvc/ Output stream width 640 Output stream height 480 Output stream bitrate 1000000 Output stream rotation 0 Output frame decimator 1 No image overlay specified Camera server Connected First frame timestamp: 5791926161024 Second frame timestamp: 5791958239848 Calculated frame delta in ns: 32078824 Calculated frame delta in 100us: 320 Calculated input frame rate is: 31 Output frame rate will be: 31 Meta data from incoming frame: magic_number 0x564F584C timestamp_ns: 5791958239848 frame_id: 519 width: 640 height: 480 size_bytes: 614400 stride: 1280 exposure_ns: 100 gain: 100 format: 8 Input parameters initialized Made rtsp_server Created RTSP server main loop context Created RTSP server main loop Created RTSP server main loop source for callback Got 2 from gst_rtsp_server_attach Stream available at rtsp://127.0.0.1:8900/live Camera server Disconnected ERROR in pipe_client_init_channel opening request pipe: No such device or address Most likely the server stopped without cleaning up Client is cleaning up pipes for the server ^CGot SIGINT, exiting Trying to stop loop g_main_loop exited voxl-streamer ending / # ``` You can see in the voxl-streamer output that it was running till voxl-uvc-server crashed. Eventually, I used Ctrl-C to terminate voxl-streamer. If you want me to try anything, just let me know. I've worked with several USB cameras myself but primarily with V4L2/OpenCV/Gstreamer. I haven't used LibUVC, can't find much documentation. Can you point me to the libuvc source? ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-06 18:47:43 UTC (in reply to Ed Sutter) @Ed-Sutter We use our own version of libuvc. You can find our project here: https://gitlab.com/voxl-public/voxl-sdk/third-party/voxl-libuvc ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-06 18:49:11 UTC (in reply to Ed Sutter) @Ed-Sutter I think the problem is the varying frame size. That's probably causing a problem somewhere (either memory leak or overflow). ## Reply by Ed Sutter · 2022-09-06 19:17:46 UTC (in reply to Eric Katzfey) @Eric-Katzfey I agree, but I don't understand why the frame size varies! Its not compressed, so it should be a fixed size per frame. Know what I mean? ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-06 19:43:37 UTC (in reply to Ed Sutter) @Ed-Sutter Yes. But maybe just a quirk of the camera. There are a few ways to go about fixing this. It looks like the frames are either the correct size or smaller. If that's the case, you could always add "fake" data to the end of the received buffer to make it the correct size in voxl-uvc-server and then write it into the pipe. You could try this and see if that makes the stream work. ## Reply by Ed Sutter · 2022-09-06 20:08:56 UTC (in reply to Eric Katzfey) @Eric-Katzfey Ok, I'll dig into this more. I need to understand it anyway, so I'll look at **libuvc** and **voxl-uvc-server** and let you know what I figure out. ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-06 20:17:05 UTC (in reply to Ed Sutter) @Ed-Sutter Excellent, thanks! ## Reply by Ed Sutter · 2022-09-07 12:11:30 UTC (in reply to Eric Katzfey) @Eric-Katzfey FYI... Using a Logitech BRIO camera seems to work fine... the frame size always returns the same value (614400 in my case). Still, I wanna spend ***some*** time attempting to figure out why the Microsoft HD camera doesn't work as well... ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-07 17:17:07 UTC (in reply to Ed Sutter) @Ed-Sutter We have tested with these cameras: * Logitech C270 * Logitech C930e * Logitech Brio (USB 2.0 and USB 3.0 on VOXL 2) * Videology 5MP Micro USB 2.0 Board Camera * PureThermal Mini with Flir Lepton * Flir Boson 640 * Blue Robotics h264 camera ## Reply by Ed Sutter · 2022-09-07 17:17:25 UTC (in reply to Eric Katzfey) @Eric-Katzfey By the way, regarding "upgrading"... All I did was opkg update; opkg upgrade. I really don't know what all that updated (aside from /etc/modalai/voxl-streamer.conf); but the output of "voxl-version" didn't change at all. Based on that it appears I'm running with a system-image that was built on 2021-06-06 (about 15 months ago). Is that the most recent? ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-07 17:19:42 UTC (in reply to Ed Sutter) @Ed-Sutter We are preparing a new VOXL platform release but are still completing an update of our ToF camera support. Once that is complete the release will go out. ## Reply by Ed Sutter · 2022-09-07 17:20:29 UTC (in reply to Eric Katzfey) @Eric-Katzfey Ok, no rush (just curious)... Is that updated by the same procedure (opkg update; opkg upgrade)? ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-07 17:22:20 UTC (in reply to Ed Sutter) @Ed-Sutter No, there is a bash script to handle it. The opkg method is good for SDK and package updates but the full platform release also contains a new system image that needs to be flashed via fastboot. ## Reply by Ed Sutter · 2022-09-07 19:08:15 UTC (in reply to Eric Katzfey) @Eric-Katzfey Update regarding the intermittent frame size when using the MS-1080P-HD camera... I pulled down the latest libuvc code from github and built it on my Ubuntu 18.04 machine. I was then able to run uvc_test on that camera and it always came back with the same (correct) frame size. Is there any chance that the libuvc version you guys have on voxl-public is a bit out-of-date? ## Reply by Ed Sutter · 2022-09-07 19:52:51 UTC (in reply to Eric Katzfey) @Eric-Katzfey Just to gather a little more data, I took the voxl-libuvc code and built it on Ubuntu18.04. Then I ran uvc_test with the MS-1080P-HD camera and it worked fine. To do this I cloned [libuvc](https://github.com/libuvc/libuvc) and voxl-libuvc, then prior to building libuvc, I copied all the source files (*.[ch]) from voxl-libuvc to libuvc. This allowed me to use voxl-libuvc source code, but still build on a native ubuntu host. Anyway, that eliminates voxl-libuvc from the suspect list. Since voxl-uvc-server's callback is fed the frame size, that kinda points to the driver as a possible suspect. Any thoughts? If interested, I can post the script I used to do that build. ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-07 20:21:39 UTC (in reply to Ed Sutter) @Ed-Sutter What is uvc_test? ## Reply by Ed Sutter · 2022-09-07 20:53:20 UTC (in reply to Eric Katzfey) @Eric-Katzfey Its the output of building test.c which is part of the libuvc code. It just uses libuvc to grab frames and they are pushed to the console with OpenCV. ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-07 21:39:21 UTC (in reply to Ed Sutter) @Ed-Sutter It looks like it prints the length each frame. Do you see it show same frame length each time or does it ever change? ## Reply by Ed Sutter · 2022-09-07 21:40:50 UTC (in reply to Eric Katzfey) @Eric-Katzfey Yes, it does print every frame, and it is consistently (i.e. always) 614400 in my case. ## Reply by Ed Sutter · 2022-09-08 20:26:20 UTC (in reply to Eric Katzfey) @Eric-Katzfey I modified the voxl-libuvc/build.sh script so that it would build the uvc_test (from test.c) program I mentioned earlier. Refer to the [README.md](https://github.com/libuvc/libuvc#getting-and-building-libuvc) file in the original libuvc code)... cmake ../libuvc -DBUILD_TEST=ON -DBUILD_EXAMPLE=ON The test.c code uses opencv to display to a monitor, so I also stubbed that code out so that the callback simply prints the received frame size. With those changes I was able to install the uvc_test program on my Voxl Flight Deck and I see the same inconsistent frame size. That eliminates any suspicion of a problem with voxl-uvc-server code; and since this works correctly on a standard x86 linux machine, this points to the driver. Are there any open issues with this that you are aware of? ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-08 21:30:39 UTC (in reply to Ed Sutter) @Ed-Sutter Thanks for the testing! No, I am not aware of any issues. ## Reply by Ed Sutter · 2022-09-15 13:38:48 UTC (in reply to Eric Katzfey) @Eric-Katzfey Just an FYI... I have been experimenting with libuvc on my host machine and I do notice occasional incorrect incoming frame sizes. So my earlier assumption that this only occurs on VOXL is false. ## Reply by Eric Katzfey (ModalAI staff) · 2022-09-15 14:26:14 UTC (in reply to Ed Sutter) @Ed-Sutter Okay, thanks! That's good to know. And really appreciate the investigation! ## Reply by Anton Gereles · 2022-09-29 16:30:29 UTC (in reply to Eric Katzfey) @Eric-Katzfey so why i bought VOXL Flight Deck month ago with 3.3.0-0.5.0 version? it was difficult for your team to upgrade to new version before sending? wonderful service for 1000$. im 2 weeks trying to get video streaming to qgc. Result - 0. ## Reply by tom (ModalAI staff) · 2022-09-29 17:06:53 UTC @Anton-Gereles If you have a specific question that we can help you with please open a new forum post and we'd be more that happy to help get you going.