USB camera live streaming through Microhard carrier board

[Jetson Nano]

Hi @alex-kushleyev @vinny @eric-katzfey
Hardware: (voxl 2 , microhard board, with lepton connected through usb port on the carrier board)

Software version: (voxl sdk 1.5.0)

What I am trying to do: to stream video through usb connection, using the UVC stream service on voxl 2 . The lepton stream through SPI was low quality compared to the usb connection.

What actually happens: The stream was successful , with good quality video stream, which I was able to see in the QGC. The problem i was facing was the irregular stopping of the streaming. I had to restart the service multiple times to see the video.

If there is something I am missing to get a regular continuous live streaming through microhard board , please do share. I want to experiment with different cameras before finalising the components, the USB connection helps me to do that.
Please help to stream live video from the lepton camera through the microhard board.

Steps to reproduce: I followed normal Microhard connection steps , along with setting up the UVC stream service for the Lepton camera, I used the groups gets usb board. The camera was detected and the stream started, after a continuous stream of 25 to 30 seconds the streaming stops, and behavior was observed mulitple times.

[Eric Katzfey]

Have you attempted any trouble shooting? What if you stream a connected camera with voxl-streamer, does it still do the same thing? Did you try streaming over WiFi or Ethernet? Does the voxl-uvc-server generate any error or diagnostic messages? Can you try a different USB camera (non-Lepton)? I'd recommend trying with a Logitech C270.

[Jetson Nano]

@Eric-Katzfey
I did not try streaming over wifi, i only try using Microhard.

The following arr log files of lepton.

This is the log file while trying to recreate the scenario. I am not able to connect to the VLC through rtsp but the rtsp stream is visible through ffplay, I attaching that log too.

the following are the log files of rtsp stream and connecting through vlc only.

voxl2:/$ voxl-streamer --standalone --port 8904 -i uvc

Waiting for pipe uvc to appear

Found Pipe

detected following stats from pipe:

w: 160 h: 120 fps: 9 format: YUV422_UYVY

Stream available at rtsp://127.0.0.1:8904/live

A new client rtsp://192.168.168.150:46960(null) has connected, total clients: 1

Camera server Connected

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

A new client rtsp://192.168.168.150:46976(null) has connected, total clients: 2

rtsp client disconnected, total clients: 1

rtsp client disconnected, total clients: 0

no more rtsp clients, closing source pipe intentionally 

this is the log file of when I ran first using ffplay, then tried with vlc. that when the error again was observed.

voxl2:/$ voxl-streamer --standalone --port 8901 -i uvc
Waiting for pipe uvc to appear
Found Pipe
detected following stats from pipe:
w: 160 h: 120 fps: 9 format: YUV422_UYVY
Stream available at rtsp://127.0.0.1:8901/live
A new client rtsp://192.168.168.150:60200(null) has connected, total clients: 1
Camera server Connected
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
A new client rtsp://192.168.168.150:60210(null) has connected, total clients: 2
rtsp client disconnected, total clients: 1
ERROR:   New frame rejected, status = -2
rtsp client disconnected, total clients: 0
no more rtsp clients, closing source pipe intentionally
Removed 1 sessions
A new client rtsp://192.168.168.150:49562(null) has connected, total clients: 1
Camera server Connected
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
A new client rtsp://192.168.168.150:49570(null) has connected, total clients: 2
rtsp client disconnected, total clients: 1

(voxl-streamer:28035): GLib-GObject-WARNING **: 16:00:31.725: instance with invalid (NULL) class pointer

(voxl-streamer:28035): GLib-GObject-CRITICAL **: 16:00:31.726: g_signal_emit_by_name: assertion 'G_TYPE_CHECK_INSTANCE (instance)' failed
rtsp client disconnected, total clients: 0
no more rtsp clients, closing source pipe intentionally
A new client rtsp://192.168.168.150:58654(null) has connected, total clients: 1
Camera server Connected
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
A new client rtsp://192.168.168.150:58656(null) has connected, total clients: 2
rtsp client disconnected, total clients: 1
rtsp client disconnected, total clients: 0
no more rtsp clients, closing source pipe intentionally
A new client rtsp://192.168.168.150:45852(null) has connected, total clients: 1
Camera server Connected
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
Removed 1 sessions

t@t:~$ ffplay -rtsp_transport tcp rtsp://192.168.168.100:8901/live 
ffplay version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from 'rtsp://192.168.168.100:8901/live': 0B f=0/0   
  Metadata:
    title           : Session streamed with GStreamer
    comment         : rtsp-server
  Duration: N/A, start: 0.592211, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 160x120, 9 fps, 8.75 tbr, 90k tbn, 18 tbc
t@t:~$ d=   4 aq=    0KB vq=    0KB sq=    0B f=1/1   

this is running without any interruptions

voxl2:/$ voxl-uvc-server  -d
loading config file
Enabling debug messages
=================================================================
width:                            160
height:                           120
fps:                              9
pipe_name:                        uvc
=================================================================
voxl-uvc-server starting
Image resolution 160x120, 9 fps chosen
UVC initialized
Device found
Device opened
uvc_get_stream_ctrl_format_size succeeded for format UYVY
Streaming starting
Got frame callback! frame_format = 4, width = 160, height = 120, length = 640, ptr = (nil)
 * got image 30
 * got image 60
 * got image 90
 * got image 120
 * got image 150
 * got image 180
 * got image 210
 * got image 240
 * got image 270
 * got image 300
 * got image 330
 * got image 360
 * got image 390
 * got image 420
 * got image 450
 * got image 480
 * got image 510
 * got image 540
 * got image 570
 * got image 600
 * got image 630
 * got image 660
 * got image 690
 * got image 720
 * got image 750
 * got image 780
 * got image 810

[Jetson Nano]

@eric-katzfey
I tried with c270 camera, the observation was similar was not able to stream in vlc, but was able to stream through ffplay.

[Eric Katzfey]

I'm confused by all of the new data. Can you summarize? You have tested with 2 different USB cameras: Lepton and c270. You have tested with 3 viewers: QGC, VLC, and ffplay. You have tested with 2 applications: voxl-uvc-server and voxl-streamer. Only one network connection: Microhard. Can you also try a MIPI camera instead of a USB camera? Then can you create a matrix saying which ones worked and which ones didn't?