USB camera live streaming through Microhard carrier board

[Jetson Nano]

@Eric-Katzfey
I did not try streaming over wifi, i only try using Microhard.

The following arr log files of lepton.

This is the log file while trying to recreate the scenario. I am not able to connect to the VLC through rtsp but the rtsp stream is visible through ffplay, I attaching that log too.

the following are the log files of rtsp stream and connecting through vlc only.

voxl2:/$ voxl-streamer --standalone --port 8904 -i uvc

Waiting for pipe uvc to appear

Found Pipe

detected following stats from pipe:

w: 160 h: 120 fps: 9 format: YUV422_UYVY

Stream available at rtsp://127.0.0.1:8904/live

A new client rtsp://192.168.168.150:46960(null) has connected, total clients: 1

Camera server Connected

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

gbm_create_device(156): Info: backend name is: msm_drm

A new client rtsp://192.168.168.150:46976(null) has connected, total clients: 2

rtsp client disconnected, total clients: 1

rtsp client disconnected, total clients: 0

no more rtsp clients, closing source pipe intentionally 

this is the log file of when I ran first using ffplay, then tried with vlc. that when the error again was observed.

voxl2:/$ voxl-streamer --standalone --port 8901 -i uvc
Waiting for pipe uvc to appear
Found Pipe
detected following stats from pipe:
w: 160 h: 120 fps: 9 format: YUV422_UYVY
Stream available at rtsp://127.0.0.1:8901/live
A new client rtsp://192.168.168.150:60200(null) has connected, total clients: 1
Camera server Connected
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
A new client rtsp://192.168.168.150:60210(null) has connected, total clients: 2
rtsp client disconnected, total clients: 1
ERROR:   New frame rejected, status = -2
rtsp client disconnected, total clients: 0
no more rtsp clients, closing source pipe intentionally
Removed 1 sessions
A new client rtsp://192.168.168.150:49562(null) has connected, total clients: 1
Camera server Connected
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
A new client rtsp://192.168.168.150:49570(null) has connected, total clients: 2
rtsp client disconnected, total clients: 1

(voxl-streamer:28035): GLib-GObject-WARNING **: 16:00:31.725: instance with invalid (NULL) class pointer

(voxl-streamer:28035): GLib-GObject-CRITICAL **: 16:00:31.726: g_signal_emit_by_name: assertion 'G_TYPE_CHECK_INSTANCE (instance)' failed
rtsp client disconnected, total clients: 0
no more rtsp clients, closing source pipe intentionally
A new client rtsp://192.168.168.150:58654(null) has connected, total clients: 1
Camera server Connected
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
A new client rtsp://192.168.168.150:58656(null) has connected, total clients: 2
rtsp client disconnected, total clients: 1
rtsp client disconnected, total clients: 0
no more rtsp clients, closing source pipe intentionally
A new client rtsp://192.168.168.150:45852(null) has connected, total clients: 1
Camera server Connected
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
Removed 1 sessions

t@t:~$ ffplay -rtsp_transport tcp rtsp://192.168.168.100:8901/live 
ffplay version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from 'rtsp://192.168.168.100:8901/live': 0B f=0/0   
  Metadata:
    title           : Session streamed with GStreamer
    comment         : rtsp-server
  Duration: N/A, start: 0.592211, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 160x120, 9 fps, 8.75 tbr, 90k tbn, 18 tbc
t@t:~$ d=   4 aq=    0KB vq=    0KB sq=    0B f=1/1   

this is running without any interruptions

voxl2:/$ voxl-uvc-server  -d
loading config file
Enabling debug messages
=================================================================
width:                            160
height:                           120
fps:                              9
pipe_name:                        uvc
=================================================================
voxl-uvc-server starting
Image resolution 160x120, 9 fps chosen
UVC initialized
Device found
Device opened
uvc_get_stream_ctrl_format_size succeeded for format UYVY
Streaming starting
Got frame callback! frame_format = 4, width = 160, height = 120, length = 640, ptr = (nil)
 * got image 30
 * got image 60
 * got image 90
 * got image 120
 * got image 150
 * got image 180
 * got image 210
 * got image 240
 * got image 270
 * got image 300
 * got image 330
 * got image 360
 * got image 390
 * got image 420
 * got image 450
 * got image 480
 * got image 510
 * got image 540
 * got image 570
 * got image 600
 * got image 630
 * got image 660
 * got image 690
 * got image 720
 * got image 750
 * got image 780
 * got image 810

[Jetson Nano]

@eric-katzfey
I tried with c270 camera, the observation was similar was not able to stream in vlc, but was able to stream through ffplay.

[Eric Katzfey]

I'm confused by all of the new data. Can you summarize? You have tested with 2 different USB cameras: Lepton and c270. You have tested with 3 viewers: QGC, VLC, and ffplay. You have tested with 2 applications: voxl-uvc-server and voxl-streamer. Only one network connection: Microhard. Can you also try a MIPI camera instead of a USB camera? Then can you create a matrix saying which ones worked and which ones didn't?

[Jetson Nano]

@Eric-Katzfey
sure I can summarize,

Camera 1 - Lepton,
QGC - Did not test
VLC - Did not stream, error message.
ffplay - stream worked

Camera 2 - logitech c270 camera
QGC - Did not test
VLC - Did not stream, error message.
ffplay - stream worked

I did not understand what you meant by mipi camera, if it is normal hires camera connected to the yellow board, it is streaming fine, slightly slow through the microhard connection, but I can see the video stream on the portal.

the following message showed up when i tried to trigged the rtsp stream through vlc

A new client rtsp://192.168.168.150:58656(null) has connected, total clients: 2
rtsp client disconnected, total clients: 1
rtsp client disconnected, total clients: 0
no more rtsp clients, closing source pipe intentionally
A new client rtsp://192.168.168.150:45852(null) has connected, total clients: 1
Camera server Connected
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm

[Eric Katzfey]

MIPI is the physical connection to the VOXL2 board. So, yes, the "normal hires camera" is connected via mipi instead of USB. I'm trying to help you debug this. Normally, when something doesn't work, you try to narrow down the issue by trying some carefully thought out variations to see what works and what doesn't. That can determine the part that is not working. For example, if you have USB camera -> voxl-uvc-server -> voxl-streamer -> Microhard -> VLC and that is showing problems and then you switch to a stream with MIPI camera -> voxl-camera-server -> voxl-streamer -> Microhard -> VLC and that doesn't have problems then it's logical to focus on the USB camera -> voxl-uvc-server part of the pipeline as the problematic part. In your first post you mention QGC stream works and then in your last reply you said you did not test with QGC so that is confusing. And I'm not sure what your end goal is. In the first post you say "Please help to stream live video from the lepton camera through the microhard board." Then you say that it is working from ffplay. So, is that okay now? Or is ffplay not an acceptable solution for you?

[Jetson Nano]

@Eric-Katzfey ,
Please excuse me for the confusion caused by me.

Goal -
my goal is to achieve continuous video stream from lepton camera connected through USB cable. the USB port I am using is Microhard.

Issues I faced -

1. During my initial testing, I was able to stream the lepton video through VLC, but the camera stream kept on getting interrupted (the camera stream ended), once I restart the uvc-service and the stream-service , it starts again. This observation was in loop.
   I wanted your help to solve this issue.
2. Once you asked for the log file, I tried to replicate this test, but the stream was not working through VLC.
   I attached the log file.
3. To verify the camera is working fine I tried with ffplay, the stream was working fine.
4. I verified the MIPI camera, I was able to see the video on the voxl-portal, but while using the stream-service, it was not working on VLC.

Now, I would like your help to solve the following problems

1. Get the video stream on VLC
2. Get the video stream to work continuously with out any interruption.
3. Also it would be great if you clear one more query of mine, whether I can setup the USB port with any camera(which supports USB protocol), If I make necessary changes to the codes or UVC-service and Stream-service.
   Thank you for your patience and looking forward to response.

[Eric Katzfey]

That's kind of pointing to VLC as the issue. Can you try with QGC?

[Jetson Nano]

@Eric-Katzfey
Hi, Sorry for the late reply.
I tested the rtsp with QGC. The stream was working fine, until suddenly the stream got stuck and I had to restart the voxl-stream command.

• I have one major question, does usb camera support continuous live video stream.
• I am planning to purchase the voxl2 mini and use the usb port for lepton, I just want to be sure that the port can support continuous rtsp stream without having any issues.
• Is there any setup, I should be knowing for seamless streaming other than what is mentioned in the docs.

thank you for time, looking forward to your response.

[Eric Katzfey]

Yes, we have tested a few USB (UVC) cameras and they can continuously live stream. So we need to figure out what is causing the problem on your setup.

[Zachary Lowell 0]

@jetson-nano I am testing this right now and am not seeing an issue - can you describe the following:

1. How long is the voxl on when you see this issue/how long has the stream been going?
2. Have you confirmed that voxl uvc camera library did not die and that is the cause?
3. Have you confirmed via the voxl streamer logs that the service did not die?

I am trying to understand which library is the source of the error in the mpa.

Thanks!