RTSP streaming issues

SKA

Hi! I need to be able to network stream to VLC with caching set to 0ms. It seems like older SDKs stream just fine with 0ms cache, but SDK 1.6.0 does not.

Using SDK 1.4.0, I'm able to stream with no problems. The voxl-streamer version is 0.7.5, and I also switched over to voxl-streamer 0.8.0 (from SDK1.6) to narrow down the root cause of the problem, but noticed that streaming worked in this configuration too.

Then, I flashed the voxl to SDK 1.6.0, and tried both versions of voxl-streamer (0.7.5 and 0.8.0) and neither of them were able to stream with 0ms cache.

Where should I look to solve this problem? Thanks!

For reference, here's my voxl-streamer.conf:

voxl2-mini-test-fixture (TF-M0104):/$ cat etc/modalai/voxl-streamer.conf
{
        "input-pipe":   "hires_misp_color",
        "bitrate":      3000000,
        "rotation":     0,
        "decimator":    1,
        "encoder":      "h264",
        "port":         8900
}

SKA

Following up on this, @Alex-Kushleyev do you have any initial leads that might help me - thanks!

Alex Kushleyev

Hi @SKA ,

Can you please clarify a few things:

using latest SDK, does rtsp streaming work, if you do NOT specify 0ms cache?
what player are you using for receiving the video? have you tried ffmpeg (if so, what is the full command?)
are you sure that you are actually have the hires_misp_color stream available from the camera server? you can check it using either voxl-portal or using voxl-inspect-cam hires_misp_color.

Alex

SKA

Hey @Alex-Kushleyev! To answer your questions:

Yes, with the latest SDK, RTSP streaming works in VLC as long as the cache is not set to 0 ms. I’ve seen stable playback once the cache is increased (around ~300 ms).
I’m using VLC to test playback, but I also tested using ffmpeg and was able to successfully record video from the RTSP stream even with 0ms cache. This led me to think that frames are being transmitted correctly, but there’s likely a timing / timestamp issue in the RTSP stream that ffmpeg is tolerant of, but causes VLC to freeze. I don't remember the exact command I used but it was similar to:

ffmpeg \
  -rtsp_transport tcp \
  -fflags nobuffer \
  -flags low_delay \
  -an \
  -i rtsp://<voxl-ip>:8900/live \
  -t 10 \
  -c copy \
  out.mp4

Yes, '''hires_misp_color''' has been available when I've tested this. When I set VLC to 0ms cache, I always only receive one frame before the stream freezes.

Alex Kushleyev

Hello @SKA,

Thank you for the details. I can't think of anything right now, but will double check this.

I know that there is one potential issue in voxl-camera-server - the SPS header (which is part of the H264 / H265 stream) is only sent by the video encoder once (at the beginning of the stream). I believe that voxl-streamer will cache the SPS packet (which contains things like resolution, etc) and i think when clients connect to voxl-streamer, then it will send that SPS header to the clients. However, if voxl-streamer does not get that first header, it will not be able to forward it to the clients. This is a guess for now, but could be related to the issue that you are seeing.

I can look into this issue. Can you please let me know how exactly to set up VLC for 0ms cache? I want to make sure that my test is the same. Also, which version of VLC are you using?

Also, i noticed one thing - you are using hires_misp_color stream from camera server, which is not actually encoded, but YUV. Then voxl-streamer will take those YUVs and encode them (using hardware encoder). Is there a reason why you are not using the hires_misp_encoded stream to get H264 directly from camera server and serve it using voxl-streamer without re-encoding? I wonder if in the old SDK you were using _color or _encoded stream?

Alex