streamer default h265
-
@Alex-Kushleyev Thanks for getting back. Here's our fork of voxl-streamer and the specific commit
As you can see, not much going on here. I just noticed how the 265 omx encoder object would be created but just unused when encoding a raw stream (using hires_small_grey in my tests which is RAW8).
When I try to connect to the stream, streamer gives the following
voxl2-mini:~$ voxl-streamer -i hires_small_grey -s -b 2000000 -p 554 Waiting for pipe hires_small_grey to appear Found Pipe detected following stats from pipe: w: 1024 h: 768 fps: 30 format: RAW8 Stream available at rtsp://127.0.0.1:554/live A new client rtsp://169.254.4.2:40732(null) has connected, total clients: 1 Camera server Connected gbm_create_device(156): Info: backend name is: msm_drm rtsp client disconnected, total clients: 0 no more rtsp clients, closing source pipe intentionallyand using ffmpeg to connect gives the following
ffplay version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2003-2021 the FFmpeg developers built with gcc 11 (Ubuntu 11.2.0-19ubuntu1) configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 [rtsp @ 0x74e358000cc0] method DESCRIBE failed: 503 Service Unavailable rtsp://169.254.4.1:554/live: Server returned 5XX Server Error reply -
@Alex-Kushleyev it's probably just my gstreamer ignorance and it not being as simple to set up an h265 stream as swapping our h264 with h265
-
@bendraper , you got 90% there!
I just tested your version and was able to reproduce the issue, but the following change fixes it:
from
"profile", G_TYPE_STRING, "baseline",to
"profile", G_TYPE_STRING, "main",There is no
baselineprofile in h265. Please try it - i was able to stream h265 with modified voxl-streamer and play usingffplay.full diff below. We will test more and probably mainline this soon.
Alex
diff --git a/src/pipeline.c b/src/pipeline.c index 53f47b7..5f88315 100644 --- a/src/pipeline.c +++ b/src/pipeline.c @@ -398,9 +398,9 @@ GstElement *create_custom_element(GstRTSPMediaFactory *factory, const GstRTSPUrl // g_object_set(context->h264_parser, "alignment", "nal", NULL); // Configure the OMX encoder - g_object_set(context->omx_encoder, "control-rate", 1, + g_object_set(context->omx_h265_encoder, "control-rate", 1, "interval-intraframes", 30, NULL); - g_object_set(context->omx_encoder, "target-bitrate", + g_object_set(context->omx_h265_encoder, "target-bitrate", context->output_stream_bitrate, NULL); // Configure the RTP input queue @@ -443,10 +443,10 @@ GstElement *create_custom_element(GstRTSPMediaFactory *factory, const GstRTSPUrl gst_caps_unref(filtercaps); // Configure the caps filter to reflect the output of the OMX encoder - filtercaps = gst_caps_new_simple("video/x-h264", + filtercaps = gst_caps_new_simple("video/x-h265", "width", G_TYPE_INT, context->output_stream_width, "height", G_TYPE_INT, context->output_stream_height, - "profile", G_TYPE_STRING, "baseline", + "profile", G_TYPE_STRING, "main", NULL); } if ( ! filtercaps) { @@ -482,10 +482,10 @@ GstElement *create_custom_element(GstRTSPMediaFactory *factory, const GstRTSPUrl context->video_rotate, context->video_rotate_filter, context->encoder_queue, - context->omx_encoder, + context->omx_h265_encoder, context->rtp_filter, context->rtp_queue, - context->rtp_payload, + context->rtp_h265_payload, NULL); } @@ -536,10 +536,10 @@ GstElement *create_custom_element(GstRTSPMediaFactory *factory, const GstRTSPUrl last_element = context->video_rotate_filter; if ( ! gst_element_link_many(last_element, context->encoder_queue, - context->omx_encoder, + context->omx_h265_encoder, context->rtp_filter, context->rtp_queue, - context->rtp_payload, + context->rtp_h265_payload, NULL)) { M_ERROR("Couldn't finish pipeline linking part 4\n"); return NULL; -
@Alex-Kushleyev Incredible, it works! Thanks Alex
-
@bendraper , great! Thank you for doing most of the work
I will add an option to specify output encoder type (for the unencoded input case) so h264/5 can be selected via command line argument.
Alex
-
@Alex-Kushleyev Awesome. On a different note, do you know why ffmpeg has no issue with the aspect ratio of the stream but in QGroundControl, the stream seems to be vertically squished? I would guess ffmpeg correctly makes some assumptions about the aspect ratio that voxl-streamer doesn't outwardly declare and maybe QGC doesn't make these assumptions?
-
@bendraper , i believe that when the encoder starts producing frames, the first frame is a header frame that has the stream description. This condition is also triggered if client disconnects (encoding stops) and then reconnects, causing the stream to start again. I am not sure what the whole pipeline looks like for getting video to QGC, but if the header is not sent over or is not parsed, this can lead to incorrect stream dimension? However i would assume that is the header is lost, the video would be completely garbled, not just aspect ratio, which probably means that QGC is in fact decoding correctly but simply rendering the image using a fixed aspect ratio…
-
I just pushed two updates to a branch of voxl-streamer:
- https://gitlab.com/voxl-public/voxl-sdk/services/voxl-streamer/-/commits/enable-h265-encode
- add option to use h265 encoder for non-encoded input frames (use either command line option
-e h265or--encoder h265or invoxl-streamer.conffile"encoder": "h264" - switch the h264 profile from
baselinetomainwhich allows usingCABACcoding, which has better compression properties than default CAVLC at some expense of HW encoder usage (negligible) and potential additional cpu usage on decoding side. However, this will definitely be better for low bandwidth streaming. h265 uses CABAC coding as the only option.
It seems like it's working well, but not much testing has been done so far..
Alex
-
@Alex-Kushleyev This is great, much appreciated! Also regarding that squished stream in QGC, seems to only be a linux thing so probably not much to do about it right now.
-
@bendraper You can also set the aspect ratio of the video in QGroundControls application settings.
-
@Eric-Katzfey Yea I think that functionality may be broken though. If I try to do the inverse aspect ratio, it just kinda shrinks the stream as opposed to stretching it back out
-
@bendraper said in streamer default h265:
@Eric-Katzfey Yea I think that functionality may be broken though. If I try to do the inverse aspect ratio, it just kinda shrinks the stream as opposed to stretching it back out
It's also seemingly fixed in the daily builds of QGC... Oh well
