how to get H256 hires encoded imgs from sdk 1.0 ?

aashry

@aashry the following is the voxl-camera-server.conf file as well for your reference. Should I change anything here to fix that issue? Please elaborate.

/**
 * voxl-camera-server Configuration File
 *
 * Each camera has configurations for up to 4 HAL3 streams:
 *    - `preview` stream for raw unprocessed images from CV cameras
 *    - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming
 *    - `large_video` 4k (ish) h264/h265 for onboard video recording to disk
 *    - `snapshot` ISP-processed JPG snapshots that get saved to disk
 *
 * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled
 *
 * This file is generated from default values by voxl-configure-cameras.
 * Do not expect arbitrary resolutions to work, the ISP and video compression
 * pipelines only support very specific resolutions.
 *
 * The default video compression mode is cqp or Constant Quantization Parameter
 *
 *
 *
 */
{
	"version":	0.1,
	"cameras":	[{
			"type":	"pmd-tof",
			"name":	"tof",
			"enabled":	true,
			"camera_id":	0,
			"fps":	5,
			"en_preview":	true,
			"preview_width":	224,
			"preview_height":	1557,
			"pre_format":	"tof",
			"ae_mode":	"off",
			"standby_enabled":	false,
			"decimator":	5
		}, {
			"type":	"imx214",
			"name":	"hires",
			"enabled":	true,
			"camera_id":	2,
			"fps":	30,
			"en_preview":	false,
			"preview_width":	640,
			"preview_height":	480,
			"pre_format":	"nv21",
			"en_small_video":	true,
			"small_video_width":	1024,
			"small_video_height":	768,
			"small_venc_mode":	"h265",
			"small_venc_br_ctrl":	"cqp",
			"small_venc_Qfixed":	30,
			"small_venc_Qmin":	15,
			"small_venc_Qmax":	40,
			"small_venc_nPframes":	9,
			"small_venc_mbps":	2,
			"en_large_video":	true,
			"large_video_width":	4096,
			"large_video_height":	2160,
			"large_venc_mode":	"h265",
			"large_venc_br_ctrl":	"cqp",
			"large_venc_Qfixed":	38,
			"large_venc_Qmin":	15,
			"large_venc_Qmax":	50,
			"large_venc_nPframes":	29,
			"large_venc_mbps":	30,
			"en_snapshot":	true,
			"en_snapshot_width":	4160,
			"en_snapshot_height":	3120,
			"ae_mode":	"isp"
		}, {
			"type":	"ov7251",
			"name":	"tracking",
			"enabled":	true,
			"camera_id":	1,
			"fps":	30,
			"en_preview":	true,
			"preview_width":	640,
			"preview_height":	480,
			"pre_format":	"raw8",
			"ae_mode":	"lme_msv",
			"ae_desired_msv":	60,
			"ae_filter_alpha":	0.600000023841858,
			"ae_ignore_fraction":	0.20000000298023224,
			"ae_slope":	0.05000000074505806,
			"ae_exposure_period":	1,
			"ae_gain_period":	1
		}]
}

aashry

Also, it seems like voxl-camera-server randomly stops working. Is SDK 1.0 completely unusable so far? or Is there a workaround that we can do?

|           Pipe Name |  bytes  | wide |  hgt |exp(ms)| gain | frame id |latency(ms)|  fps |  mbps  t
|   hires_large_color | Server Disconnected
| hires_large_encoded | Server DisconnectedMost likely the server stopped without cleaning up
Client is cleaning up pipes for the serverdMost likely the server stopped without cleaning up
ERROR in pipe_client_init_channel opening request pipe: No such device or addressing request pipe: Ns
| hires_small_encoded | Server DisconnectedERROR in pipe_client_init_channel opening request pipe: Ns
|    hires_small_grey | Server DisconnectedClient is cleaning up pipes for the server
ERROR in pipe_client_init_channel opening request pipe: No such device or address cleaning up
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the serverdMost likely the server stopped without cleaning up
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning upn pipe_client_init_channel opening request pipe: Ns
Client is cleaning up pipes for the serveraning up
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the server
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the server
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the server
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the server

aashry

@aashry

Edit1: Everything above applies to the h264 encoding as well, I haven't been able to view it in the portal and it has the same problem with the ros node.

aashry

Anyone can help?

wilkinsaf

@aashry What UAV are you flying? Is this an off the shelf one from modalai or did you assemble it yourself?

Usually that error means there is a hardware issue with the cameras

Bernhard

Hi @aashry, did you actually solve this problem?
I am running into the same issue.

Camera config is alright, stream config seems to be right as well, but I cannot get an encoded stream in either voxl-portal or vlc.

Any help would be greatly appreciated.

BR

Zachary Lowell 0

Hi @aashry voxl-camera-server with H265 support was implemented and included in an SDK past 1.0 - any reason why you are not upgrading to 1.3.3?

The other option is you mix and match versions of voxl-camera-server with your current SDK version, but odds are that will have quite a bit of conflicts as it is dependent on new versions of modalpipe and such.

Zach

Zachary Lowell 0

Also @Bernhard - you will never be able to view the encoded stream on voxl-portal, just on VLC or something like QGC. SDK 1.0 does support h264 which is software encoding though so innately it will be slower - does this work out the gate for you?

Alex Kushleyev

@Bernhard , you may be missing a step, if you want to stream the encoded stream using RTSP protocol, please take a look at voxl-streamer, which is our standard package : https://docs.modalai.com/voxl-streamer/

voxl-streamer can perform the following functions:

receive unencoded yuv (or raw8 monochrome) images from camera server and encode them using VOXL2's hardware encoder (using gstreamer + OMX api) and serve the encoded video as rtsp service
receive encoded video stream from camera server and serve the encoded video directly

To play the h264 / h265 stream you can use VLC as Zach mentioned, or ffmpeg / ffplay:

ffplay rtsp://<board-ip>:port/live

port number is printed when you run voxl-streamer (port can be modified with a command line argument)

Alex

Bernhard

@Alex-Kushleyev I have tried using VLC as well, that does not work.
voxl-streamer is configured correctly (I think):

"input-pipe":   "hires_small_encoded",
"bitrate":      1500000,
"rotation":     0,
"decimator":    1,
"port": 8900

But when I try to connect via rtsp://<IP>:8900/live nothing happens.

Alex Kushleyev

@Bernhard , please follow these steps:

make sure voxl-camera-server is running either in the background service or manually started in a terminal
confirm that hires_small_encoded stream is being generated by the camera server:
- voxl-inspect-cam hires_small_encoded should show stream statistics like FPS, frame resolution and size
- if you do not see any stats using the inspect-cam tool, make sure the small encoded stream is enabled in your /etc/modalai/voxl-camera-server.conf
assuming the encoded stream is working, launch voxl-streamer:
- voxl-streamer -i hires_small_encoded
- it should also tell you the URL to connect to, although often the ip in the url is local (127.0.0.1) so you need to replace that with actual network IP address

Alex

Bernhard

@Alex-Kushleyev thanks - I think the voxl-streamer was enabled, but not running.
Problem solved, thank you!