# Streaming 4k video from VOXL, Source: https://forum.modalai.com/topic/728/streaming-4k-video-from-voxl Category: Flight Core and Legacy VOXL (https://forum.modalai.com/category/9/flight-core-and-legacy-voxl) Tags: voxl Posted: 2022-02-09 23:05:12 UTC by SmittyHalibut Replies: 4 · Views: 1367 ## SmittyHalibut · 2022-02-09 23:05:12 UTC While working on [my H.265 problem](https://forum.modalai.com/topic/712/streaming-h-265-hevc-from-hires-camera-on-voxl/16), I wanted to confirm that I could at least get 4k video off VOXL in the already supported modes. This is what I've done: * Configured `voxl-camera-server`'s `preview` profile for 3840x2160, 30fps, `nv21` format. All defaults, except increase the resolution. * Configured `voxl-streamer`'s `hires` profile for 3840x2160, 50Mbps bitrate, everything else defaults. * Restarted `voxl-camera-server` and started `voxl-streamer`. * `voxl-inspect-cam` confirms `voxl-camera-server` is sending 3840x2160 at 29.9fps, `NV21` format. * With no clients, `top` claims 50% idle CPU, and no errors on `voxl-streamer` console. As soon as I connect with VLC, I start getting errors on `voxl-streamer` console (see below), the CPU jumps to ~1% idle (about 50% user, 45% system, and a few percent in Idle, IRQ, SIRQ, etc.) with `voxl-streamer` taking about 65 to 70% of total CPU, `mm-qcamera-daemon` and `voxl-camera-server` are taking up most of the rest of it. Frame rate in VLC is a frame every few seconds. Video is often blocky (like missed frames). The "Statistics" tab of VLC is showing Lost frames, Decoded and Displayed don't match, etc. In short, VOXL isn't keeping up with 4K Video, in the Default configs. Not by a long shot. Is there something else I need to do to get 4K video streamed off VOXL? Or do something differently? Thanks for your help! -Mark ## Reply by Chad Sweet (ModalAI staff) · 2022-02-10 15:48:07 UTC You'll need to isolate a bad network first. 50Mbps is really high. Typical debugging for this would be to start at a low bitrate and work up. 4k should work fine ## Reply by Eric Katzfey (ModalAI staff) · 2022-02-10 17:28:01 UTC @SmittyHalibut Best to move everything up slowly. And one thing at a time. For example, move to 720p and see how the system is performing. Then increase bitrate to say 5Mbps and see how things are going. Another really nice way to debug things with voxl-streamer is to use the video-test configuration. That takes the camera and MPA part out of the picture and let's you just focus on resolution, frame rate, bit rate, etc. Then add in the camera after everything looks solid with video-test. ## Reply by SmittyHalibut · 2022-02-11 19:43:12 UTC It looks like my VOXL is over-heating and down-clocking. I've stripped the config down to the following: * `voxl-streamer` only. `voxl-camera-server` is stopped. * `voxl-streamer` configured to use `video-test` at 3840x2160x30fps with a bit rate of 1Mbps (I also tried 5Mbps and 10Mbps; the problems mentioned below were the same.) ``` "video-test": { "input": { "interface": "test", "frame": { "width": 3840, "height": 2160, "format": "uyvy" } }, "output": { "stream": { "rotation": 0, "width": 3840, "height": 2160, "rate": 30, "bitrate": 1000000 } } ``` Observed results with one stream going: * The video stream is still choppy (and very blocky at 1Mbps, as expected), lots of dropped frames. * `voxl-streamer` is consumping about 75% total CPU. `thermal-engine` is about 5%, and the rest of "other" and about 12.5% idle. * CPU consumption is 100% on CPU2 and CPU3, and about 50% on CPU0 and CPU1. ``` Mem: 1182424K used, 2678896K free, 17100K shrd, 16348K buff, 215972K cached CPU0: 45.9% usr 21.1% sys 0.0% nic 29.2% idle 0.0% io 2.5% irq 1.1% sirq CPU1: 66.0% usr 12.8% sys 0.0% nic 19.2% idle 0.0% io 1.2% irq 0.5% sirq CPU2: 99.4% usr 0.1% sys 0.0% nic 0.0% idle 0.0% io 0.0% irq 0.3% sirq CPU3: 98.5% usr 0.5% sys 0.0% nic 0.1% idle 0.0% io 0.7% irq 0.0% sirq Load average: 5.98 4.01 2.74 5/591 3516 PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND 3491 3228 root S 781m 20.6 0 76.8 voxl-streamer -c video-test 1995 1 root S < 2154m 56.8 0 5.6 /usr/bin/thermal-engine 2378 2 root SW 0 0.0 0 1.0 [msm_thermal:fre] 188 2 root SW 0 0.0 1 0.5 [kgsl_worker_thr] 249 2 root SW 0 0.0 2 0.4 [kworker/u8:6] 2212 1 root S 18872 0.4 0 0.3 /usr/bin/voxl-cpu-monitor ``` * `voxl-cpu-monitor` is warning of overheating, and slow clock rates: ``` Name Freq (MHz) Temp (C) Util (%) ----------------------------------- cpu0 307.2 81.6 12.72 cpu1 307.2 81.6 15.26 cpu2 1248.0 92.5 58.04 cpu3 1248.0 93.8 58.04 Total 93.8 36.01 10s avg 35.95 ----------------------------------- GPU 0.0 74.6 0.00 GPU 10s avg 0.00 ----------------------------------- memory temp: 77.1 C memory used: 1156/3770 MB ----------------------------------- Flags CPU freq scaling mode: auto GPU freq scaling mode: auto CPU OVERHEAT WARNING ----------------------------------- ``` And this is AFTER putting a heat-sink on the CPU/RAM stack (An unused copper RPi sink I had laying about.) * Also note the GPU is at 0%. Is the hardware H.264 encoder in the GPU, or part of the CPU? Your suggestion of lowering the resolution and working my way up doesn't make sense, if "4k should work fine." Unless the resolution at which it starts working reliably will give you some hint at what the problem is. Other than lowering the resolution, I don't know how to scale this back any more to get it to work correctly. Can anyone else see something I'm doing wrong? It really looks like its trying to encode in software. Is there an OMX daemon I need to start, or library to install, or something to get it to use the hardware encoder? Here's a FLIR I took to confirm which chip was getting hot, and where the heat sink was needed. I think it's interesting to see that the heat is obviously only half the chip, showing that only two of the four CPUs were being stressed. ![FLIR of VOXL 2022-02-11.jpeg](https://forum.modalai.com/assets/uploads/files/1644608371498-flir-of-voxl-2022-02-11.jpeg) ## Reply by SmittyHalibut · 2022-02-11 19:50:56 UTC Switching tack: I've configured `voxl-camera-server` to encode H.265 and write to `/data/misc/camera`: ``` Mem: 1396848K used, 2464472K free, 17096K shrd, 16692K buff, 318292K cached CPU0: 32.2% usr 21.0% sys 0.0% nic 42.8% idle 0.2% io 2.1% irq 1.4% sirq CPU1: 22.3% usr 29.8% sys 0.0% nic 46.5% idle 0.0% io 0.8% irq 0.4% sirq CPU2: 53.7% usr 1.2% sys 0.0% nic 44.5% idle 0.0% io 0.2% irq 0.2% sirq CPU3: 50.6% usr 0.4% sys 0.0% nic 48.7% idle 0.0% io 0.2% irq 0.0% sirq Load average: 5.62 4.10 3.21 3/582 3801 PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND 3600 1 root S 850m 22.4 0 30.1 /usr/bin/voxl-camera-server 1966 1 root S 665m 17.5 0 19.6 /system/bin/mm-qcamera-daemon 2321 1 root S 3988 0.1 1 0.9 /sbin/leprop-service 2159 1 system S 372m 9.8 1 0.7 /sbin/logd 7 2 root SW 0 0.0 1 0.4 [rcu_preempt] 3445 2 root SW 0 0.0 1 0.3 [kworker/u8:14] ``` And the CPU is in good shape: ``` Name Freq (MHz) Temp (C) Util (%) ----------------------------------- cpu0 480.0 66.3 25.86 cpu1 480.0 65.6 22.50 cpu2 2150.4 70.1 81.82 cpu3 2150.4 66.9 22.68 Total 70.1 38.21 10s avg 38.38 ----------------------------------- GPU 0.0 60.3 0.00 GPU 10s avg 0.00 ----------------------------------- memory temp: 62.1 C memory used: 1376/3770 MB ----------------------------------- Flags CPU freq scaling mode: auto GPU freq scaling mode: auto ----------------------------------- ``` And this is `voxl-camera-server` encoding H.265, at 3840x2160x30fps, writing to flash. So either `voxl-streamer` is doing software encoding very poorly, or something about the network stack is VERY broken. I'm inclined to think the former.