High res camera latency

[bschulzhf]

voxl-inspect-cam -a shows that the hires_down_large_color latency is >900ms. What is the cause of the latency and can it be reduced?

Thanks!

[bschulzhf]

As an update, I was able to get the latency down to about 50-60ms by making the following changes to /etc/modalai/voxl-camera-server.conf. Are there any other optimizations I can make?

{
"type": "imx412",
"name": "hires_down",
"enabled": true,
"camera_id": 2,
"fps": 10,
"en_preview": true,
"preview_width": 4056,
"preview_height": 3040,
"en_raw_preview": false,
"en_small_video": false,
"small_video_width": 1024,
"small_video_height": 768,
"small_venc_mode": "h264",
"small_venc_br_ctrl": "cqp",
"small_venc_Qfixed": 30,
"small_venc_Qmin": 15,
"small_venc_Qmax": 40,
"small_venc_nPframes": 9,
"small_venc_mbps": 2,
"small_venc_osd": false,
"en_large_video": false,
"large_video_width": 4056,
"large_video_height": 3040,
"large_venc_mode": "h264",
"large_venc_br_ctrl": "cqp",
"large_venc_Qfixed": 40,
"large_venc_Qmin": 15,
"large_venc_Qmax": 50,
"large_venc_nPframes": 29,
"large_venc_mbps": 40,
"large_venc_osd": false,
"en_snapshot": false,
"en_snapshot_width": 4056,
"en_snapshot_height": 3040,
"exif_focal_length": 3.0999999046325684,
"exif_focal_length_in_35mm_format": 17,
"exif_fnumber": 1.2400000095367432,
"snapshot_jpeg_quality": 75,
"ae_mode": "lme_msv",
"gain_min": 54,
"gain_max": 8000,
"ae_desired_msv": 60,
"exposure_min_us": 20,
"exposure_max_us": 33000,
"exposure_soft_min_us": 5000,
"ae_filter_alpha": 0.600000023841858,
"ae_ignore_fraction": 0.20000000298023224,
"ae_slope": 0.05000000074505806,
"ae_exposure_period": 1,
"ae_gain_period": 1
}

[Alex Kushleyev]

Hi @bschulzhf ,

voxl-inspect-cam -a command should not be generally used as it puts a lot of stress on the system. just use it to inspect the stream that you need, such as voxl-inspect-cam hires_down_large_color.

Also, in your camera config changes, it is not clear exactly what changes you actually made, since you did not provide a diff (but i can see that you probably disabled all the streams except for the preview stream). In any case, you should revert your changes and just inspect the stream you need to estimate the latency.

If you want to provide more information about your application, we can suggest appropriate camera server parameter changes. I do see that ae_desired_msv": 60 is potentially too dark (the target average pixel value for auto exposure control) - you can try increasing it and see if the image is too dark.

Alex