IMX664 MISP decoding issue
-
Hi i am currently working with the IMX 664 camera, and i want to use the MISP functionality so i can set the auto white-balance to manual, in order for it to stay constant, and the gamma correction to 1.0.
The images i get from the camera looks to be misdecoded:
it this a known issue, or is there a fix for this?Currently i am running this on the VOXL2 with SDK version 1.4.5, and my voxl-camera-server.conf looks like this:
/** * voxl-camera-server Configuration File * * Each camera has configurations for up to 4 HAL3 streams: * - `preview` stream for raw unprocessed images from CV cameras * - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming * - `large_video` 4k (ish) h264/h265 for onboard video recording to disk * - `snapshot` ISP-processed JPG snapshots that get saved to disk * * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled * * This file is generated from default values by voxl-configure-cameras. * Do not expect arbitrary resolutions to work, the ISP and video compression * pipelines only support very specific resolutions. * * The default video compression mode is cqp or Constant Quantization Parameter * * * */ { "version": 0.1, "fsync_en": true, "fsync_gpio": 109, "cameras": [{ "type": "imx412", "name": "hires", "enabled": true, "camera_id": 0, "fps": 30, "en_preview": true, "preview_width": 3840, "preview_height": 2160, "en_raw_preview": false, "en_small_video": false, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h264", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "small_venc_osd": false, "en_large_video": false, "large_video_width": 4056, "large_video_height": 3040, "large_venc_mode": "h264", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 40, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 40, "large_venc_osd": false, "en_snapshot": false, "en_snapshot_width": 4056, "en_snapshot_height": 3040, "exif_focal_length": 3.0999999046325684, "exif_focal_length_in_35mm_format": 17, "exif_fnumber": 1.2400000095367432, "snapshot_jpeg_quality": 75, "ae_mode": "off", "gain_min": 54, "gain_max": 8000 }, { "type": "imx664", "name": "hires2", "enabled": true, "camera_id": 1, "fps": 30, "en_preview": true, "preview_width": 1344, "preview_height": 760, "en_raw_preview": true, "en_small_video": false, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h264", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "small_venc_osd": false, "en_large_video": false, "large_video_width": 2688, "large_video_height": 1512, "large_venc_mode": "h264", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 40, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 40, "large_venc_osd": false, "en_snapshot": false, "en_snapshot_width": 2688, "en_snapshot_height": 1512, "exif_focal_length": 3.0999999046325684, "exif_focal_length_in_35mm_format": 17, "exif_fnumber": 1.2400000095367432, "snapshot_jpeg_quality": 75, "ae_mode": "off", "gain_min": 54, "gain_max": 8000, "en_misp": true, "misp_width": -1, "misp_height": -1, "misp_venc_enable": false, "misp_venc_mode": "h264", "misp_venc_br_ctrl": "cqp", "misp_venc_Qfixed": 30, "misp_venc_Qmin": 15, "misp_venc_Qmax": 50, "misp_venc_nPframes": 29, "misp_venc_mbps": 2, "misp_venc_osd": false, "misp_awb": "manual", "misp_gamma": 1.0 }] } -
@mark ,
Please try using input resolution of 2704x1540, not the binned one which you have currently selected. Then you can set MISP width and height whatever you need and the output should be properly downscaled and cropped. Choosing higher input resolution will result in a better downscaled image due to debayering artifacts at lower resolution.
The reason why the image decoding does not work has to do with the image line stride of the raw10 buffer not matching the requirement of the opencl debayering functions. In newer versions of camera server, we automatically re-align the image to make it work, but that requires copying the whole bayer image into a different buffer (more cpu overhead).
Using input resolution of 2704x1540 will result in best results in terms of image quality and cpu usage. This resolution works up to 120FPS. Binned mode does not actually provide any advantages (specifically for the IMX664 camera), not even faster read-out or higher max fps or anything like that.
Please try that and let me know if that works for you.
Alex
-
hi @Alex-Kushleyev thanks for the feedback, but it seems like the resolution of 2704x1540 is not supported according to the camera. Selecting the biggest supported resolution from this list (2688x1520) this issue still occurs. The regular preview does seem to work on the resolution 2688x1512, but this format is not supported anymore when i turn on the "en_raw_preview" setting.
DEBUG: Checking Gain limits for Camera: hires2 DEBUG: Using gain limits min: 54 max: 8000 DEBUG: cam ID 1 checking for fmt: 37 w: 2704 h: 1540 o: 0 DEBUG: ==> option 0: format= 34 width=2688 height=1512 is_input=0 DEBUG: ==> option 4: format= 34 width=2688 height=1512 is_input=1 DEBUG: ==> option 8: format= 35 width=2688 height=1512 is_input=0 DEBUG: ==> option 12: format= 35 width=2688 height=1512 is_input=1 DEBUG: ==> option 16: format= 33 width=2688 height=1512 is_input=0 DEBUG: ==> option 20: format= 34 width=1920 height=1440 is_input=0 DEBUG: ==> option 24: format= 34 width=1920 height=1440 is_input=1 DEBUG: ==> option 28: format= 35 width=1920 height=1440 is_input=0 DEBUG: ==> option 32: format= 35 width=1920 height=1440 is_input=1 DEBUG: ==> option 36: format= 33 width=1920 height=1440 is_input=0 DEBUG: ==> option 40: format= 34 width=1928 height=1208 is_input=0 DEBUG: ==> option 44: format= 34 width=1928 height=1208 is_input=1 DEBUG: ==> option 48: format= 35 width=1928 height=1208 is_input=0 DEBUG: ==> option 52: format= 35 width=1928 height=1208 is_input=1 DEBUG: ==> option 56: format= 33 width=1928 height=1208 is_input=0 DEBUG: ==> option 60: format= 34 width=1920 height=1080 is_input=0 DEBUG: ==> option 64: format= 34 width=1920 height=1080 is_input=1 DEBUG: ==> option 68: format= 35 width=1920 height=1080 is_input=0 DEBUG: ==> option 72: format= 35 width=1920 height=1080 is_input=1 DEBUG: ==> option 76: format= 33 width=1920 height=1080 is_input=0 DEBUG: ==> option 80: format= 34 width=1600 height=1200 is_input=0 DEBUG: ==> option 84: format= 34 width=1600 height=1200 is_input=1 DEBUG: ==> option 88: format= 35 width=1600 height=1200 is_input=0 DEBUG: ==> option 92: format= 35 width=1600 height=1200 is_input=1 DEBUG: ==> option 96: format= 33 width=1600 height=1200 is_input=0 DEBUG: ==> option 100: format= 34 width=1440 height=1080 is_input=0 DEBUG: ==> option 104: format= 34 width=1440 height=1080 is_input=1 DEBUG: ==> option 108: format= 35 width=1440 height=1080 is_input=0 DEBUG: ==> option 112: format= 35 width=1440 height=1080 is_input=1 DEBUG: ==> option 116: format= 33 width=1440 height=1080 is_input=0 DEBUG: ==> option 120: format= 34 width=1280 height= 960 is_input=0 DEBUG: ==> option 124: format= 34 width=1280 height= 960 is_input=1 DEBUG: ==> option 128: format= 35 width=1280 height= 960 is_input=0 DEBUG: ==> option 132: format= 35 width=1280 height= 960 is_input=1 DEBUG: ==> option 136: format= 33 width=1280 height= 960 is_input=0 DEBUG: ==> option 140: format= 34 width=1280 height= 800 is_input=0 DEBUG: ==> option 144: format= 34 width=1280 height= 800 is_input=1 DEBUG: ==> option 148: format= 35 width=1280 height= 800 is_input=0 DEBUG: ==> option 152: format= 35 width=1280 height= 800 is_input=1 DEBUG: ==> option 156: format= 33 width=1280 height= 800 is_input=0 DEBUG: ==> option 160: format= 34 width=1280 height= 768 is_input=0 DEBUG: ==> option 164: format= 34 width=1280 height= 768 is_input=1 DEBUG: ==> option 168: format= 35 width=1280 height= 768 is_input=0 DEBUG: ==> option 172: format= 35 width=1280 height= 768 is_input=1 DEBUG: ==> option 176: format= 33 width=1280 height= 768 is_input=0 DEBUG: ==> option 180: format= 34 width=1280 height= 720 is_input=0 DEBUG: ==> option 184: format= 34 width=1280 height= 720 is_input=1 DEBUG: ==> option 188: format= 35 width=1280 height= 720 is_input=0 DEBUG: ==> option 192: format= 35 width=1280 height= 720 is_input=1 DEBUG: ==> option 196: format= 33 width=1280 height= 720 is_input=0 DEBUG: ==> option 200: format= 34 width=1080 height=1080 is_input=0 DEBUG: ==> option 204: format= 34 width=1080 height=1080 is_input=1 DEBUG: ==> option 208: format= 35 width=1080 height=1080 is_input=0 DEBUG: ==> option 212: format= 35 width=1080 height=1080 is_input=1 DEBUG: ==> option 216: format= 33 width=1080 height=1080 is_input=0 DEBUG: ==> option 220: format= 34 width=1024 height= 738 is_input=0 DEBUG: ==> option 224: format= 34 width=1024 height= 738 is_input=1 DEBUG: ==> option 228: format= 35 width=1024 height= 738 is_input=0 DEBUG: ==> option 232: format= 35 width=1024 height= 738 is_input=1 DEBUG: ==> option 236: format= 33 width=1024 height= 738 is_input=0 DEBUG: ==> option 240: format= 34 width=1024 height= 768 is_input=0 DEBUG: ==> option 244: format= 34 width=1024 height= 768 is_input=1 DEBUG: ==> option 248: format= 35 width=1024 height= 768 is_input=0 DEBUG: ==> option 252: format= 35 width=1024 height= 768 is_input=1 DEBUG: ==> option 256: format= 33 width=1024 height= 768 is_input=0 DEBUG: ==> option 260: format= 34 width= 864 height= 480 is_input=0 DEBUG: ==> option 264: format= 34 width= 864 height= 480 is_input=1 DEBUG: ==> option 268: format= 35 width= 864 height= 480 is_input=0 DEBUG: ==> option 272: format= 35 width= 864 height= 480 is_input=1 DEBUG: ==> option 276: format= 33 width= 864 height= 480 is_input=0 DEBUG: ==> option 280: format= 34 width= 800 height= 600 is_input=0 DEBUG: ==> option 284: format= 34 width= 800 height= 600 is_input=1 DEBUG: ==> option 288: format= 35 width= 800 height= 600 is_input=0 DEBUG: ==> option 292: format= 35 width= 800 height= 600 is_input=1 DEBUG: ==> option 296: format= 33 width= 800 height= 600 is_input=0 DEBUG: ==> option 300: format= 34 width= 800 height= 480 is_input=0 DEBUG: ==> option 304: format= 34 width= 800 height= 480 is_input=1 DEBUG: ==> option 308: format= 35 width= 800 height= 480 is_input=0 DEBUG: ==> option 312: format= 35 width= 800 height= 480 is_input=1 DEBUG: ==> option 316: format= 33 width= 800 height= 480 is_input=0 DEBUG: ==> option 320: format= 34 width= 720 height=1280 is_input=0 DEBUG: ==> option 324: format= 34 width= 720 height=1280 is_input=1 DEBUG: ==> option 328: format= 35 width= 720 height=1280 is_input=0 DEBUG: ==> option 332: format= 35 width= 720 height=1280 is_input=1 DEBUG: ==> option 336: format= 33 width= 720 height=1280 is_input=0 DEBUG: ==> option 340: format= 34 width= 720 height= 480 is_input=0 DEBUG: ==> option 344: format= 34 width= 720 height= 480 is_input=1 DEBUG: ==> option 348: format= 35 width= 720 height= 480 is_input=0 DEBUG: ==> option 352: format= 35 width= 720 height= 480 is_input=1 DEBUG: ==> option 356: format= 33 width= 720 height= 480 is_input=0 DEBUG: ==> option 360: format= 34 width= 640 height= 512 is_input=0 DEBUG: ==> option 364: format= 34 width= 640 height= 512 is_input=1 DEBUG: ==> option 368: format= 35 width= 640 height= 512 is_input=0 DEBUG: ==> option 372: format= 35 width= 640 height= 512 is_input=1 DEBUG: ==> option 376: format= 33 width= 640 height= 512 is_input=0 DEBUG: ==> option 380: format= 34 width= 640 height= 480 is_input=0 DEBUG: ==> option 384: format= 34 width= 640 height= 480 is_input=1 DEBUG: ==> option 388: format= 35 width= 640 height= 480 is_input=0 DEBUG: ==> option 392: format= 35 width= 640 height= 480 is_input=1 DEBUG: ==> option 396: format= 33 width= 640 height= 480 is_input=0 DEBUG: ==> option 400: format= 34 width= 640 height= 400 is_input=0 DEBUG: ==> option 404: format= 34 width= 640 height= 400 is_input=1 DEBUG: ==> option 408: format= 35 width= 640 height= 400 is_input=0 DEBUG: ==> option 412: format= 35 width= 640 height= 400 is_input=1 DEBUG: ==> option 416: format= 33 width= 640 height= 400 is_input=0 DEBUG: ==> option 420: format= 34 width= 640 height= 360 is_input=0 DEBUG: ==> option 424: format= 34 width= 640 height= 360 is_input=1 DEBUG: ==> option 428: format= 35 width= 640 height= 360 is_input=0 DEBUG: ==> option 432: format= 35 width= 640 height= 360 is_input=1 DEBUG: ==> option 436: format= 33 width= 640 height= 360 is_input=0 DEBUG: ==> option 440: format= 34 width= 352 height= 288 is_input=0 DEBUG: ==> option 444: format= 34 width= 352 height= 288 is_input=1 DEBUG: ==> option 448: format= 35 width= 352 height= 288 is_input=0 DEBUG: ==> option 452: format= 35 width= 352 height= 288 is_input=1 DEBUG: ==> option 456: format= 33 width= 352 height= 288 is_input=0 DEBUG: ==> option 460: format= 34 width= 320 height= 240 is_input=0 DEBUG: ==> option 464: format= 34 width= 320 height= 240 is_input=1 DEBUG: ==> option 468: format= 35 width= 320 height= 240 is_input=0 DEBUG: ==> option 472: format= 35 width= 320 height= 240 is_input=1 DEBUG: ==> option 476: format= 33 width= 320 height= 240 is_input=0 DEBUG: ==> option 480: format= 34 width= 240 height= 320 is_input=0 DEBUG: ==> option 484: format= 34 width= 240 height= 320 is_input=1 DEBUG: ==> option 488: format= 35 width= 240 height= 320 is_input=0 DEBUG: ==> option 492: format= 35 width= 240 height= 320 is_input=1 DEBUG: ==> option 496: format= 33 width= 240 height= 320 is_input=0 DEBUG: ==> option 500: format= 34 width= 176 height= 144 is_input=0 DEBUG: ==> option 504: format= 34 width= 176 height= 144 is_input=1 DEBUG: ==> option 508: format= 35 width= 176 height= 144 is_input=0 DEBUG: ==> option 512: format= 35 width= 176 height= 144 is_input=1 DEBUG: ==> option 516: format= 33 width= 176 height= 144 is_input=0 DEBUG: ==> option 520: format= 37 width=2688 height=1520 is_input=0 DEBUG: ==> option 524: format= 38 width=2688 height=1520 is_input=0 DEBUG: ==> option 528: format= 32 width=2688 height=1520 is_input=0 DEBUG: ==> option 532: format= 36 width=2688 height=1520 is_input=0 DEBUG: ==> option 536: format= 37 width=1344 height= 760 is_input=0 DEBUG: ==> option 540: format= 38 width=1344 height= 760 is_input=0 DEBUG: ==> option 544: format= 32 width=1344 height= 760 is_input=0 DEBUG: ==> option 548: format= 36 width=1344 height= 760 is_input=0 DEBUG: ==> option 552: format= 37 width= 192 height=1080 is_input=0 DEBUG: ==> option 556: format= 38 width= 192 height=1080 is_input=0 DEBUG: ==> option 560: format= 32 width= 192 height=1080 is_input=0 DEBUG: ==> option 564: format= 36 width= 192 height=1080 is_input=0 ERROR: Camera 1 failed to find supported preview config: 2704x1540 WARNING: Failed to start cam hires2 due to invalid resolution WARNING: assuming cam is missing and trying to compensate{ "version": 0.1, "fsync_en": true, "fsync_gpio": 109, "cameras": [{ "type": "imx412", "name": "hires", "enabled": true, "camera_id": 0, "fps": 30, "en_preview": true, "preview_width": 3840, "preview_height": 2160, "en_raw_preview": false, "en_small_video": false, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h264", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "small_venc_osd": false, "en_large_video": false, "large_video_width": 4056, "large_video_height": 3040, "large_venc_mode": "h264", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 40, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 40, "large_venc_osd": false, "en_snapshot": false, "en_snapshot_width": 4056, "en_snapshot_height": 3040, "exif_focal_length": 3.0999999046325684, "exif_focal_length_in_35mm_format": 17, "exif_fnumber": 1.2400000095367432, "snapshot_jpeg_quality": 75, "ae_mode": "off", "gain_min": 54, "gain_max": 8000 }, { "type": "imx664", "name": "hires2", "enabled": true, "camera_id": 1, "fps": 30, "en_preview": true, "preview_width": 2688, "preview_height": 1520, "en_raw_preview": true, "en_small_video": false, "small_video_width": 1024, "small_video_height": 768, "small_venc_mode": "h264", "small_venc_br_ctrl": "cqp", "small_venc_Qfixed": 30, "small_venc_Qmin": 15, "small_venc_Qmax": 40, "small_venc_nPframes": 9, "small_venc_mbps": 2, "small_venc_osd": false, "en_large_video": false, "large_video_width": 2688, "large_video_height": 1512, "large_venc_mode": "h264", "large_venc_br_ctrl": "cqp", "large_venc_Qfixed": 40, "large_venc_Qmin": 15, "large_venc_Qmax": 50, "large_venc_nPframes": 29, "large_venc_mbps": 40, "large_venc_osd": false, "en_snapshot": false, "en_snapshot_width": 2688, "en_snapshot_height": 1512, "exif_focal_length": 3.0999999046325684, "exif_focal_length_in_35mm_format": 17, "exif_fnumber": 1.2400000095367432, "snapshot_jpeg_quality": 75, "ae_mode": "off", "gain_min": 54, "gain_max": 8000, "en_misp": true, "misp_width": 1920, "misp_height": 1080, "misp_venc_enable": false, "misp_venc_mode": "h264", "misp_venc_br_ctrl": "cqp", "misp_venc_Qfixed": 30, "misp_venc_Qmin": 15, "misp_venc_Qmax": 50, "misp_venc_nPframes": 29, "misp_venc_mbps": 2, "misp_venc_osd": false, "misp_awb": "manual", "misp_gamma": 1.0 -
@mark , it looks like your version of voxl sdk has an early version of the imx664 driver (not supporting 2704x1540 resolution.
Are you able to upgrade sdk to 1.5.x or 1.6.x?
Alternatively, i could share the latest camera drivers to use with 1.4.5.
I would also recommend updating voxl-camera-server to the latest, but that would also require updating other packages.. would you consider upgrading the sdk or are you locked to 1.4.5?
Alex
