Hi @jameskuesel ,
When you enable en_raw_preview, which is what we do when MISP is enabled, the preview_width and preview_height basically force the selection of a specific camera mode with those dimensions. This means that the camera will send a bayer image of that size.
After the image is received by the ISP (which does nothing else in RAW-only mode), then MISP consumes the raw bayer image and performs debayering on the GPU. Then misp_width and misp_height will be used to specify the dimensions of the output image, sampled from the original bayer image.
If the output dimensions have exactly the same ratio as the input dimensions, then MISP will perform (arbitrary) down-scaling only (no crop = same horizontal and vertical FOV). However, if the output dims have different aspect ratio, the output image will be a cropped (and downscaled) version of the original image, such that the width and height ratio is maintained (features are not stretched). The crop is selected to maximize the fit of the output image within the input image.
With that in mind, if you select 4040x3040 as preview resolution and select misp resolution such that both width and height are scaled by the same factor (does not have to be integer), you can calibrate the intrinsics using the small image and after the calibration you can upscale the intrinsics:
multiply the principal point offsets by the same factor
multiply the focal length by the same factor
keep the same fisheye distortion coefficients (since they are a function of angle, not pixels)
See another version of this explanation in a different context : https://forum.modalai.com/topic/4900/running-qvio-on-a-hires-camera/12 (look for "intrinsics")
Input to the EIS algorithm is the full bayer image that is defined by the preview_width and preview_height.
the output image size is defined by misp_width and misp_height, just like when EIS is off. The misp resolution can be arbitrary
an important parameter is misp_zoom, which defines the relationship between the original image and the output image, in terms of zoom level. if you set zoom = 1.0 with EIS enabled, then the misp output will produce the largest
undistorted image that fits within the original bayer image
there will be no deformation (stretching). the EIS ROI size will fit within the original bayer image and the zoom level is really controlling how much the FOV is reduced and how much stabilization margin you have.
you can enable the ROI display (PIP or side-by-side) to see how the EIS ROI fits within the original full image
the actual resolution of the misp output has nothing to do with how the EIS ROI fits within the original bayer image : the zoom value and fisheye undistortion dictate that. The output misp resolution just defines how many pixels the output image will contain, not the field of view (FOV).
for output resolution, you can pick whatever you need for your application.
By the way, you can have multiple EIS outputs on the same camera concurrently (MISP supports up to 4 channels). So you can have one stream that is small resolution (for streaming over wifi) and another one higher resolution (and can be different aspect ratio) for high quality recording. https://docs.modalai.com/camera-video/electronic-image-stabilization/#using-eis-with-misp-channels
Please note that currently, when you use multiple misp channels from the same camera, the image is processed separately, meaning if you have a large resolution and small resolution misp stream, the small resolution is NOT generated by downsampling the output of the large stream (which could be a nice optimization, in some cases).
Please let me know if you have any other questions.
Alex
