AR0144 RGB output on VOXL2
-
@Alex-Kushleyev, any update on whether I can buy 4 of these RGB AR0144s?
-
@Jordyn-Heil , sorry for the delay.
We do have the color AR0144 in stock. I just need to double check there will be no conflicts in communication.
I will get back to you today or tomorrow after I test the 4 ar0144 and one imx412.
Alex
-
Regarding the lens FOV, the lens specs are provided in the camera docs (162 deg diagonal). It calibrates just fine as a fisheye lens with very low reprojection error (<0.5 pixels). The AR0144 lens does have significant lens shading, which is evident around the edges. We have computed the lens shading compensation that normalized those areas, but the result is higher noise in those areas due to quantization. The intensity drop right at the corners is around 8x compared to the center (significant!). This is because the lens is small (the imx412 lens has much better response but it is also a much larger lens).
For the color use case, the lens shading becomes even mote interesting as the different colors are distorted differently, so each color needs to have its own LSC (lens shading compensation). I have started working on RGB LSC but it is not finished yet. As a result you get not only brightness drop at the edges but also discoloration.
Will send you some image samples.
The lens has an M7 thread (i believe), so using M12 is not possible.
Let me know if you have any more questions about this or if i missed something.
Alex
-
@Alex-Kushleyev, great! If there are no conflicts in communication, I'd still like to move forward with buying them as soon as possible, so just let me know what I need to do to purchase them.
If I'm trying to do full 360 panoramic stitching with four of these sensors, will the fisheye lens distortion drastically impact the efficacy of doing a seamless stitch (particularly with the discoloration from lack of an RGB LSC)?
-
@Jordyn-Heil , for ordering, since it's a custom order, please send us a message via http://modalai.com/contact and provide the following part number. We will send you an invoice.
MSU-M0166-2-01 (which includes MCCA-M0166-2 + M10000858 lens (AR0144 Camera Color Coax))Do you want to wait until I confirm 4x concurrency?
Regarding panoramic stitching, yes obviously lens shading will be an issue, however if you have 4 cameras, you only need 90 degrees of horizontal FOV per camera and the current lens provides 139 degrees of horizontal FOV (https://docs.modalai.com/M0166/). So i think 90 degrees + small overlap won't be too bad (but LSC correction will be required to achieve seamless stitching)
. Also, the VOXL2 GPU can do the stitching, I can help you get started (should be a fun project).Here is an old example i found of original vs LSC corrected monochrome images:
original:
LSC corrected:
You see the corners are heavily quantized because they were so dark and pixel resolution is not enough. The original RAW image is 8 bit. The AR0144 camera does support 10 and 12 bit output, and the 12 bit data would help a lot with Quantization effects after correction. the 10 and 12 bit stream is experimental and there are some issues (will not go into detail here), that is why we using 8 bit, which is stable.
Alex
-
One more thing.. Do you need global shutter cameras? or is a small size a requirement? You could use the IMX412 cameras, which are rolling shutter, but much higher resolution and color quality. Have you considered that option?
Alex
-
@Alex-Kushleyev, thanks for the photos! Yes, global shutter and small size are requirements, so I think the color AR0144s are my best option, but I'll wait to order until you confirm 4x concurrency.
Also, if you can help me get started with using the VOXL2 GPU for panoramic stitching, that would be awesome! Thanks for all your help!
-
Hi @Jordyn-Heil ,
OK, so did some digging. Essentially, the 4x AR0144 + 1x IMX412 would be the same as C27 config (https://docs.modalai.com/voxl2-coax-camera-bundles/#mdk-m0173-1-02) + another AR0144 connected to J8 via M0155 (so that would be J8L, camera slot 4).
There are also some other options like moving IMX412 to J8, but in either case, unfortunately there is a change required in one of the AR0144 to switch the i2c slave address from default. This means moving a resistor on the camera from one pad to another.
I am going to double check this by actually switching the slave ID, but just wanted to give you an update..
Alex
-
OK, I was able to confirm this configuration:
- VOXL2
- M0173 + 3x AR0144 + 1x IMX412 (C27 config)
- M0155 + 1x AR0144 plugged into J8
- AR0144 i2c slave id change via resistor
- small camera driver modification for new slave id
- also verified the synchronization between the 4 AR0144
The resistor swap was relatively easy, but i used a special "tweezer" soldering iron tip. I can be difficult without the right tools.. Let me check if we can do a swap for the custom order..
See the image with marked slave id select resistor that needs to move to a pad right above the current placement. It can be a bit tricky to do it
Alex
-
Here are two image samples taking using IMX412 and AR0144. IMX412 is slightly cropped on the sides, but it's also a slightly narrower lens. AR0144 image is without LSC correction. Also images slightly out of focus because the test pattern is very close to the camera. The brightness in the IMX412 image is more evenly distributed (larger, better lens), so the center of the image is not over-exposed, unlike the AR0144. This should be corrected with the proper LSC..
IMX412
AR0144-color
