AR0144 RGB output on VOXL2
-
@Alex-Kushleyev, that would be amazing! Will the VOXL 2 be able to support 6 of these simultaneously, like it can with the M0149? Also, do you know where I can find a CAD model for the M0166? I can't find it on the datasheet you have for it.
-
VOXL can support 6 independent cameras (up to 4K each, with some limitations). Depending on the adapters used, there would potentially need to be a small kernel change. The Kernel for voxl2 is public and I could point you to the changes if needed.
Can you please clarify how exactly you are connecting six M0149 cameras to VOXL2? I don't think it's possible to use them all without a kernel change.
Using 6 uCoax versions of AR0144 may not be possible at the moment, but we can get close (5). You could use M0173 and connect 4 AR0144 there (+ TOF), but then use VOXL2 J8L to connect 5th AR0144 without any kernel changes (just 1.0.1 kernel variant). Single camera could be connected either using M0172 adapter or M0155 (uCoax) or M0076 / M0135 for old style connectors. For dual uCoax on J8, i don't think we have an adapter for right now, but you could connect two M0149's using an M0135 interposer.
If you use VOXL2 mini + M0188, you could actually connect 5 ucoax cameras (there are 6 connectors, but the 6th sensor is not yet enabled in the kernel). https://docs.modalai.com/M0188/#image-sensor-interfaces
*** EDIT: due to a subtle detail that VOXL2 has only 4 independent CCI (I2C) busses for camera control connecting more than 4 cameras with the same slave ID would create a conflict during communication. So, connecting 5 or 6 AR0144 cameras would actually be an issue.. However, the M0166 camera does support a change of CCI Slave address via a resistor on the camera PCB. That would require removing one 0402 resistor and installing another resistor on the camera PCB (and would void the warranty...). Additionally, it would require a small change in the camera driver, which is not difficult to support.. You would modify the CCI slave id for the 2 cameras so that they do not conflict with the other 4 cameras. So if you REALLY need more than 4 AR0144 cameras connected to a single VOXL2, it is possible.. However, I still do not see how the 6th M0166 camera can be enabled just yet, so 5 should be possible.
Lets discuss further..
The 3D CAD drawing of M0166 can be found here : https://storage.googleapis.com/modalai_public/modal_drawings/M0166_3D.stp
Alex
-
(it seems the M0166 CAD file does not include a lens holder and lens.. will check if we have one)
-
@Alex-Kushleyev, thank you for your help!
On second thought, 5 cameras should work for my application. I am thinking about using 4 of these RGB M0166 variants, connected to the VOXL2 via the M0173 connector module, and then having a single high-res M0161 camera attached to the VOXL2 through a separate camera group (via the M0155 adaptor). In this case, since I am using a separate camera module for the 5th camera, this should solve the same slave ID conflict without having to make an external change to the camera PCB, correct? Would I still need to modify the camera driver or kernel in a particular way to support this?
I'd like to move forward with buying 4 of these RGB M0166 variants if that's possible. Do these modules have an IR-cut filter?
-
If you use 4 AR0144 connected to M0173 and 5th hires IMX412 (M0161) attached to J8 via M0155, then everything should work out of the box without any hw or kernel changes. I can actually test this pretty easily. Let me do that.
Also I can send you some sample images comparing mono and color AR0144.
We do have limited quantity of the color AR0144, let me check on that as well..
Alex
-
@Alex-Kushleyev, let me know how the test goes, that would be super helpful to know. It would also be nice to see the sample images comparing the mono and color AR0144. I really appreciate your help with this!
-
@Alex-Kushleyev, with the smaller number of cameras I can connect to the VOXL2, I’d like to increase the field of view of each AR0144 to maximize coverage. From my research, I understand the AR0144 sensor modules use a 1/4" optical format and are compatible with M12 lenses. Can you confirm if that’s correct?
Also, how well do the AR0144-based camera modules (like the M0166) perform with large field-of-view M12 fisheye lenses (e.g., >190°)? Are there any known issues with image quality, distortion correction, or sensor limitations when using those types of lenses?
-
@Alex-Kushleyev, any update on whether I can buy 4 of these RGB AR0144s?
-
@Jordyn-Heil , sorry for the delay.
We do have the color AR0144 in stock. I just need to double check there will be no conflicts in communication.
I will get back to you today or tomorrow after I test the 4 ar0144 and one imx412.
Alex
-
Regarding the lens FOV, the lens specs are provided in the camera docs (162 deg diagonal). It calibrates just fine as a fisheye lens with very low reprojection error (<0.5 pixels). The AR0144 lens does have significant lens shading, which is evident around the edges. We have computed the lens shading compensation that normalized those areas, but the result is higher noise in those areas due to quantization. The intensity drop right at the corners is around 8x compared to the center (significant!). This is because the lens is small (the imx412 lens has much better response but it is also a much larger lens).
For the color use case, the lens shading becomes even mote interesting as the different colors are distorted differently, so each color needs to have its own LSC (lens shading compensation). I have started working on RGB LSC but it is not finished yet. As a result you get not only brightness drop at the edges but also discoloration.
Will send you some image samples.
The lens has an M7 thread (i believe), so using M12 is not possible.
Let me know if you have any more questions about this or if i missed something.
Alex
-
@Alex-Kushleyev, great! If there are no conflicts in communication, I'd still like to move forward with buying them as soon as possible, so just let me know what I need to do to purchase them.
If I'm trying to do full 360 panoramic stitching with four of these sensors, will the fisheye lens distortion drastically impact the efficacy of doing a seamless stitch (particularly with the discoloration from lack of an RGB LSC)?
-
@Jordyn-Heil , for ordering, since it's a custom order, please send us a message via http://modalai.com/contact and provide the following part number. We will send you an invoice.
MSU-M0166-2-01 (which includes MCCA-M0166-2 + M10000858 lens (AR0144 Camera Color Coax))Do you want to wait until I confirm 4x concurrency?
Regarding panoramic stitching, yes obviously lens shading will be an issue, however if you have 4 cameras, you only need 90 degrees of horizontal FOV per camera and the current lens provides 139 degrees of horizontal FOV (https://docs.modalai.com/M0166/). So i think 90 degrees + small overlap won't be too bad (but LSC correction will be required to achieve seamless stitching)
. Also, the VOXL2 GPU can do the stitching, I can help you get started (should be a fun project).Here is an old example i found of original vs LSC corrected monochrome images:
original:
LSC corrected:
You see the corners are heavily quantized because they were so dark and pixel resolution is not enough. The original RAW image is 8 bit. The AR0144 camera does support 10 and 12 bit output, and the 12 bit data would help a lot with Quantization effects after correction. the 10 and 12 bit stream is experimental and there are some issues (will not go into detail here), that is why we using 8 bit, which is stable.
Alex
-
One more thing.. Do you need global shutter cameras? or is a small size a requirement? You could use the IMX412 cameras, which are rolling shutter, but much higher resolution and color quality. Have you considered that option?
Alex
