IMX412 Poor performance vs IMX214?

jameskuesel

Hi Alex, Thanks for the reply.

Unfortunately, I didn’t see any improvement with any of the changes you suggested. That being said I was a little confused on some of the setup.  To reiterate, this was done on a

Starling2 with IMX412 M0161 attached via a coax cable.

After a stock flash of Voxl-SDK-1.4.0, running   logcat | grep -i tuning  would return com.qti.tuned.imx412_fpv.bin as being loaded. However, your response seemed to imply that com.qti.tuned.cmk_imx577.bin would be loaded after a stock flash. After removing com.qti.tuned.cmk_imx577.bin there seemed to be no change. Logcat still implied com.qti.tuned.imx412_fpv.bin was being loaded. However, I did also try removing com.qti.tuned.imx412_fpv.bin which then indeed resulted in com.qti.tuned.default.bin being loaded. I saw a difference in how bright the image was but not how sharp it looked.  I did another stock flash to reset everything and proceed with running your linking command. That looked same as stock. But as previously mentioned, if com.qti.tuned.imx412_fpv.bin was already being loaded, then linking com.qti.tuned.imx412_fpv.bin would probably look the same, which it did.  I also attempted to force the system to use com.qti.tuned.cmk_imx577.bin by replacing the contents of com.qti.tuned.imx412_fpv.bin with it. This provided an even brighter image, but still no noticeable improvement in sharpness.

I’ve included the pictures here and what driver I THINK was being used. To me it seems that for the Starling2 kernel that maybe com.qti.tuned.cmk_imx577.bin isn’t being loaded by default.

The downsampled images are hard to compare so I included a drive link again.

com.qti.tuned.imx412_fpv.bin (Stock??) VVV

com.qti.tuned.default.bin VVV
IMX412_M0161_kernel2_J6Uppe_com.qti.tuned.default_downsampled.jpg

com.qti.tuned.cmk_imx577.bin (forced it to use (or at least I think I did))  VVV
IMX412_M0161_kernel2_J6Uppe_com.qti.tuned.cmk_imx577_downsampled.jpg

Moving on, decided to do some more experimentation which was even more puzzling. We happen to have an

IMX412 M0107 that attaches via a ribbon

module that we attached on J6Lower (with a different breakout board). To achieve this had to do a re-flash and use Kernel #1 (not the Starling2 Kernel as it seems like that one is incompatible). This surprisingly produced great results, much brighter, and much much better sharpness. The difference is night and day compared to the coax M0161. This time, it does seem that com.qti.tuned.cmk_imx577.bin was the one being loaded as you implied might happen after a stock flash.

A little confusing on what exactly is going on here. Obviously they must be using different drivers as this test and our Starling2 test use different ports. (J6Lower vs J6Upper). Furthermore, these experiments required using different kernels.

Should there be this big a difference based on using a coax vs the ribbon (M0161 vs M0107) or is it because of the different kernels? Do you have any sample images from the Starling2 Hires (M0161) with coax so we know what we should be expecting?

What we saw with the ribbon (M0107) is what we are aiming for but with the coax (M0161).

Thanks again for the help, any insight would be much appreciated.

Drive Link (better for comparison in sharpness)
Drive Pictures

Alex Kushleyev

@jameskuesel ,

Can you please let me know what steps you follow to set up M0161 with the latest SDK? Do you modify voxl camera server conf file after set up? Also which camera stream are you saving image from and how are you saving?

M0161 and M0107 are more or less identical except for the cable type, so i would expect similar performance.

It looks like the lens may be out of focus. There is usually some glue on the threads that prevents the lens from twisting in the threads. I will double check whether you can just twist the lens to loosen the glue or remove the glue (in order to re-focus).

I will take a few image samples using m0161 and m0107 - it is possible to connect them both to voxl2 at the same time.

We can also do another test - log the raw (unmodified) image, which will be in 10 bit bayer format and i have some tools to convert that to rgb and save to jpg, etc. if raw image looks blurry, then camera must be out of focus. I will provide instructions.

Alex

jameskuesel

@Alex-Kushleyev

Hi Alex,

To get the M0161 (COAX) all I did was run the install script, and then selected all the options relating to the starling2 since that is what I was collecting these on.

I've been recording the preview frame (en_preview: true) modified to be at full res (4056x3040), stream via voxl-logger.

voxl-logger --cam hires_color.

I did a similar process for the M0107 (RIBBON), except selecting the first kernel option. Then I modified the custom_camera_config.txt in /data/modalai and just set

J6_LOWER_SENSOR="imx412"
J6_LOWER_NAME="hires"
J6_LOWER_ROTATE="false"

I think you are correct that our Starling M0161 lens is a little bit out of focus. However, we tried with another M0161 module we had around that looked more "in focus" but still had the sharpness issues. The resulting images between the M0107 and M0161 look different enough that I feel it is something more than just the focus but I am unsure.

M0161 but DIFFERENT module (more focused?)

Anyways, yes let me know when you get some sample images and would also love to experiment with the logging of the raw images. Also, how did you set up both modules at the same time? I was trying to figure that out but had no luck.

Alex Kushleyev

@jameskuesel ,

You can plug M0107 into VOXL2 J8L (lower) connector because J8 is not occupied with the M0173 Camera Front End (which takes up J6 and J7). You would need either M0135 (dual) or M0076 (single) camera interposer, one of which you are probably already using for testing M0107 (plus the flex cable).

J8_LOWER_SENSOR="imx412"
J8_LOWER_NAME="hires"
J8_LOWER_ROTATE="false"

You can use voxl-camera-server -l to see which cameras are detected and the camera in slot J8L should be detected with the camera slot ID 4.

If you need additional help with setting up M0107 on J8L, let me know.

I will be testing the image quality and will get back to you.

Since I am pretty sure the issue is with focus, you can try the following..

M0161 cameras have the lenses secured with UV-cure epoxy, but you should be able to untwist the lens. Hold the camera firmly by the plastic housing of the module and twist the lens counter-clock wise. It will feel tight but should start turning.
If we are lucky, then untwisting can bring the image back into focus, but if you see that the image is getting more out of focus, then the epoxy would have to be removed or you would need a different lens
i will try to see if epoxy can be removed from the threads
if you have another M12 similar lens you could try to screw it in and see if you can focus it better than how it is now. Maybe take a look and see if the lens on your M0107 is sealed with the glue or maybe it is easily removable, you can swap them.
if you remove the lens from the camera completely, please make sure that dust does not get into the camera sensor or on lens, otherwise it will obviously affect the image quality.

Alex

Dobry Kolacz

If you cannot get the lens to spin with the epoxy in place, you can take some small tweezers and pick away at the glue. That should eventually free up enough of the threads/barrel to allow for focal adjustment.

Please ping me if you have issues getting the lens loose

Alex Kushleyev

Thanks @Dobry-Kolacz for the tips!

Also, just wanted to confirm that after removing the lens, i was able to completely remove the glue from threads using tweezers. It was actually pretty easy, just need to get the fine tip between the threads and drive it along the threads under the glue and the glue comes straight up. This should allow full adjustment.

Please note that there may be some uncured epoxy hiding on the threads, which you should wipe off with a microfiber cloth, if you take out the lens completely.

Alex

Dobry Kolacz

You're welcome!

The epoxy used is UV cure epoxy. Any uncured epoxy should be able to be wiped away with a microfiber as Alex mentioned, using isopropyl alcohol in conjunction does wonders.

We have recently switched to using RTV silicone to secure these lenses in place which seems to be working well for us.

Dobry Kolacz

We focus the lens then apply just a small drop on the outside of the barrel/threads where the lens interfaces with the housing

jameskuesel

@Alex-Kushleyev Sorry for the delay got sidetracked with some other things but wanted to close this ticket out.

Focus did indeed seem to be the issue. However, seems like most of our M0161 modules simply did not come with any epoxy, or at the very least not enough to prevent the lens from spinning accidentally. Something to watch out for.

Also, I found that at least on Voxl-SDK-1.4.0 for me it seems

com.qti.tuned.imx412_fpv.bin

is indeed what is being loaded as default. By renaming com.qti.tuned.cmk_imx577.bin to force the camera server to load it I found the picture at least aesthetically more pleasing. But that is obviously subjective.

Anyways, yeah focus seemed to be the major issue.

Alex Kushleyev

@jameskuesel , thank you for letting me know about the focus issue. It seems we had a few cameras shipped out without being properly focused - we will make sure to take care of it in the future. Sorry about that.

Regarding the image quality, after adjusting the lens focus, are you satisfied with the image resolution / quality? Also i believe the color balance outdoors should look better under natural light.

Alex