Posts made by Alex Kushleyev

@cguzikowski ,

Can you please clarify what you meant by the following statement in your previous post:

"Also as a side note, the metadata seems to not be recording when using voxl-record-raw-image -j"

Thanks!

@Igor , we just uploaded the emulator docker image, please see details in this post : https://forum.modalai.com/topic/5179/qrb5165-emulator-image-1-6

@huy ,

The latest qrb5165 emulator docker image is v 1.5. We have been updating our downloads section, so it was inadvertently removed.

Right now, this docker image is available in the Downloads / Miscellaneous section and is called 865-rootfs_m0054_qrb5165-emulator-v1.5.tar. You can download it from there.

However, the name and location of this docker image is slightly incorrect. It will be renamed to qrb5165-emulator-v1.5.tar and moved to the Docker Images section soon. So if you cant find it under Miscellaneous section, please look at the Docker Images section.

https://developer.modalai.com/

Alex

Hi @Myles-Levine ,

I just want to double check something to make sure there is no confusion.

• The lepton sensor is an IR (thermal) camera. it does not measure range. The lepton sensor plugs into the large square socket that is mounted to the M0157 / M0187 board (they are very similar).
• The infrared TOF sensor is a smaller (rectangular) sensor, which is marked as U1 on M0187 : https://docs.modalai.com/M0187/ .

Have you checked whether the Lepton sensor is working using voxl-portal, which would allow you to look at the IR image coming from the sensor?

Alex

@Myles-Levine , sorry I thought you were referring to the other TOF sensor.

You will need a new M0187 board : https://docs.modalai.com/M0187/ . Please reach out to us via this form https://www.modalai.com/pages/contact-us , since we don't have that part available on the official page.

Is your Lepton sensor working properly?

Alex

@Myles-Levine , you can purchase the TOF sensor here : https://www.modalai.com/products/m0178?variant=48528287793456

@yaoyuh , it looks like the order has been taken care of. please let us know if you have any more questions.

Alex

@bendraper , have you considered running a docker container (VM) on your Windows machine with a minimal Linux kernel that would use the Linux drivers for NCM and create the proper network connection between VOXL2 and Linux container. Then you could potentially set up the routing table in Windows so that it can access the device via the Linux container and linux container may need to have some networking magic down as well to pass through the communication?

Alex

@Igor , sorry, we are working on restoring it.

@Gerhold-Ten-Voorde , have you considered adding a spacer between the camera sensor PCB and the current lens holder? At least for testing purposes, that should work.

I will check if we can share the lens holder part numbers.

Alex

OK, some updates..

• the reason why the resolution 9216x6944 was not accepted for snapshot is that the camera pipeline has a list of allowed resolutions and that resolution was not one of them. I could add the resolution 9216x6944 to the list, but actually another resolution (9216x6912) was already supported, so i updated the ov64b driver to include that. The latest driver is available here : https://storage.googleapis.com/modalai_public/temp/ov64b/20260417/ov64b_20260417.zip

  • this driver has the following resolutions : 9248x6944, 9216x6944 9216x6912 and the other lower resolutions. You can use 9216x6912 since it's compatible with misp and isp snapshot.

• I also added 9216x6912 resolution to the supported misp resolutions in camera server (dev branch) as well as allowed using misp with snapshot (previously it would throw an error since we thought that previously they were not compatible).

• Actually, snapshot does work with raw_preview + misp, but here is a warning that gets printed continuously:

WARNING: preview buffer pool for Cam(hires), Frame(30) has 0 free, skipping request

we can work around this warning by adding the following line to the voxl-camera-server.conf for the ov64b camera. this limits the number of buffers that are queued up for the streams and resolves the warning.

"max_request_queue_depth": 6,

So with these changes, you should be able to use two configurations

• preview + raw_preview + misp + snapshot
• small_video + snapshot

Please use the latest camera server from dev branch.

Right now, if you enable misp + small+video + snapshot, it does not work properly, but i will try it figure it out. Here is an example set up for small video + snapshot, but you can easily change it to use misp by disabling small_video and enabling preview + misp (and set auto exposure to "auto" from "isp" (auto will use misp auto exposure)).

Please try it out. I do agree that the ISP output has better overall image quality than MISP, also related to pretty significant lens shading of this small lens on ov64b. We will work on improvements of misp image processing.

For now, you just need to decide whether you need both RAW bayer and ISP JPG, in which case you cannot use small_video for streaming (until i resolve the issue). This means slightly worse image quality for live streaming. But if you are ok with ISP jpg, you can disable misp and use small_video stream for streaming.

By the way, i did not see any artifacts in the jpeg from ISP. Can you please let me know if the artifact is in every frame and where exactly it is located? or is the occurrence and location random?

{
                        "type":                 "ov64b",
                        "name":                 "hires",
                        "enabled":              true,
                        "camera_id":            1,
                        "fps":                  30,
                        "en_preview":           false,
                        "en_misp":              false,
                        "max_request_queue_depth": 6,
                        "preview_width":        9216,
                        "preview_height":       6912,
                        "en_raw_preview":       true,
                        "en_small_video":       true,
                        "en_large_video":       false,
                        "en_snapshot":          true,
                        "ae_mode":              "isp",
                        "gain_min":             100,
                        "gain_max":             1600,
                        "misp_width":           1600,
                        "misp_height":          1200,
                        "misp_venc_enable":     false,
                        "misp_venc_mode":       "h265",
                        "misp_venc_br_ctrl":    "cqp",
                        "misp_venc_Qfixed":     38,
                        "misp_venc_Qmin":       15,
                        "misp_venc_Qmax":       50,
                        "misp_venc_nPframes":   29,
                        "misp_venc_mbps":       30,
                        "misp_venc_osd":        false,
                        "misp_awb":             "auto",
                        "misp_gamma":           1,
                        "misp_zoom":            1,
                        "ae_desired_msv":       110,
                        "exposure_min_us":      20,
                        "exposure_max_us":      33000,
                        "exposure_soft_min_us": 5000,
                        "ae_filter_alpha":      0.6,
                        "ae_ignore_fraction":   0.2,
                        "ae_slope":     0.05,
                        "ae_exposure_period":   1,
                        "ae_gain_period":       1,
                        "small_video_width":    1600,
                        "small_video_height":   1200,
                        "small_venc_mode":      "h264",
                        "small_venc_br_ctrl":   "cqp",
                        "small_venc_Qfixed":    30,
                        "small_venc_Qmin":      15,
                        "small_venc_Qmax":      40,
                        "small_venc_nPframes":  9,
                        "small_venc_mbps":      2,
                        "en_snapshot_width":    9216,
                        "en_snapshot_height":   6912,
                        "exif_focal_length":    3.1,
                        "exif_focal_length_in_35mm_format":     17,
                        "exif_fnumber": 1.24,
                        "snapshot_jpeg_quality":        75
                }

@cguzikowski ,

I am testing OV64b now. I confirmed that snapshot of size 9216x6944 is not possible (i think i can enable it), but 9248x6944 works.

Also, please see this post, there was a bug that broke snapshot in recent releases, which i just fixed. perhaps that was an issue for you when you switched to latest camera server : https://forum.modalai.com/topic/5166/snapshot-crashes-voxl-camera-server

I am going to see if i can get MISP running together with ISP snapshot, right now it only runs when preview and misp are disabled and i have small_video and snapshot streams enabled and snapshot resolution set to 9248x6944 and small_video resolution to something smaller.

Alex

@rddrone , you are absolutely right, the snapshot feature was broken. We somehow missed it.

I just fixed it :

• https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commit/6e02fdf2fe6a893b1c9e102a346050dece3e1a7d

and removed references to old buffer groups

• https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commit/b84e10c326ad5abd4077c71f28c6d6a4522cc8a0

Please try it out!

Alex

@yaoyuh, sorry for the delay. It looks like we have the lenses in stock, so you could fill out the contract form for a custom order and mention part number "M10000513 - Lens for M0161 Camera without IR filter".

https://www.modalai.com/pages/contact-us

Also, you can remove the IR filter from your current lenses. You have to be a bit careful so that the pieces of filter are not spread in your work space. My suggestion would be to first place a small piece of tape on top of the filter which will keep all the pieces together if the filter breaks during removal. The filter has a rectangular shape and is attached to the back side of the lens with an adhesive in 4 corners. It should be possible to just pry it off from the back of the lens (after completely removing the lens from the camera module). When you remove the lens from the camera module, you should cover it up so that dust does not get inside and on the sensor itself.

Please note that removing the filter from the lens is not an official guideline, but you could do it at your own risk.

Please see the following post where we also discussed removing the IR filter from AR0144 (which was actually more difficult because the filter there has a round shape) : https://forum.modalai.com/topic/4826/msu-m0149-1-ir-filter

Alex

@cguzikowski , if you want, you can attache a cropped image, so that it's smaller size.

In general, the pixel noise increases as you the pixel gain is increased. Part of Auto Exposure control is controlling exposure time and gain (both of which contribute to the image brightness), but higher gain has higher pixel noise and higher exposure values will result in motion blur.

The Qualcomm ISP has lots of liters, including noise reduction, which are applied when you use the ISP snapshot (even though they have not been tuned for the particular camera, the ISP output may result in better de-noised image). There are several types of filters, for example spatial (such as bilateral filter) and temporal (TNR - temporal noise reduction), and it's usually a combination of both with the filter weights increasing as the gain increases (more noise requires more noise reduction).

Currently, MISP does not have any de-noise filtering, but we are working on adding some. That is why i was discussing with you the ability to process the image offline (from the original bayer source). You could save the full raw image and perform any filtering you need in post processing. This approach is similar to using a RAW image on a fancy camera and then importing that into image processing software on a laptop / desktop, which can perform a lot more filters / effects directly on the raw image (loss-less).

What i suspect is that you are testing in the low-light environment and you are seeing the effects of high gain (high pixel noise). the MISP auto exposure tries to balance exposure and gain and there are a few parameters for that, but in your tests you should see what exposure and gain values the camera is at when you see the noisy image. You can use voxl-portal to control exposure and gain to see what the difference is. I believe the max gain for ov64b is 16x (1600).

I am about to set up your use case again for testing and i will investigate the noise and the original ISP snapshot artifact and we can also compare the image noise from the ISP snapshot and misp snapshot.

Unfortunately the lens for ov64b in the Hadron unit is very small, which reduces the amount of light that gets into the sensor. The sensor is 8+K resolution which means the pixel size is small, so the amount of light that gets to each pixel is small. To compensate for that, we would typically want a larger lens, but it is probably not possible to change the lens in this specific Hadron unit (we have not tried).

If you want to explore offline image processing, you would need:

• the original raw bayer (which you can already save)
• the gain and exposure used for that frame (i will need to check, i thin we had an option to save the exposure and gain as part of the file name).
• then you can have offline processing that is dependent on exposure / gain and apply the filters of your choice.
• the only down side is that the raw snapshots are huge, but you can probably zip them up if needed to store a lot of them on voxl2 before offloading.

Alex

@rddrone , thanks for the information. I will look into this. I have not heard of this issue before, so maybe something new, but i will test it out today.

Alex

@Anthony-Kang , we are looking into this. However, initial attempts to enable Intra Refresh on VOXL2 via the OMX api did not succeed either for h264 or h265. Also h265 does not work with fixed Macro Blocks (Coding Tree Units instead), so it seems the OMX structures for this param would not work with h265.

Alex

@Igor ,

You are absolutely correct about the incorrect parsing of the accel data. This is a typo and it was not detected because the effect of it is very small (4 LSB of the 20-bit number). The accelerometer noise in flight will dominate and these bytes will have no effect. We will fix this bug.

Regarding the order of the reading of FIFO_COUNT, we looked at many other sources, including the reference code from TDK and all the implementations read the FIFO_COUNTH first, so we suspect that the documentation has a typo (which makes sense, otherwise the implementation would have to have separate transactions for reading the H and L parts).

Thank you again for your bug report.

Alex

@rddrone , can you please provide your voxl-camera-server.conf to help us debug / reproduce?

Alex

@RoyAzriel , we were temporarily out of stock on some components and C29 configuration of Starling 2 Max is available again.

The TOF sensor kit (M0177 + M0171 + TOF Sensor) will plug in directly into M0173 camera front end and the Front camera mount already has a spot to mount for the TOF sensor. So you should be able to turn C28 into C29. See C27 here for reference how the TOF sensor is connected : https://docs.modalai.com/voxl2-coax-camera-bundles/#kit-pictures-and-descriptions

Alex