ModalAI Forum

Alex Kushleyev

please see : https://forum.modalai.com/topic/5529/starling-2-max-v4-c29-current-assembly-step-cad-availability -- this has a link to the latest Starling 2 Max V4 but in C29 configuration. Does that work for you? We will see if we can make a C28 version as well.

Alex Kushleyev

The setup worked exactly as you described above in the table.

All 5 AR0144 are running sync'ed, i installed R5 in M0173 in order to patch through the sync line to camera in slot 1 (which also uses the alt slave id 0x20). The camera plugged into slot 1 has the resistor swap to make it 0x20. I have not tested the software SID remap, that will be next.

I only have Boson320 at the moment, so that's what I tested, but Boson640 will work just as well. Lepton and Boson can run concurrently, so we have 7 cameras there.

No frame losses, sync works

output of voxl-inspect-cpu while running voxl-inspect-cam for the 7 streams (no voxl-portal).

Do you have all the hardware needed to try this? if you just have M0173 without the resistor fix, you can run cam 1 without sync just for initial test.

Alex

Alex Kushleyev

@mrawding , just to follow up on my suggestion regarding trying different lenses. If you cannot find a standalone lens option, consider an option of buying another camera module (like a OV5640 designed for ESP32 or similar) and using a lens from that. I can see that there are many options with different lenses and they look like M7 thread. You can try extracting the lens from that camera (may need to dissolve the glue that holds the lens in the threads) and test it on AR0144. The AR0144 has 1/4" sensor size and max CRA of 28 degrees.

Alex Kushleyev

hi @박민솔 , thank you for clarifying

A few follow up items:

The software-based slave id remap will work across camera server restarts because the Kernel will remap the ids during every camera power on. the cameras are powered on one at a time, so there is no race condition, so their slave ids can be remapped right after power on. If camera server is shut down and restarted, the Kernel will do this procedure again.
I am not sure if the change will make it into the SDK. The change should not affect any other camera including AR0144 in the standard modes of operation, but this would need to be tested. I will check with the team after testing.
Sure, the modified AR0144 is good as a fallback

AR0144 vs IMX412

Please keep in mind that AR0144, even though is wide angle, has a significant lens shading, making the image at the edges harder to use. We do have LSC (lens shading correction) in our camera server, but there is loss of quality at the edges
if you do overlap of AR0144 cameras for 360 views, you will have to do LSC so that the seams are not obvious. You probably have thought of that, just wanted to make sure
Synchronization: we are experimenting with hardware-based and software-based sync of hires cameras, so that may be an option in the near future to sync IMX412 to the same pulse that AR0144 is using for sync.
rolling shutter effects. yes, that is a valid concern. by the way, we have been working on EIS with rolling shutter compensation that may be interesting for you to consider for the future. This is IMX412. Hand-held test with lots of shaking : https://www.youtube.com/watch?v=3Yh9mOs0apo
in any case let's continue with the 5x AR0144, i hope to do some testing later today.

We are out of stock of Color AR0144, and there is currently no plan to make more, unfortunately, due to low demand.

Alex

Vinny

Hi @김다연
On that same page is this link

It should have all the info you need.

Alex Kushleyev

@박민솔 , i also wanted to double check. Is there a reason why you want 5x AR0144 and not a combination of AR0144 and IMX412? IMX412 is a very nice camera and has a good lens and provides much higher resoution image (up to 4040x3040). Is it the extra weight or rolling shutter the reason to choose all AR0144 ?

LukeNow

Going to close this ticket out since it seems to have resolved the issue for the user

Alex Kushleyev

@juancho248

Sorry for the delay, please see the answers below:

@juancho248 said:

Which Starling 2 Max configuration provides the most suitable high-resolution RGB cameras for our application?

All versions of Starling 2 Max come with dual IMX412 cameras (one front-facing, one down-facing).

@juancho248 said:

Is the C29 configuration currently supported, and what are its camera and depth-sensing capabilities?

yes, c29 is supported. Note that TOF (depth sensor) does not work outdoors due to sunlight

@juancho248 said:

Can the RGB cameras be configured for downward-facing image acquisition?

The rear IMX412 is already down-facing. The front camera is forward-facing, so you would need to make modification to make it down-facing

@juancho248 said:

Can the platform capture full-resolution still images programmatically, rather than only video frames?

Yes, you can capture full resolution images, including RAW10 , which can be processed offline for best results. Processed YUV also supported.

@juancho248 said:

What camera controls are available through the SDK, including exposure, gain, image format, resolution, and capture timing?

exposure, gain can be changed in real time. resolution can only be changed before camera server is started, but we support multiple streams with multiple resolutions (so you could have one full resolution stream for sparse captures and lower resolution for video feed). SW zoom is also available for processed images (YUV and video).

@juancho248 said:

Can captured images be associated with timestamps and available vehicle pose or localization metadata?

yes, each image has metadata, including timestamp, exposure, gain. The timestamp is using linux monotonic time, which can be correlated with other data sources that are logged with same time source.

@juancho248 said:

What onboard storage options and image-transfer interfaces are supported?

built-in storage (EMMC 120GB split into two partitions, typically at least 80-90GB free on a fresh system install - dont remember the exact number). you could support USB-based storage as well. Transfer images can be done using the USB / ADB interfrace. The adb itself is slow (<5MB/s) but a usb-ethernet can be enabled on the same USB connection, that is typically ~35MB/s transfers. If you use wifi on the drone, that is also an option. wifi speed depends on typical factors.

@juancho248 said:

Can images be stored locally during a mission and transferred after landing without requiring continuous Internet connectivity?

Yes, as long as you don't run out of local storage. Note that RAW10 images from IMX412 at full resolution (4040x3040) are about 15MB each. YUV of the same resolution are 18MB each. You can also compress them to JPG on voxl2.

Please let us know if you have any other questions of if I missed anything.

Alex

Alex Kushleyev

Hi @박민솔 ,

There is a way to change the AR0144 slave address in software (after camera hardware reset) so that to identical modules can coexist on the same CCI bus. In short, since the sensors are powered on one by one, during the AR0144 power on, the slave address can be changed (by the kernel) from the default address (0x30) to a new value, so that the camera no longer responds to the default value. So, the two conflicting AR0144 sensors would have their slave addresses changed to a different non-default value, then used as usual.

We already tested this in another setup, but I am going to test this feature. A kernel change is required, but it is not significant. This would simplify things if it works. However, we do have the modified AR0144 cameras to order if you still wanted to go that route.

Alex Kushleyev

Alright, i will try to get this done by end of this week.

A few notes:

Lepton and Boson can actually run together, so i will try that (since Lepton uses SPI, not MIPI protocol)
I will be testing with M0173 with resistor fix due to availability on my end
M0153 note about Voxl2 J7 is outdated. It works on Voxl2 J8 as well, in fact M0181 best fits into J8 and mounting hole aligns, see image below
Boson and AR0144 have different CCI slave addresses, so they will not conflict
i will double check how to purchace the ALT address AR0144
will try to release the AR0144 drivers at the same time as i test, hopefully by end of this week

ModalAI Forum

Global Moderators

Posts

Member List