Posts made by Alex Kushleyev

@Jordyn-Heil , you will need to use a recent version of voxl-camera-server (you can build it from dev or get a package from the nightly builds).

Then edit the camera server config file (/etc/modalai/voxl-camera-server.conf) and replace the camera type ar0144 with ar0144-color. If you get an error (unknown camera type), it means you are using older version of camera server.

Currently, the debayering is happening on the CPU.

Here is the place where processing is happening for the color version of AR0144. https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/hal3_camera_mgr.cpp?ref_type=heads#L1477

• debayer from GRBG to RGB
• run AWB (auto white balance)
• run AE (auto exposure)
• send out RGB, grey, and original raw bayer frames

Please try it out.

Alex

@Idan-Fiksel , it looks like it worked since your machine variant is now 1.0.0 . I assume it was 1.0.1. Sorry the numbering in the name of the kernel is 00.0 vs 00.1…

@Idan-Fiksel , yes two identical cameras (TOF in this case) should work on J6L and J8L. Two things to check in this case (I already verified):

• make sure there is no CCI (I2C) slave address conflict, in other words the camera ports using different CCI busses. I verified this using https://docs.modalai.com/voxl2-connectors/ - J6L uses CCI0 bus (CAM0_CCI0) and J8L uses CCI1 bus (CAM4_CCI1)
• make sure the camera driver is present for the camera slots you need (0 and 4):

ls /usr/share/modalai/chi-cdk/irs2975c
com.qti.sensormodule.irs2975c_0.bin  com.qti.sensormodule.irs2975c_2.bin  com.qti.sensormodule.irs2975c_4.bin
com.qti.sensormodule.irs2975c_1.bin  com.qti.sensormodule.irs2975c_3.bin  com.qti.sensormodule.irs2975c_5.bin

The last thing you may need to limit the maximum "exposure" of the TOF sensor (maximum emitted power) using voxl-camera-server.conf in order to prevent drawing too much current (in bursts. This will depend on other load on VOXL2's power supply, etc and will need testing to make sure that VOXL2's power supply is not affected by drawing too much current under full system load.

Please consult the following document for the power consumption and different operating modes of the PMD TOF sensor : https://docs.modalai.com/M0178/#currentpower-consumption

Alex

I also double checked the schematic and did not notice anything wrong. Please follow Vinny’s suggestions and definitely check if the Lepton is making the clicking sound, which would mean it is powered on, it should be clicking periodically in default mode after power on.

@Jetson-Nano , this looks like a sensor issue. VOXL2 does not do any configuration of the sensor's parameters, other than it can change the calibration interval based on config file, I believe.

What you are seeing appears to be artifacts in the sensor itself, maybe due to noise on power rail(s). This is not an expected behavior, but it should not be caused by a SW configuration from the VOXL2 side.

Alex

The latest firmware for M0065 can be found on gitlab here

or directly on voxl2 in /usr/share/modalai/voxl2-io-tools/firmware/

Once updated, the firmware will be detected as version 2. The previous version 1 is no longer supported due to several bugs.

Alex

@daz_22 , is there any motion at all in the motor when you try to spin it up using voxl-esc-spin commands? In other words, is the motor completely stationary or does it look like it's attempting to start?

Alex

@QSL , are you using kernel variant 0 or 1?

you can check using voxl-version, whether you see mach var 1.0.0 or 1.0.1.

1.0.1 kernel is used with M0173 Starling Camera Front End due to I/O pin configuration specific to M0173.

If you are not using M0173, for example when you use M0084 connected to VOXL2 J7, you need to switch to kernel variant 0. by either re-installing the VOXL2 SDK or relfashing the correct kernel from the SDK.

cd voxl2_SDK_<voxl-sdk-version>/system-image
adb reboot bootloader

#flash variant 0
fastboot flash boot_a m0054-1-var00.0-kernel.img
fastboot flash boot_b m0054-1-var00.0-kernel.img

#or flash variant 1
fastboot flash boot_a m0054-1-var00.1-kernel.img
fastboot flash boot_b m0054-1-var00.1-kernel.img

#reboot
fastboot reboot

Alex

@QSL

For connecting the second TOF to VOXL2, you would need to use VOXL2's J8 port because J6 and J7 will be taken up by the M0173 with the three front cameras.

So you need M0172 plugged into J8 and then plug in M0177 + M0171/M0178 interposer + (actual TOF sensor) assembly into M0172. You can also use an extension M0170 instead of M0177 (https://www.modalai.com/products/msu-m0170).

using M0177:

(voxl2 J8) <-> (M0172 J1L) <-> (M0177) <-> (M0171 / M0178) <-> (TOF Module)

using M0170

(voxl2 J8) <-> (M0172 J1L) <-> (M0170) <-> (M0171 / M0178) <-> (TOF Module)

Please keep in mind that M0177 and M0171 / M0178 and TOF module are usually sold as one unit. Also, M0171 and M0178 are identical except for mounting tabs which can be removed from one to make the other.

I hope that helps!

Alex

M0172

M0177

M0171 / M0178

Actual TOF Sensor Module

M0170

@smiley ,

The Boson and Hadron are supported the same way by the VOXL2 camera pipeline (same drivers, camera server settings). You just need to use the correct cables / interposers for connecting the sensor depending on the selection of Boson or Hadron. There is no distinction at all from the VOXL2 SW and it does not know whether a standalone Boson or Boson inside Hadron is connected.

Please let me know if you have any other questions.

Alex

It looks like this issue was only in the SDK 1.4.4 . Other SDK releases should be OK. A bug in the printing function in libmodal-journal library caused the prints to stop showing up.

Alex

Initial docs are up : https://docs.modalai.com/camera-video/electronic-image-stabilization/

@JP-Drone , If your camera is flipped (rotated 180), you just need to update your extrinsics to tell EIS the orientation of camera w.r.t IMU:

https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/voxl2-eis.md?ref_type=heads#extrinsic-calibration

Instead of right side up

"RPY_parent_to_child":    [0, 90, 90]

your extrinsic file should be

"RPY_parent_to_child":    [0, 90, -90]

In other words, your /etc/modalai/extrinsics.conf should contain the following entry:

{
    "parent": "imu_apps",
    "child":  "<camera_name>",
    "T_child_wrt_parent": [0.0, 0.0, 0.0],
    "RPY_parent_to_child":    [0, 90, -90]
}

(I am assuming VOXL2 is mounted in normal orientation, right side up)

Please try.

Also, we recently merged the EIS feature to dev branch of camera server, still working on better documentation.

Alex

Here are two image samples taking using IMX412 and AR0144. IMX412 is slightly cropped on the sides, but it's also a slightly narrower lens. AR0144 image is without LSC correction. Also images slightly out of focus because the test pattern is very close to the camera. The brightness in the IMX412 image is more evenly distributed (larger, better lens), so the center of the image is not over-exposed, unlike the AR0144. This should be corrected with the proper LSC..

IMX412

AR0144-color

OK, I was able to confirm this configuration:

• VOXL2
• M0173 + 3x AR0144 + 1x IMX412 (C27 config)
• M0155 + 1x AR0144 plugged into J8
  • AR0144 i2c slave id change via resistor
  • small camera driver modification for new slave id
• also verified the synchronization between the 4 AR0144

The resistor swap was relatively easy, but i used a special "tweezer" soldering iron tip. I can be difficult without the right tools.. Let me check if we can do a swap for the custom order..

See the image with marked slave id select resistor that needs to move to a pad right above the current placement. It can be a bit tricky to do it

Alex

Hi @Jordyn-Heil ,

OK, so did some digging. Essentially, the 4x AR0144 + 1x IMX412 would be the same as C27 config (https://docs.modalai.com/voxl2-coax-camera-bundles/#mdk-m0173-1-02) + another AR0144 connected to J8 via M0155 (so that would be J8L, camera slot 4).

There are also some other options like moving IMX412 to J8, but in either case, unfortunately there is a change required in one of the AR0144 to switch the i2c slave address from default. This means moving a resistor on the camera from one pad to another.

I am going to double check this by actually switching the slave ID, but just wanted to give you an update..

Alex

@Jetson-Nano , generally it is not possible to cause a VOXL2 reboot during normal operation. There are a few cases which can cause a kernel panic and the board would reboot into a recovery mode for debugging. This only occurs during testing some experimental features and you would normally not see this behavior. If the board simply reboots and is running again after reboot, it is very likely a power failure.

The power cable, if wiggled a bit, can become temporarily normal again, so it is possible what happened to you. The main issue is that wiggling the power cable, in general, loosens the contacts that make contact with the pins, causing the contact pressure to weaken and contact resistance to increase.

Alex

@tonygurney , please see the doc page regarding GPIO and the offset:

https://docs.modalai.com/voxl2-linux-user-guide/#gpios

specifically,

Starting SDK 1.2.X/System Image 1.7.3, GPIOs are exported by default in /sys/class/gpio, the GPIO have an offest of 1100 (e.g. GPIO 84 = 1184).

One more thing.. Do you need global shutter cameras? or is a small size a requirement? You could use the IMX412 cameras, which are rolling shutter, but much higher resolution and color quality. Have you considered that option?

Alex

@Jordyn-Heil , for ordering, since it's a custom order, please send us a message via http://modalai.com/contact and provide the following part number. We will send you an invoice.

MSU-M0166-2-01

(which includes MCCA-M0166-2 + M10000858 lens (AR0144 Camera Color Coax))

Do you want to wait until I confirm 4x concurrency?

Regarding panoramic stitching, yes obviously lens shading will be an issue, however if you have 4 cameras, you only need 90 degrees of horizontal FOV per camera and the current lens provides 139 degrees of horizontal FOV (https://docs.modalai.com/M0166/). So i think 90 degrees + small overlap won't be too bad (but LSC correction will be required to achieve seamless stitching)
. Also, the VOXL2 GPU can do the stitching, I can help you get started (should be a fun project).

Here is an old example i found of original vs LSC corrected monochrome images:

original:

LSC corrected:

You see the corners are heavily quantized because they were so dark and pixel resolution is not enough. The original RAW image is 8 bit. The AR0144 camera does support 10 and 12 bit output, and the 12 bit data would help a lot with Quantization effects after correction. the 10 and 12 bit stream is experimental and there are some issues (will not go into detail here), that is why we using 8 bit, which is stable.

Alex