Teledyne FLIR Lepton 3.5 Integration

[Alex Kushleyev]

@george-kollamkulam , the lepton tracker is an implementation of optic flow tracker for the lepton data, you can see the details here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-lepton-tracker/-/blob/master/src/feature_tracker.cpp

[george kollamkulam]

@Alex-Kushleyev I tried using the Lepton 3.5 sensor again, but this time I was unable to see the feed on the voxl portal, although voxl-lepton-server and voxl-camera-services were running. Additionally, I couldn't hear the clicking sound from the lepton sensor that was previously present. What could be the issue here? Is there anything I'm missing?

I also noticed that the rear tracking camera wasn't functional (I'm not sure if this is related to the functioning of the lepton camera). It was seen on the voxl system check page on the voxl portal.

[Alex Kushleyev]

@george-kollamkulam , are you saying that just the Lepton and the tracking cameras are not working now, but all the other cameras are working fine?

voxl-camera-server and voxl-lepton-server are completely independent and should not be interfering. However, Please try to unplug the lepton sensor from M0173 board and see if the tracking camera starts working again.

it seems like maybe there is some power issue, since lepton is not clicking, since it should be doing that as soon as it gets power.

Alex

[george kollamkulam]

@Alex-Kushleyev I did a power cycle on the lepton sensor and it seems to be working now (click sound and feed available). All cameras except the rear tracking camera seems to be working fine (screenshot attached below). It doesn't show up on the voxl portal. How can we resolve this?

[Alex Kushleyev]

@george-kollamkulam , can you please provide output of “voxl-camera-server -l”? This will show us which cameras are detected (excluding lepton). Note that running this command will stop the camera server if it is running in the background.

[george kollamkulam]

@Alex-Kushleyev
existing instance of voxl-camera-server found, attempting to stop it
Number of cameras detected: 4

On the voxl portal system check, it returned false for tracking_rear (VOXL-Camera-Server) and error for imageSensor2 (VOXL Image Sensors).

[Alex Kushleyev]

@george-kollamkulam ,

You are using a version of SDK which had a bug that was suppressing most of the prints, so voxl-camera-server -l is not reporting any information.

To fix this, please use the latest version of libmodal-journal package (you can get it from here : http://voxl-packages.modalai.com/dists/qrb5165/dev/binary-arm64//libmodal-journal_0.2.6-202505081055_arm64.deb

How many cameras do you actually have on your Starling 2? is it 4? Two tracking, one Hires camera and on TOF?

Alex

[george kollamkulam]

@Alex-Kushleyev apart from the lepton sensor, I have 5 (1 hires, 1 tof and 3 tracking).

Can you please point me to the resource that explains how to update the libmodal-journal package? Would I need to reinstall ROS2 Foxy or anything else?

Thanks,
George

[Alex Kushleyev]

@george-kollamkulam , in order to install the updated libmodal-journal package, you just download the file that i linked to VOXL2, lets say you put it in /home/root/.

Then, using adb you go to /home/root and execute:

dpkg -i libmodal-journal_0.2.6-202505081055_arm64.deb

Then you just run voxl-camera-server -l and you should see a lot more information being printed.

Alex

[george kollamkulam]

@Alex-Kushleyev here's the system response.

voxl2:/$ cd /home/root
voxl2:~$ dpkg -i libmodal-journal_0.2.6-202505081055_arm64.deb
(Reading database ... 137565 files and directories currently installed.)
Preparing to unpack libmodal-journal_0.2.6-202505081055_arm64.deb ...
Unpacking libmodal-journal (0.2.6-202505081055) over (0.2.5) ...
Setting up libmodal-journal (0.2.6-202505081055) ...
Processing triggers for libc-bin (2.27-3ubuntu1.6) ...
voxl2:~$ dpkg -l | grep libmodal-journal
ii libmodal-journal 0.2.6-202505081055 arm64 Journaling library
voxl2:~$ voxl-camera-server -l
existing instance of voxl-camera-server found, attempting to stop it
DEBUG: Attempting to open the hal module
DEBUG: SUCCESS: Camera module opened on attempt 0
DEBUG: ----------- Number of cameras: 4

DEBUG: Cam idx: 0, Cam slot: 0, Slave Address: 0x0030, Sensor Id: 0x0356
DEBUG: Cam idx: 1, Cam slot: 1, Slave Address: 0x0034, Sensor Id: 0x0577
DEBUG: Cam idx: 2, Cam slot: 3, Slave Address: 0x007A, Sensor Id: 0x2975
DEBUG: Cam idx: 3, Cam slot: 6, Slave Address: 0x0030, Sensor Id: 0x0356
DEBUG: Note: This list comes from the HAL module and may not be indicative
DEBUG: of configurations that have full pipelines

DEBUG: Number of cameras: 4

====================================
Stats for camera: 0

ANDROID_SCALER_AVAILABLE_RAW_SIZES:
These are likely supported by the sensor
1280 x 800
1280 x 800

ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS:
These are NOT necessarily supported by the sensor
1280 x 800 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 800 HAL_PIXEL_FORMAT_BLOB
1280 x 768 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 768 HAL_PIXEL_FORMAT_BLOB
1280 x 720 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 720 HAL_PIXEL_FORMAT_BLOB
1024 x 738 HAL_PIXEL_FORMAT_YCbCr_420_888
1024 x 738 HAL_PIXEL_FORMAT_BLOB
1024 x 768 HAL_PIXEL_FORMAT_YCbCr_420_888
1024 x 768 HAL_PIXEL_FORMAT_BLOB
864 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
864 x 480 HAL_PIXEL_FORMAT_BLOB
800 x 600 HAL_PIXEL_FORMAT_YCbCr_420_888
800 x 600 HAL_PIXEL_FORMAT_BLOB
800 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
800 x 480 HAL_PIXEL_FORMAT_BLOB
720 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
720 x 480 HAL_PIXEL_FORMAT_BLOB
640 x 512 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 512 HAL_PIXEL_FORMAT_BLOB
640 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 480 HAL_PIXEL_FORMAT_BLOB
640 x 400 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 400 HAL_PIXEL_FORMAT_BLOB
640 x 360 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 360 HAL_PIXEL_FORMAT_BLOB
352 x 288 HAL_PIXEL_FORMAT_YCbCr_420_888
352 x 288 HAL_PIXEL_FORMAT_BLOB
320 x 240 HAL_PIXEL_FORMAT_YCbCr_420_888
320 x 240 HAL_PIXEL_FORMAT_BLOB
240 x 320 HAL_PIXEL_FORMAT_YCbCr_420_888
240 x 320 HAL_PIXEL_FORMAT_BLOB
176 x 144 HAL_PIXEL_FORMAT_YCbCr_420_888
176 x 144 HAL_PIXEL_FORMAT_BLOB
1280 x 800 HAL_PIXEL_FORMAT_RAW10
1280 x 800 HAL_PIXEL_FORMAT_RAW12
1280 x 800 HAL_PIXEL_FORMAT_RAW16
1280 x 800 HAL_PIXEL_FORMAT_RAW_OPAQUE

ANDROID_SENSOR_INFO_SENSITIVITY_RANGE
min = 54
max = 1596

ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY
1596

ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE
min = 0ns
max = 1318629735ns

====================================
Stats for camera: 1

ANDROID_SCALER_AVAILABLE_RAW_SIZES:
These are likely supported by the sensor
4056 x 3040
3840 x 2160
1920 x 1080
1920 x 1080
1920 x 1080
1280 x 720
1280 x 720

ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS:
These are NOT necessarily supported by the sensor
4056 x 3040 HAL_PIXEL_FORMAT_YCbCr_420_888
4056 x 3040 HAL_PIXEL_FORMAT_BLOB
4000 x 3000 HAL_PIXEL_FORMAT_YCbCr_420_888
4000 x 3000 HAL_PIXEL_FORMAT_BLOB
3840 x 2160 HAL_PIXEL_FORMAT_YCbCr_420_888
3840 x 2160 HAL_PIXEL_FORMAT_BLOB
3264 x 2448 HAL_PIXEL_FORMAT_YCbCr_420_888
3264 x 2448 HAL_PIXEL_FORMAT_BLOB
3200 x 2400 HAL_PIXEL_FORMAT_YCbCr_420_888
3200 x 2400 HAL_PIXEL_FORMAT_BLOB
2976 x 2976 HAL_PIXEL_FORMAT_YCbCr_420_888
2976 x 2976 HAL_PIXEL_FORMAT_BLOB
2688 x 1512 HAL_PIXEL_FORMAT_YCbCr_420_888
2688 x 1512 HAL_PIXEL_FORMAT_BLOB
2592 x 1944 HAL_PIXEL_FORMAT_YCbCr_420_888
2592 x 1944 HAL_PIXEL_FORMAT_BLOB
2048 x 1536 HAL_PIXEL_FORMAT_YCbCr_420_888
2048 x 1536 HAL_PIXEL_FORMAT_BLOB
1920 x 1440 HAL_PIXEL_FORMAT_YCbCr_420_888
1920 x 1440 HAL_PIXEL_FORMAT_BLOB
1928 x 1208 HAL_PIXEL_FORMAT_YCbCr_420_888
1928 x 1208 HAL_PIXEL_FORMAT_BLOB
1920 x 1080 HAL_PIXEL_FORMAT_YCbCr_420_888
1920 x 1080 HAL_PIXEL_FORMAT_BLOB
1600 x 1200 HAL_PIXEL_FORMAT_YCbCr_420_888
1600 x 1200 HAL_PIXEL_FORMAT_BLOB
1440 x 1080 HAL_PIXEL_FORMAT_YCbCr_420_888
1440 x 1080 HAL_PIXEL_FORMAT_BLOB
1280 x 960 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 960 HAL_PIXEL_FORMAT_BLOB
1280 x 800 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 800 HAL_PIXEL_FORMAT_BLOB
1280 x 768 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 768 HAL_PIXEL_FORMAT_BLOB
1280 x 720 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 720 HAL_PIXEL_FORMAT_BLOB
1080 x 1080 HAL_PIXEL_FORMAT_YCbCr_420_888
1080 x 1080 HAL_PIXEL_FORMAT_BLOB
1024 x 738 HAL_PIXEL_FORMAT_YCbCr_420_888
1024 x 738 HAL_PIXEL_FORMAT_BLOB
1024 x 768 HAL_PIXEL_FORMAT_YCbCr_420_888
1024 x 768 HAL_PIXEL_FORMAT_BLOB
864 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
864 x 480 HAL_PIXEL_FORMAT_BLOB
800 x 600 HAL_PIXEL_FORMAT_YCbCr_420_888
800 x 600 HAL_PIXEL_FORMAT_BLOB
800 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
800 x 480 HAL_PIXEL_FORMAT_BLOB
720 x 1280 HAL_PIXEL_FORMAT_YCbCr_420_888
720 x 1280 HAL_PIXEL_FORMAT_BLOB
720 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
720 x 480 HAL_PIXEL_FORMAT_BLOB
640 x 512 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 512 HAL_PIXEL_FORMAT_BLOB
640 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 480 HAL_PIXEL_FORMAT_BLOB
640 x 400 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 400 HAL_PIXEL_FORMAT_BLOB
640 x 360 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 360 HAL_PIXEL_FORMAT_BLOB
352 x 288 HAL_PIXEL_FORMAT_YCbCr_420_888
352 x 288 HAL_PIXEL_FORMAT_BLOB
320 x 240 HAL_PIXEL_FORMAT_YCbCr_420_888
320 x 240 HAL_PIXEL_FORMAT_BLOB
240 x 320 HAL_PIXEL_FORMAT_YCbCr_420_888
240 x 320 HAL_PIXEL_FORMAT_BLOB
176 x 144 HAL_PIXEL_FORMAT_YCbCr_420_888
176 x 144 HAL_PIXEL_FORMAT_BLOB
4056 x 3040 HAL_PIXEL_FORMAT_RAW10
4056 x 3040 HAL_PIXEL_FORMAT_RAW12
4056 x 3040 HAL_PIXEL_FORMAT_RAW16
4056 x 3040 HAL_PIXEL_FORMAT_RAW_OPAQUE
3840 x 2160 HAL_PIXEL_FORMAT_RAW10
3840 x 2160 HAL_PIXEL_FORMAT_RAW12
3840 x 2160 HAL_PIXEL_FORMAT_RAW16
3840 x 2160 HAL_PIXEL_FORMAT_RAW_OPAQUE
1920 x 1080 HAL_PIXEL_FORMAT_RAW10
1920 x 1080 HAL_PIXEL_FORMAT_RAW12
1920 x 1080 HAL_PIXEL_FORMAT_RAW16
1920 x 1080 HAL_PIXEL_FORMAT_RAW_OPAQUE

ANDROID_SENSOR_INFO_SENSITIVITY_RANGE
min = 100
max = 35200

ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY
2200

ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE
min = 0ns
max = 243753000ns

====================================
Stats for camera: 2

ANDROID_SCALER_AVAILABLE_RAW_SIZES:
These are likely supported by the sensor
240 x 1629

ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS:
These are NOT necessarily supported by the sensor
240 x 320 HAL_PIXEL_FORMAT_YCbCr_420_888
240 x 320 HAL_PIXEL_FORMAT_BLOB
176 x 144 HAL_PIXEL_FORMAT_YCbCr_420_888
176 x 144 HAL_PIXEL_FORMAT_BLOB
240 x 1629 HAL_PIXEL_FORMAT_RAW10
240 x 1629 HAL_PIXEL_FORMAT_RAW12
240 x 1629 HAL_PIXEL_FORMAT_RAW16
240 x 1629 HAL_PIXEL_FORMAT_RAW_OPAQUE

ANDROID_SENSOR_INFO_SENSITIVITY_RANGE
min = 54
max = 0

ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY
0

ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE
min = 0ns
max = 0ns

====================================
Stats for camera: 3

ANDROID_SCALER_AVAILABLE_RAW_SIZES:
These are likely supported by the sensor
1280 x 800
1280 x 800

ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS:
These are NOT necessarily supported by the sensor
1280 x 800 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 800 HAL_PIXEL_FORMAT_BLOB
1280 x 768 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 768 HAL_PIXEL_FORMAT_BLOB
1280 x 720 HAL_PIXEL_FORMAT_YCbCr_420_888
1280 x 720 HAL_PIXEL_FORMAT_BLOB
1024 x 738 HAL_PIXEL_FORMAT_YCbCr_420_888
1024 x 738 HAL_PIXEL_FORMAT_BLOB
1024 x 768 HAL_PIXEL_FORMAT_YCbCr_420_888
1024 x 768 HAL_PIXEL_FORMAT_BLOB
864 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
864 x 480 HAL_PIXEL_FORMAT_BLOB
800 x 600 HAL_PIXEL_FORMAT_YCbCr_420_888
800 x 600 HAL_PIXEL_FORMAT_BLOB
800 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
800 x 480 HAL_PIXEL_FORMAT_BLOB
720 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
720 x 480 HAL_PIXEL_FORMAT_BLOB
640 x 512 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 512 HAL_PIXEL_FORMAT_BLOB
640 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 480 HAL_PIXEL_FORMAT_BLOB
640 x 400 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 400 HAL_PIXEL_FORMAT_BLOB
640 x 360 HAL_PIXEL_FORMAT_YCbCr_420_888
640 x 360 HAL_PIXEL_FORMAT_BLOB
352 x 288 HAL_PIXEL_FORMAT_YCbCr_420_888
352 x 288 HAL_PIXEL_FORMAT_BLOB
320 x 240 HAL_PIXEL_FORMAT_YCbCr_420_888
320 x 240 HAL_PIXEL_FORMAT_BLOB
240 x 320 HAL_PIXEL_FORMAT_YCbCr_420_888
240 x 320 HAL_PIXEL_FORMAT_BLOB
176 x 144 HAL_PIXEL_FORMAT_YCbCr_420_888
176 x 144 HAL_PIXEL_FORMAT_BLOB
1280 x 800 HAL_PIXEL_FORMAT_RAW10
1280 x 800 HAL_PIXEL_FORMAT_RAW12
1280 x 800 HAL_PIXEL_FORMAT_RAW16
1280 x 800 HAL_PIXEL_FORMAT_RAW_OPAQUE

ANDROID_SENSOR_INFO_SENSITIVITY_RANGE
min = 54
max = 1596

ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY
1596

ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE
min = 0ns
max = 1318629735ns

====================================
Number of cameras detected: 4