Starling 2 loses all cameras; voxl-camera-server -l reports 0 cameras even after voxl-configure-cameras 27, voxl-configure-mpa, and reflash

irw

Hi ModalAI team,

I’m seeing an intermittent but recurring camera failure on a Starling 2. Last night I reflashed and the cameras started working again temporarily, but the issue came back. The vehicle did not crash; it stopped flying unexpectedly and was set down gently. The camera subsystem is essential for my use case. Any help is much appreciated.

What hardware are you using? If you have specific SKU or serial number, that is also helpful

I’m using a Starling 2 / D0014 with VOXL2 / M0054.

Current reported SKU info:

family code: MRB-D0014 (starling-2)
compute board: 4 (voxl2)
hw version: 1
cam config: 27
tx config: 9 (ghost)
SKU: MRB-D0014-4-V1-C27-T9-M0-X0
What version of software are you using? voxl-version is very helpful for determining software version

I have seen the issue on multiple software states:

Before reflash:

system-image: 1.7.8-M0054-14.1a-perf
voxl-suite: 1.3.0

After reflash:

system-image: 1.8.06-M0054-14.1a-perf
voxl-suite: 1.6.2

So this issue is not confined to one single software version.

How have you configured the software? What is the output of voxl-inspect-services

I ran the following:

voxl-inspect-sku
voxl-configure-cameras 27
voxl-configure-mpa
reboot

Those steps completed successfully.

After reboot, voxl-inspect-services shows many services running, but voxl-camera-server is Enabled and Not Running.

Relevant part of voxl-inspect-services:

Also, the current /etc/modalai/voxl-camera-server.conf is the generated 5-camera C27 config.

Do you have any logs? Uploading PX4 logs to https://logs.px4.io
can be very helpful.

Logs are attached in problem.txt: https://limewire.com/d/wiHXP#Bn7w1f14lV

The main issue is that the camera HAL does not seem to detect any cameras at all.

voxl-camera-server -l gives:

DEBUG: Attempting to open the hal module
DEBUG: SUCCESS: Camera module opened on attempt 0
DEBUG: ----------- Number of cameras: 0

DEBUG: Note: This list comes from the HAL module and may not be indicative
DEBUG: of configurations that have full pipelines

DEBUG: Number of cameras: 0

journalctl -u voxl-camera-server shows that the server loads the expected multi-camera config, but then exits with:

ERROR: Found zero cameras connected, need at least one.

dmesg / kernel logs show repeated low-level camera bring-up failures, including:

cam_sensor_cci_i2c_read ... rc = -19
cam_vana get failed -517
cam_vio get failed -517
sensor ID reads of 0x0 instead of expected IDs such as:
0x577
0x356
0x2975

This makes it look like the issue is below ROS and below the camera-server JSON config, since the HAL itself reports zero detected cameras.

One more detail: a full reflash restored camera functionality temporarily once, but the failure later returned about an hour later.

Can you share pictures of your setup?

Attached: https://limewire.com/d/wiHXP#Bn7w1f14lV

Have you looked at the source code? If so, can you point to any potential issues?

I have not dug deeply into the source code.

Based on the logs, my guess is that this is not a simple user-space config issue, because:

voxl-configure-cameras 27 completes successfully
voxl-configure-mpa completes successfully
reflash temporarily helped once
but voxl-camera-server -l still sometimes returns 0 cameras
kernel logs show regulator / I2C / sensor probe failures

Summary

The issue is an intermittent full camera bring-up failure on a Starling 2. The camera subsystem sometimes completely disappears, voxl-camera-server -l reports 0 cameras, and voxl-camera-server will not start. I have already tried:

voxl-configure-cameras 27
voxl-configure-mpa
reboot
full reflash

Reflash temporarily restored functionality once, but the issue returned.

Does this look like a known intermittent M0173/front-end/camera bring-up issue, or is there another non-reflash diagnostic path you recommend?

Thanks.

Alex Kushleyev

Hi @irw ,

This seems like a hardware issue, potentially related to PMIC. Are you able to share full dmesg output from boot, when this issue is happening?

One thing to try would be to disconnect M0173 board from VOXL2 and see if you can reproduce the same messages related to cam_sensor*power_up -- these are definitely not normal. Perhaps there is a short on one of the camera power rails, which is preventing PMIC from working properly.

Alex

irw

@Alex-Kushleyev Hi Alex thanks for your insight.

Here's the full dmesg output: https://limewire.com/d/QEPzc#F58jn5myhd when this issue is happening.
I am currently disconnecting the M0173 board, I will share the same debug outputs once the board is removed as soon as I can.

Cheeers