Outdoor metric depth + velocity estimation on Starling 2 Max for target tracking

[Masa Chau]

Hi all,

I'm a researcher evaluating the Starling 2 Max for an outdoor + indoor target-tracking project, and I'd appreciate guidance from the team and anyone running a similar setup before we order.

Our onboard pipeline is a YOLO detector, a BoT-SORT tracker, and a Kalman filter that estimates a moving target's position and velocity, with a small RL policy for guidance.

From reading the docs and forum, here's my current understanding, and I want to check I have it right:

Outdoor depth: The PMD ToF is indoor-only. Depth-from-stereo (voxl-dfs-server) exists, but the supported stereo hardware is EOL and the Starling 2 Max's color cameras are rolling shutter, so DFS isn't really an option on this airframe. That seems to leave monocular depth-from-mono for outdoor metric range. Is that the current recommended path, or is there a supported stereo/other option I've missed? For those using depth-from-mono outdoors, what usable range and accuracy are you seeing on a moving target at around 10 to 30 metres?

Velocity estimation: For estimating a target's relative velocity outdoors, what would you recommend fusing? I'm aware VIO (OpenVINS / qVIO) handles ego-motion, and I've seen the thread where running both servers at once produced bad velocity output, so I'll pin to one. My question is more about relative-velocity-to-target: is the expectation that we get target range from a depth model and differentiate through the Kalman filter, or is there a better-supported approach on VOXL?

GPS: I've read the EMI threads (GPS relocation to the rear, Lepton cable disconnect, wifi dongle offset, shielding/mast, the V3 GNSS changes). For a unit ordered today, is the EMI issue considered resolved in current hardware, or should we still plan to apply mitigations? Does that change with a 5G modem config?

For context we'd be on a C28 camera config (no ToF), WiFi modem, developing a custom stack via the SDK. Any pointers from people doing outdoor tracking work on this platform would be hugely appreciated.

Thanks!
Masa

[DavidTorres]

Hi @masa-chau,

It seems you are on the right track. A few pointers from my side:

First, what are you actually trying to achieve? If you can avoid tracking in 3D and perform the task in 2D, I would strongly recommend doing that. Recovering 3D position from 2D observations is quite hard and can be very finicky/unreliable. For many tracking tasks, you can get away with tracking entirely in image space.

If you really need to track in 3D:

• Use the PX4 estimator output. VIO is fused into EKF2 in PX4, so I would probably use the PX4 estimated poses/velocities as the state of the drone rather than trying to fuse the raw VIO yourself. I believe these should be available in ROS 2 through voxl-microdds-agent, although someone from ModalAI should confirm the exact topics/data being exposed.

• Your 2D observations from the object detector are fine, but pay close attention to latency. Also, I would not expect reliable detection at very long distances with the Starling 2 Max fisheye camera; in practice, the useful detection range may be significantly below what you might initially expect.

• I would not rely on the ToF camera for this.

• For the actual tracking algorithm, I would look at an EKF or MAP estimator that fuses the 2D observations (e.g. bearing and potentially bounding-box size) with the drone's 3D pose/velocity. The important point is that a single 2D observation does not give you reliable depth. To estimate the object's distance over time, you need additional constraints, typically through a motion model or assumptions about how the tracked object moves.

Whether the object is flying or driving, you will probably need to make some approximation about its motion. I would look into the existing literature on monocular 3D object tracking / bearing-only tracking for this part.

Good luck!