AprilTag relocalization fails with down-facing camera due to camera extrinsics mismatch (roll/pitch out of bounds)
-
Re: Apriltag relocalization not relocalizing?
Hello, I have the following issue similar to one above about apriltag relocalization with fixed frame:
Platform / Version:
Starling 2 (C27)
voxl-suite v1.5.0
Front-facing camera used for VIO
Down-facing camera used for AprilTag detectionIssue
AprilTag relocalization works correctly when the tag detector runs on the front-facing camera, but consistently fails when running on the down-facing camera, even though:- The down-facing camera intrinsics are correct
- The camera→body extrinsics in /etc/modalai/extrinsics.conf are correct
- R_tag_to_fixed is correctly set for a flat ground tag R_tag_to_fixed= [[0, -1, 0], [1, 0, 0], [0, 0, 1]]
- The tag is physically flat and the vehicle is level
- With the down-facing camera, voxl-vision-hub repeatedly reports: "WARNING, apriltag roll/pitch out of bounds" does not relocalize.
Confirmation
Using the same tag, same fixed-frame configuration, and same environment:- Detection from front camera → relocalization succeeds.
- Detection from down-facing camera → relocalization rejected.
- Swapping only the camera input is sufficient to reproduce the failure.
Conclusion
This strongly suggests that the AprilTag relocalization path in voxl-vision-hub is composing R_tag_to_cam → R_cam_to_imu/body using the wrong camera extrinsics (appears to default to the front/tracking camera), rather than the extrinsics corresponding to the camera that actually produced the detection. As a result, the inferred R_tag_to_local ∘ R_fixed_to_tag contains spurious roll/pitch and is rejected by the ±10° gate.Could you please solve the issue in new SDK? (You might add additional parameters in /etc/modalai/voxl-vision-hub.conf to select fixed frame pipe)
Best,
Kağan