AprilTag relocalization fails with down-facing camera due to camera extrinsics mismatch (roll/pitch out of bounds)
-
Re: Apriltag relocalization not relocalizing?
Hello, I have the following issue similar to one above about apriltag relocalization with fixed frame:
Platform / Version:
Starling 2 (C27)
voxl-suite v1.5.0
Front-facing camera used for VIO
Down-facing camera used for AprilTag detectionIssue
AprilTag relocalization works correctly when the tag detector runs on the front-facing camera, but consistently fails when running on the down-facing camera, even though:- The down-facing camera intrinsics are correct
- The camera→body extrinsics in /etc/modalai/extrinsics.conf are correct
- R_tag_to_fixed is correctly set for a flat ground tag R_tag_to_fixed= [[0, -1, 0], [1, 0, 0], [0, 0, 1]]
- The tag is physically flat and the vehicle is level
- With the down-facing camera, voxl-vision-hub repeatedly reports: "WARNING, apriltag roll/pitch out of bounds" does not relocalize.
Confirmation
Using the same tag, same fixed-frame configuration, and same environment:- Detection from front camera → relocalization succeeds.
- Detection from down-facing camera → relocalization rejected.
- Swapping only the camera input is sufficient to reproduce the failure.
Conclusion
This strongly suggests that the AprilTag relocalization path in voxl-vision-hub is composing R_tag_to_cam → R_cam_to_imu/body using the wrong camera extrinsics (appears to default to the front/tracking camera), rather than the extrinsics corresponding to the camera that actually produced the detection. As a result, the inferred R_tag_to_local ∘ R_fixed_to_tag contains spurious roll/pitch and is rejected by the ±10° gate.Could you please solve the issue in new SDK? (You might add additional parameters in /etc/modalai/voxl-vision-hub.conf to select fixed frame pipe)
Best,
Kağan -
Re: Apriltag relocalization not relocalizing?
Hello, I have the following issue similar to one above about apriltag relocalization with fixed frame:
Platform / Version:
Starling 2 (C27)
voxl-suite v1.5.0
Front-facing camera used for VIO
Down-facing camera used for AprilTag detectionIssue
AprilTag relocalization works correctly when the tag detector runs on the front-facing camera, but consistently fails when running on the down-facing camera, even though:- The down-facing camera intrinsics are correct
- The camera→body extrinsics in /etc/modalai/extrinsics.conf are correct
- R_tag_to_fixed is correctly set for a flat ground tag R_tag_to_fixed= [[0, -1, 0], [1, 0, 0], [0, 0, 1]]
- The tag is physically flat and the vehicle is level
- With the down-facing camera, voxl-vision-hub repeatedly reports: "WARNING, apriltag roll/pitch out of bounds" does not relocalize.
Confirmation
Using the same tag, same fixed-frame configuration, and same environment:- Detection from front camera → relocalization succeeds.
- Detection from down-facing camera → relocalization rejected.
- Swapping only the camera input is sufficient to reproduce the failure.
Conclusion
This strongly suggests that the AprilTag relocalization path in voxl-vision-hub is composing R_tag_to_cam → R_cam_to_imu/body using the wrong camera extrinsics (appears to default to the front/tracking camera), rather than the extrinsics corresponding to the camera that actually produced the detection. As a result, the inferred R_tag_to_local ∘ R_fixed_to_tag contains spurious roll/pitch and is rejected by the ±10° gate.Could you please solve the issue in new SDK? (You might add additional parameters in /etc/modalai/voxl-vision-hub.conf to select fixed frame pipe)
Best,
Kağan@kgn-mdlai Thank you for bringing this up, a fix to this is currently being worked on.
Thanks -
Re: Apriltag relocalization not relocalizing?
Hello, I have the following issue similar to one above about apriltag relocalization with fixed frame:
Platform / Version:
Starling 2 (C27)
voxl-suite v1.5.0
Front-facing camera used for VIO
Down-facing camera used for AprilTag detectionIssue
AprilTag relocalization works correctly when the tag detector runs on the front-facing camera, but consistently fails when running on the down-facing camera, even though:- The down-facing camera intrinsics are correct
- The camera→body extrinsics in /etc/modalai/extrinsics.conf are correct
- R_tag_to_fixed is correctly set for a flat ground tag R_tag_to_fixed= [[0, -1, 0], [1, 0, 0], [0, 0, 1]]
- The tag is physically flat and the vehicle is level
- With the down-facing camera, voxl-vision-hub repeatedly reports: "WARNING, apriltag roll/pitch out of bounds" does not relocalize.
Confirmation
Using the same tag, same fixed-frame configuration, and same environment:- Detection from front camera → relocalization succeeds.
- Detection from down-facing camera → relocalization rejected.
- Swapping only the camera input is sufficient to reproduce the failure.
Conclusion
This strongly suggests that the AprilTag relocalization path in voxl-vision-hub is composing R_tag_to_cam → R_cam_to_imu/body using the wrong camera extrinsics (appears to default to the front/tracking camera), rather than the extrinsics corresponding to the camera that actually produced the detection. As a result, the inferred R_tag_to_local ∘ R_fixed_to_tag contains spurious roll/pitch and is rejected by the ±10° gate.Could you please solve the issue in new SDK? (You might add additional parameters in /etc/modalai/voxl-vision-hub.conf to select fixed frame pipe)
Best,
Kağan@kgn-mdlai Unfortunately, the April Tag code is quite outdated and hasn't been tested on any of our latest hardware. As it is now it is provided as reference code and if you are able to get things working correctly on your own we are happy to accept PRs to the repo. We do plan an update of this in a future revision of the SDK but there is no scheduled date that we can provide right now.
-
@kgn-mdlai Unfortunately, the April Tag code is quite outdated and hasn't been tested on any of our latest hardware. As it is now it is provided as reference code and if you are able to get things working correctly on your own we are happy to accept PRs to the repo. We do plan an update of this in a future revision of the SDK but there is no scheduled date that we can provide right now.
@Eric-Katzfey
────────────────────────────────────────────────────────────────────────────────
system-image: 1.8.02-M0054-14.1a-perf
kernel: #1 SMP PREEMPT Mon Nov 11 22:47:44 UTC 2024 4.19.125
────────────────────────────────────────────────────────────────────────────────
hw platform: M0054
mach.var: 1.0.1
SKU: MRB-D0014-4-V1-C27-T9-M22-X0
────────────────────────────────────────────────────────────────────────────────
voxl-suite: 1.4.5
────────────────────────────────────────────────────────────────────────────────
Packages:I am getting this issue as well, where I constantly get WARNING, apriltag roll/pitch out of bounds issue with the front or down tracking camera.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login