AprilTag Localization and landing

[Rupesh]

Hi,
I'm using Starling 2 Max drone. I'm trying to localize the drone with AprilTag. I followed all the instructions from the ModalAI documentation, but cannot get it to work. I feel like the instructions for Relocalization are incomplete. The voxl-inspect-tags is working perfectly, and voxl-inspect-services also shows everything working. I also changed the offboard mode to "follow_tag" as shown in the figure below. Can anyone help me with how to proceed? Or i must use docker and run the script file? If so, is offboard_follow_tag.c safe to use? Because the documentation says "offboard_follow_tag.c: (may be outdated and no longer functional)".

[ApoorvThapliyal]

@Rupesh Thanks for your question. Our subject matter expert is currently unavailable, but we will review this and get back to you within a few days.

[ApoorvThapliyal]

@Rupesh What rotation matrix are you currently using in your tag_locations.conf?
You can try setting it to:

[[0, 1, 0],
[0, 0, 1],
[1, 0, 0]]
Keep in mind that relocalization will only work reliably at certain viewing angles relative to the AprilTag. A good way to test is to rotate the tag and observe at which orientations voxl-vision-hub --debug_fixed_frame gives a value and not WARNING, apriltag roll/pitch out of bounds. Improvements are in progress to make this more robust across a wider range of angles.

[Rupesh]

@ApoorvThapliyal Thank you for your reply.
I tried voxl-vision-hub --debug_fixed_frame while showing AprilTag to the tracking camera with different orientations, but got some warnings about qvio.

Since this was a warning about qvio, I checked voxl-inspect-qvio, and it was working okay.

Additionally, I wanted to integrate AprilTag Localization with voxl2_takeoff_land.py. Is it possible to make it work if I add the following code between the 'takeoff' and 'landing' commands?
try:
await drone.offboard.set_position_ned(PositionNedYaw(0, 0, -1.0, 0))
await drone.offboard.start()
except OffboardError as e:
print(f"Offboard start failed: {e._result.result}")

await asyncio.sleep(20)

print("-- Stopping Offboard mode")
try:
    await drone.offboard.stop()
except OffboardError as e:
    print(f"Offboard stop failed: {e._result.result}")

I tried to operate it without propellers and got the following logs in PX4.

The set points appear to be at the origin instead of those defined by AprilTag. Can you please help me with this?

[Rupesh]

@ApoorvThapliyal, with the setting you provided, I tried flying with propellers on in offboard mode. Once I armed and switched to offboard mode, the drone took off and stayed at around 10 cm height for 10 seconds. During that time, it didn't detect AprilTag and failed to align, as the height was too low for detection. However, after 10 seconds, it suddenly began to gain height, and as it approached the ceiling, I took control and landed. From the log file, we can see that the drone was attempting to align its X and Y positions with AprilTag; however, the issue was that the Z setpoints continually decreased. Can you please help me solve this?

Additionally, it would be more beneficial if we could run it using MAVSDK and customize it for our specific code.

[ApoorvThapliyal]

@Rupesh Apologies for the late reply
It looks like there’s an issue with how the AprilTag system is relocalizing the drone. In my tests, the z-value appears to be misinterpreted, which could explain why your drone was climbing in altitude to correct its height estimate. We’re actively working on a fix.