The related config is very basic:

    "en_apriltag_fixed_frame": true,
    "fixed_apriltags": [
        {
            "tag_id": 1,
            "R_tag_to_fixed": [
                [0, -1, 0],
                [1, 0, 0],
                [0, 0, 1]
            ],
            "T_tag_wrt_fixed": [0, 0, 0]
        }
    ],
    "en_transform_mavlink_pos_setpoints_from_fixed_frame": true,
    "en_fixed_frame_pipe": false,

I set the tag-size to 0.16, as each square of the 8x8 tag grid is 2cm. A change of this parameter never had an effect to the detected tag position from voxl-vision-px4, which I find very strange.

When I run voxl-vision-px4 -p for debug odometry in fixed frame I get lots of out of bounds warnings. A "first-detection" with a followed position "reset" of fixed frame does not happen.

T_body_wrt_fixed: -0.00  0.00 -0.20  RPY:  0.00  0.00 -0.00
T_body_wrt_fixed: -0.00  0.00 -0.20  RPY:  0.00  0.00  0.00
WARNING, apriltag roll/pitch out of bounds

Sometimes, the tag is detected and the position is set to the fixed frame which is way off. This causes voxl-vision-px4 to diverge within a few seconds, even if the drone does not move at all. It crashes in the end of that process and resets the frame.

When I run voxl-test-vision-lib -a it prints

ID:  1 hamming: 0 margin:89.98 XYZ:  0.67  0.06  1.85 RPY: -0.51  0.25  0.25 hz: 13.6

which seams reasonable, even if there is a rotation offset (btw. This is in the frame of the camera right? The z-axis point 45 deg downwards away from the drone)

I recalibrated the tracking camera and reset the config multiple times. Any idea what else could be wrong here? Here is a picture of the setup, it is very similar to your walkthrough video, the voxl is about 1 meter away in y and z directions.

Any suggestions would be greatly appreciated!