@Alex-Kushleyev
Thankyou for the advice!
There are few seconds of flight where the drone does want to stay in place and VIO seems to be working but the drift then starts to go haywire and I have to fight the controls to keep it in one spot, I am testing this in a very well lit feature-rich space as well.
When you said "Make sure you calibrate your camera's intrinsics with the same rotation (enabled or disabled)" did you mean to calibrate after all these parameters are set correctly? I calibrated the camera after the hardware was all installed in the drone with the rotate flag set in the custom can config. Or are you implying there is a rotate flat you have to specify when doing the actual calibration procedure ex: voxl-calibrate-camera tracking -f -s -l... or in the intrinsic YAML file there needs to be a rotate parameter?
Maybe you can see what might have gone wrong, I got a fairly good calibration recently so here are my results.
Also, I am running px4 off the flight core V2 connected to voxl2 over uart. the parameters and everything else stayed the same when switching to the AR0144 tracking sensor so I don't think anything is wrong there. Just to reiterate I had the older tracking camera working with follow tag as well and it was very stable so the only issues I believe could be causing this are related to the new AR0144 setup I have. I am also on the newest SDK 1.3.0 as it just came out a couple of days ago.
For reference here is a pic of the cameras mounting configuration to make sense of the extrinsics config
I am under the assumption the bottom "positive Y" axis of the camera is where the MIPI cables come out from which seems to be the convention with all of the image sensors I've used. thus I have rotated the Z on the tracking cam to -90 so the MIPI points upwards and the X axis is pointing out the left side of the drone.
In order to install the tracking cam the lens does have to be taken off but it's refocused during installation I ensured there's no contamination on the sensor or lens during install as well.
Below are my current extrinsics and intrinsics.
"extrinsics": [{
"parent": "imu_apps",
"child": "tracking",
"T_child_wrt_parent": [0.0501, 0.0060, 0.0093],
"RPY_parent_to_child": [0, 45, -90]
}, {
"parent": "body",
"child": "imu_apps",
"T_child_wrt_parent": [0.1167, 0.0060, 0.0053],
"RPY_parent_to_child": [0, 0, 0]
}, {
"parent": "body",
"child": "tof",
"T_child_wrt_parent": [0.1752, 0, 0.0034],
"RPY_parent_to_child": [0, 90, -90]
}, {
"parent": "body",
"child": "ground",
"T_child_wrt_parent": [0, 0, 0.0328],
"RPY_parent_to_child": [0, 0, 0]
}]
}
%YAML:1.0
---
M: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [ 5.6009618399934220e+02, 0., 6.4970452549577044e+02, 0.,
5.6164176916165263e+02, 4.0012359829168020e+02, 0., 0., 1. ]
D: !!opencv-matrix
rows: 4
cols: 1
dt: d
data: [ -4.7175359821771193e-02, 3.8442932109403055e-02,
-1.7319222703866245e-02, -3.5236303868186605e-04 ]
reprojection_error: 3.0046768943236762e-01
width: 1280
height: 800
distortion_model: fisheye
calibration_time: "2024-06-04 22:33:29"
