VIO errors with new AR0144 tracking camera unstable flight/crashing
-
[[topic:post-is-deleted]]
Here are a few suggestions to help debug this:
- I noticed that you have a 180 degree rotation specified for the camera config. This rotation is achieved by reversing the readout order from the camera, which affects the intrinsic calibration. Make sure you calibrate your camera's intrinsics with the same rotation (enabled or disabled) as you are testing, otherwise the intrinsics will not be valid.
- whenever working on a new setup, always test VIO in hand-held tests in order to eliminate other variables. If VIO does not work in hand-held tests, there is no hope that it will work in flight
- do you ever have cases where VIO works for some time, maybe 5-10 seconds while you are moving (and then blows up) or is it always blowing up as soon as you start moving? If the latter is the case, majority of the cases is incorrect extrinsic parameters (specifically, camera to IMU rotation). did you calculate your extrinsics based on rotated or not rotated camera configuration?
- please share the results of your intrinsic calibration, just to double check it
- double check camera focus, anything farther than 6 inches or so should be in clear focus (fisheye lenses have very short focal distance and large depth of field)
Alex
-
A Alex Kushleyev referenced this topic on
-
Here are a few suggestions to help debug this:
- I noticed that you have a 180 degree rotation specified for the camera config. This rotation is achieved by reversing the readout order from the camera, which affects the intrinsic calibration. Make sure you calibrate your camera's intrinsics with the same rotation (enabled or disabled) as you are testing, otherwise the intrinsics will not be valid.
- whenever working on a new setup, always test VIO in hand-held tests in order to eliminate other variables. If VIO does not work in hand-held tests, there is no hope that it will work in flight
- do you ever have cases where VIO works for some time, maybe 5-10 seconds while you are moving (and then blows up) or is it always blowing up as soon as you start moving? If the latter is the case, majority of the cases is incorrect extrinsic parameters (specifically, camera to IMU rotation). did you calculate your extrinsics based on rotated or not rotated camera configuration?
- please share the results of your intrinsic calibration, just to double check it
- double check camera focus, anything farther than 6 inches or so should be in clear focus (fisheye lenses have very short focal distance and large depth of field)
Alex
This post is deleted! -
Here are a few suggestions to help debug this:
- I noticed that you have a 180 degree rotation specified for the camera config. This rotation is achieved by reversing the readout order from the camera, which affects the intrinsic calibration. Make sure you calibrate your camera's intrinsics with the same rotation (enabled or disabled) as you are testing, otherwise the intrinsics will not be valid.
- whenever working on a new setup, always test VIO in hand-held tests in order to eliminate other variables. If VIO does not work in hand-held tests, there is no hope that it will work in flight
- do you ever have cases where VIO works for some time, maybe 5-10 seconds while you are moving (and then blows up) or is it always blowing up as soon as you start moving? If the latter is the case, majority of the cases is incorrect extrinsic parameters (specifically, camera to IMU rotation). did you calculate your extrinsics based on rotated or not rotated camera configuration?
- please share the results of your intrinsic calibration, just to double check it
- double check camera focus, anything farther than 6 inches or so should be in clear focus (fisheye lenses have very short focal distance and large depth of field)
Alex
-
[[topic:post-is-deleted]]
@Gary-Holmgren , I think your imu -> tracking camera rotation should be:
"RPY_parent_to_child": [0, 45, 90] (+90, not -90). Even though your flex is pointing up, you have specified in camera server config to rotate the image (in the camera ISP itself).
Try that..
What i meant about the calibration / rotation.. make sure that you set the
"en_rotate": trueinvoxl-camera-server.confand then perform the camera calibration, so that you calibrate the camera with the correct internal (to camera) image rotation. If you perform intrinsics calibration and then change the internal camera rotation, the calibration will not be correct (principal points will be flipped, at least).I checked, the video, it basically shows ZERO feature tracking, so the transform issue is probably causing this.
Alex
-
@Gary-Holmgren , I think your imu -> tracking camera rotation should be:
"RPY_parent_to_child": [0, 45, 90] (+90, not -90). Even though your flex is pointing up, you have specified in camera server config to rotate the image (in the camera ISP itself).
Try that..
What i meant about the calibration / rotation.. make sure that you set the
"en_rotate": trueinvoxl-camera-server.confand then perform the camera calibration, so that you calibrate the camera with the correct internal (to camera) image rotation. If you perform intrinsics calibration and then change the internal camera rotation, the calibration will not be correct (principal points will be flipped, at least).I checked, the video, it basically shows ZERO feature tracking, so the transform issue is probably causing this.
Alex
This post is deleted! -
@Gary-Holmgren , I think your imu -> tracking camera rotation should be:
"RPY_parent_to_child": [0, 45, 90] (+90, not -90). Even though your flex is pointing up, you have specified in camera server config to rotate the image (in the camera ISP itself).
Try that..
What i meant about the calibration / rotation.. make sure that you set the
"en_rotate": trueinvoxl-camera-server.confand then perform the camera calibration, so that you calibrate the camera with the correct internal (to camera) image rotation. If you perform intrinsics calibration and then change the internal camera rotation, the calibration will not be correct (principal points will be flipped, at least).I checked, the video, it basically shows ZERO feature tracking, so the transform issue is probably causing this.
Alex
-
[[topic:post-is-deleted]]
@Gary-Holmgren , great! The transforms can be tricky sometimes especially with the added complexity of the image rotation in the camera…
Alex
Reading along? Create a free ModalAI Forum account to join in.
With an account you can reply, ask your own question, get an email when a ModalAI engineer answers, and mark the reply that solved it. Your place in each thread is saved between visits.
Questions about VOXL, Flight Core, ESCs and ModalAI drones are answered here by the engineers who build them.
Register Login