VIO errors with new AR0144 tracking camera unstable flight/crashing
-
This post is deleted!
Here are a few suggestions to help debug this:
- I noticed that you have a 180 degree rotation specified for the camera config. This rotation is achieved by reversing the readout order from the camera, which affects the intrinsic calibration. Make sure you calibrate your camera's intrinsics with the same rotation (enabled or disabled) as you are testing, otherwise the intrinsics will not be valid.
- whenever working on a new setup, always test VIO in hand-held tests in order to eliminate other variables. If VIO does not work in hand-held tests, there is no hope that it will work in flight
- do you ever have cases where VIO works for some time, maybe 5-10 seconds while you are moving (and then blows up) or is it always blowing up as soon as you start moving? If the latter is the case, majority of the cases is incorrect extrinsic parameters (specifically, camera to IMU rotation). did you calculate your extrinsics based on rotated or not rotated camera configuration?
- please share the results of your intrinsic calibration, just to double check it
- double check camera focus, anything farther than 6 inches or so should be in clear focus (fisheye lenses have very short focal distance and large depth of field)
Alex
-
A Alex Kushleyev referenced this topic on
-
Here are a few suggestions to help debug this:
- I noticed that you have a 180 degree rotation specified for the camera config. This rotation is achieved by reversing the readout order from the camera, which affects the intrinsic calibration. Make sure you calibrate your camera's intrinsics with the same rotation (enabled or disabled) as you are testing, otherwise the intrinsics will not be valid.
- whenever working on a new setup, always test VIO in hand-held tests in order to eliminate other variables. If VIO does not work in hand-held tests, there is no hope that it will work in flight
- do you ever have cases where VIO works for some time, maybe 5-10 seconds while you are moving (and then blows up) or is it always blowing up as soon as you start moving? If the latter is the case, majority of the cases is incorrect extrinsic parameters (specifically, camera to IMU rotation). did you calculate your extrinsics based on rotated or not rotated camera configuration?
- please share the results of your intrinsic calibration, just to double check it
- double check camera focus, anything farther than 6 inches or so should be in clear focus (fisheye lenses have very short focal distance and large depth of field)
Alex
This post is deleted! -
Here are a few suggestions to help debug this:
- I noticed that you have a 180 degree rotation specified for the camera config. This rotation is achieved by reversing the readout order from the camera, which affects the intrinsic calibration. Make sure you calibrate your camera's intrinsics with the same rotation (enabled or disabled) as you are testing, otherwise the intrinsics will not be valid.
- whenever working on a new setup, always test VIO in hand-held tests in order to eliminate other variables. If VIO does not work in hand-held tests, there is no hope that it will work in flight
- do you ever have cases where VIO works for some time, maybe 5-10 seconds while you are moving (and then blows up) or is it always blowing up as soon as you start moving? If the latter is the case, majority of the cases is incorrect extrinsic parameters (specifically, camera to IMU rotation). did you calculate your extrinsics based on rotated or not rotated camera configuration?
- please share the results of your intrinsic calibration, just to double check it
- double check camera focus, anything farther than 6 inches or so should be in clear focus (fisheye lenses have very short focal distance and large depth of field)
Alex
-
This post is deleted!
@Gary-Holmgren , I think your imu -> tracking camera rotation should be:
"RPY_parent_to_child": [0, 45, 90] (+90, not -90). Even though your flex is pointing up, you have specified in camera server config to rotate the image (in the camera ISP itself).
Try that..
What i meant about the calibration / rotation.. make sure that you set the
"en_rotate": trueinvoxl-camera-server.confand then perform the camera calibration, so that you calibrate the camera with the correct internal (to camera) image rotation. If you perform intrinsics calibration and then change the internal camera rotation, the calibration will not be correct (principal points will be flipped, at least).I checked, the video, it basically shows ZERO feature tracking, so the transform issue is probably causing this.
Alex
-
@Gary-Holmgren , I think your imu -> tracking camera rotation should be:
"RPY_parent_to_child": [0, 45, 90] (+90, not -90). Even though your flex is pointing up, you have specified in camera server config to rotate the image (in the camera ISP itself).
Try that..
What i meant about the calibration / rotation.. make sure that you set the
"en_rotate": trueinvoxl-camera-server.confand then perform the camera calibration, so that you calibrate the camera with the correct internal (to camera) image rotation. If you perform intrinsics calibration and then change the internal camera rotation, the calibration will not be correct (principal points will be flipped, at least).I checked, the video, it basically shows ZERO feature tracking, so the transform issue is probably causing this.
Alex
This post is deleted! -
@Gary-Holmgren , I think your imu -> tracking camera rotation should be:
"RPY_parent_to_child": [0, 45, 90] (+90, not -90). Even though your flex is pointing up, you have specified in camera server config to rotate the image (in the camera ISP itself).
Try that..
What i meant about the calibration / rotation.. make sure that you set the
"en_rotate": trueinvoxl-camera-server.confand then perform the camera calibration, so that you calibrate the camera with the correct internal (to camera) image rotation. If you perform intrinsics calibration and then change the internal camera rotation, the calibration will not be correct (principal points will be flipped, at least).I checked, the video, it basically shows ZERO feature tracking, so the transform issue is probably causing this.
Alex
-
This post is deleted!
@Gary-Holmgren , great! The transforms can be tricky sometimes especially with the added complexity of the image rotation in the camera…
Alex
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login