TOF Sensor placement affects Tracking Camera/qvio

[jaredjohansen]

There are three parts to this question. Each one involves indoor environments.

PART I
We purchased a TOF sensor and connected it to the MIPI port that is typically designated for the stereo cameras. We placed the TOF sensor on the front sensor board (to the right of the hires camera and above the right stereo camera). We ran voxl-configure-cameras and used option 6 (hires + TOF + tracking). We can see all camera streams in voxl-portal.

In voxl-portal, when we view the Tracking camera, we observe intermittent flashes of light. These flashes of light come from the TOF sensor and affect the exposure of the tracking camera and influence the location and quantity of keypoints used in qvio.

Have you observed similar phenomenon and have you found this affects qvio? Where do you recommend placing the TOF sensor?

(We found this situation mainly occurs when the drone is on the ground.
When it is flying, the light from the TOF sensor has to travel further and doesn't affect the TOF sensor as described above. We also found this situation is ameliorated when there are a lot of bright lights around to drown out the light flashes coming from the TOF sensor.)

PART II

With the setup I described, we would place our drone on different surfaces. On certain surfaces (e.g., where the entire surface is the same color), qvio would struggle to find any keypoints and would often output a stream of zeroes. The stream of zeroes made it dangerous to fly in offboard mode since it relies on the odometry data provided by qvio.
On other surfaces (e.g., a wood grain table), qvio found plenty of keypoints and qvio would work every time.

We observed that qvio relies on keypoints created from high gradient regions in the image (e.g., corners and edges of light/dark). For certain situations, when taking off, we found it was necessary to put a high-contrasting pattern (e.g., QR code) on the floor in front of the tracking camera to allow qvio to work properly.

Are there any workarounds to this situation so that it doesn't require a specialized board below the tracking camera during takeoff?

PART III

qvio occassionally "goes off the rails" in two ways: (1) producing an odometry location that is reasonable one moment and unreasonable the next (e.g., 20 meters away) and (2) producing odometry locations that are moving at high rates (e.g., 1's or 10's meters per second) even when the drone is stationary (e.g., on the ground).

In offboard mode, the first problem causes our drone to think it is far away from the target location. When it attempts to fly to the target location (that is now 20 meters away), it can crash into walls. (Obviously we can put some checks into our code to avoid this situation...and have already done so.)

Have you observed similar phenomenon? Is there any guidance to (a) prevent this phenomenon or (b) guard against it?

[Moderator]

The flashing from the TOF sensor is normal and should not effect VIO.

See the documentation troubleshooting VIO, that should help with debugging https://docs.modalai.com/flying-with-vio/#troubleshooting-vio