EV data causing the drone to crash when in stabilize mode

[Sem Andeweg]

I am flying PX4 v1.11 and v1.13 on the ModalAI Flight core with their VOXL companion computer that gives VIO data - and we experience a bug in PX4 that results in crashes. It's something that should not happen, and normally does not happen but when it does we need to emergency land or if possible switch to acro mode and then safely make an emergency landing.

So what happens: In stabilize mode when flying the drone, after flying for some time without any troubles, the drone suddenly pitches or rolls in one direction to its maximum angle. Even counter-steering the drone in the other direction only works to some degree, it will still go in that direction. When changing to acro, we can still safely control and land the drone.

Here we have a Log of one instance. You can see the drone is only in stabilize mode, and at the end of the flight, the pilot is at full pitch and rolls to make a safe emergency landing. I have a video of this, and this just happens when hovering, the drone is not at any cruising speed or something. The drone is just scanning something an object from top to bottom.
https://logs.px4.io/plot_app?log=cde7b479-812b-4d3f-8035-c8c79fc4ca3e

The way we solve this problem is when we fly in stabilize mode we change the EKF2_AID_MASK from 280 to 0.

So what we think happens is that the VIO data becomes bad (due to low features or whatever), and think the drone is going in the wrong direction. So the drone wants to counteract this and rolls and pitches in the opposite direction the VIO thinks it's going to. This is something that would happen in position-hold mode. But here comes the bug, this also happens in modes that “don't” use the VIO data. We see this happening in modes like stabilize, altitude hold mode, and position failsafe.

How is this possible? And can we switch this off? Or is this a bug that needs to be fixed?

[Steve Turner]

Sem, (I don't work for modal!)

A few ideas..

Have you tried disabling the QVIO service and adjusting the EKF2_AID_MASK values to ignore the vision inputs across the modes you've tested with PX4?

See how the platform behaves and that should either point to VIO or rule it out. In stabilized mode and without GPS and VIO inputs the platform should be relatively stable.

[Cliff Wong]

Yes, please disable the qvio-server when running non-evdata modes. Note, you will have an issue when AID_MASK = 0 and you try to go into position/offboard/auto modes due to you will be in dead reckon mode and may not have an established (or correct to ground truth) NED origin.

[Steve Turner]

VIO/ EV data causing the drone to crash when in stabilize mode · Issue #20950 · PX4/PX4-Autopilot

I am flying PX4 v1.11 and v1.13 on the ModalAI Flight core with their VOXL companion computer that gives VIO data - and we experience a bug in PX4 that results in crashes. It's something that should not happen, and normally does not happ...

GitHub (github.com)

@Sem-Andeweg Just cross-posting here too for anyone that comes back to it. I'm interested in these issues as well.

[Sem Andeweg]

@Steve-Turner - Yeah I have put EKF2_AID_MASK to 0. Then the drone would just be behaving normally.

@Cliff-Wong - I understand that I would have to disable qvio-server, if I would only fly in non-evdata modus. But when flying with evdata, if then the evdata goes bad, I need to switch to non-evdata to be able to save the drone from crashing. But now the drone will still crash!