Do your tests notice small changes in MAVLink handling on VOXL?

[Yannik Wimhöfer]

Hi all, I am an independent security engineer working on a small tool that checks whether a test suite notices small changes in MAVLink parsing code. I plant a small change in the parser (for example a flipped condition) and see if the existing tests fail. If they do not, the change goes unnoticed.

On the MAVLink C library (c_library_v2) I planted 40 such changes. With the corpus I used, 22 were noticed and 18 were not. Two of the missed ones I could turn into small tests that pass on the original code and fail on the changed code, 3 of 3 runs each, in the PX4 SITL coverage build: a truncated payload where the length is visible mid frame, and a frame where the SIGNED flag is handled wrongly. Limit: this tests the library header, not PX4's own MAVLink handling.

My question for people who ship PX4 or similar firmware on VOXL: how do you check that your tests would catch a small change in MAVLink handling? Is this something you do today, and if so how?

If it helps, I am happy to run this on one component of your code for free and share the findings with you. No certification claims, just a list of changes your tests did not notice.

Thanks, Yannik