Immediate crashes in HITL

[shawn_ricardo]

The input was:

1. qshell commander arm
2. qshell commander takeoff

During the hold, the quad flipped.

Here is the PX4 flight log, which shows interesting actuator behavior:
https://review.px4.io/plot_app?log=518ccf49-36ed-4509-b928-da3dda44c7fd

The motor output diverges significantly around 2:12

At this time, the roll angle, pitch angle, and yaw angle executed deviated significantly from the commanded values:

Lastly, there were no failsafe flags indicating a loss of sensor input (e.g., EKF divergence, GPS loss, etc)....

[Only non-zero streams have been selected in the image below]

Could this be an issue with the actuation commands coming from the VOXL2?

[shawn_ricardo]

Here is a readout of the qshell dsp_hitl status command, just to validate the connection:

[Eric Katzfey]

@shawn_ricardo What command did you use to start HITL? Are you trying hold mode with GPS?

[shawn_ricardo]

Hi @Eric-Katzfey,

To start HITL, I used voxl-px4-hitl

I ran the commands:

1) qshell commander arm
2) qshell commander takeoff

which will eventually go into HOLD mode.

As far as the sensor positioning input, I'm assuming that I'm using GPS because I followed this instruction to turn off qvio:

NOTE: Ensure that voxl-qvio-server is disabled. If it is enabled, it will confuse gazebo due to gazebo sending fake sensor data to the qvio algorithm and causing poor flight dynamics in HITL. This can be done with the systemctl disable voxl-qvio-server command.

Here are the services running via voxl-inspect-services. Note that voxl-px4 is not running because the sim is not running:

[shawn_ricardo]

@Eric-Katzfey , would you happen to recommend next troubleshooting steps?

[Eric Katzfey]

@shawn_ricardo I'll try it on my setup and see what happens.

[Eric Katzfey]

@shawn_ricardo I just tried it and it worked fine. But I had all services disabled except for voxl-wait-for-fs, voxl-mavlink-server, and voxl-cpu-monitor. So maybe try that.

[Eric Katzfey]

@shawn_ricardo Can you capture the log of startup messages for voxl-px4-hitl so I can look for any error messages, or incorrect settings?

[shawn_ricardo]

Hi @Eric-Katzfey ,

I've tried those settings and the drone is able to stay hovering much long than expected. the positioning appears to be better, as well.

What service(s) might be getting in the way of HITL performance?

Also, I noticed during flight the drone vibrates significantly in HITL as soon as I publish offboard commands while in HOLD mode according to PX4's offboard guidelines. As soon as I stop publishing to /fmu/in/trajectory_setpoint, the vibrations stop. It is as if the drone is toggling between offboard and hold modes.

I have "offboard_mode" set to "off" in /etc/modalai/voxl-vision-hub.conf. I am running my own ros2 offboard node. I've enabled voxl-cpu-monitor, voxl-microdds-agent, voxl-modem, and voxl-wait-for-fs. I am running the deb package of voxl_mpa_to_ros2 . The command I am running is ros2 run voxl_mpa_to_ros2 voxl_mpa_to_ros2_node

I've tested the ability to publish offboard commands while in position hold mode in a completely separate px4 sim environment (just regular PX4 sim) and verified that drone position does not "vibrate" while offboard commands are being publishing and hold mode is the active mode.

What could be the cause of this behavior issue?

[Eric Katzfey]

@shawn_ricardo To figure out which service could be causing the issue just try starting each one and see which one causes the issue.

[shawn_ricardo]

@Eric-Katzfey

Okay, I'll keep this in mind as I develop toward offboard control. If I find any more information, I'll follow up here.

Just a side note: I don't see this behavior in real-world flight during manual or position, seems to be isolated to HITL at this point in time.

Thanks!