Several modules failing during startup making controller unusable as-shipped

[jcroughan]

I recently received a new Flight Core V2, and there appear to be several issues with the firmware as shipped.

1. It won't connect with telemetry radios after an airframe is set. I was able to connect with a standard holybro telemetry radio at first, but then after I picked a generic quad x (ID 4001) airframe and rebooted, it locked me out. When I try to connect QGC says 'Cannot connect, access is denied.' I can still connect via usb when this is happening, and the telemetry radios are showing the lights indicating that they are communicating with each other, but the flight controller itself appears to be denying access.
2. The position/orientation estimator appears to not be starting successfully. This is a much bigger issue, since it renders the flight controller a brick. I can still spin up the motors just fine using the motor tests, but it is failing the prearm checks for all flight modes due to lack of a position and orientation estimate. If I run the 'top' command in the mavlink console, I can see that the estimator isn't running. When I run dmesg, this is the entire output I get:

NuttShell (NSH) NuttX-11.0.0
nsh> dmesg -f &
dmesg [909:100]
nsh> HW arch: MODALAI_FC_V2
HW type: V230
HW version: 0x00000003
HW revision: 0x00000000
FW git-hash: 259b3c097773507e581e2930093e4958f7726381
FW version: Release 1.13.2 (17629951)
OS: NuttX
OS version: Release 11.0.0 (184549631)
OS git-hash: 91bece51afbe7da9db12e3695cdbb4f4bba4bc83
Build datetime: Mar 10 2023 16:10:22
Build uri: localhost
Build variant: default
Toolchain: GNU GCC, 9.3.1 20200408 (release)
PX4GUID: 0006000000003333363031315103002c0046
MCU: STM32H7[4|5]xxx, rev. V
INFO [param] selected parameter default file /fs/mtd_params
INFO [param] importing from '/fs/mtd_params'
INFO [parameters] BSON document size 1317 bytes, decoded 1317 bytes (INT32:28, FLOAT:41)
INFO [param] selected parameter backup file /fs/microsd/parameters_backup.bson
Board architecture defaults: /etc/init.d/rc.board_arch_defaults
Board defaults: /etc/init.d/rc.board_defaults
INFO [dataman] data manager file '/fs/microsd/dataman' size is 62560 bytes
Loading airframe: /etc/init.d/airframes/4001_quad_x
Board sensors: /etc/init.d/rc.board_sensors
voxlpm #0 on I2C bus 3 (external) address 0x44
WARN [SPI_I2C] Already running on bus 3
WARN [SPI_I2C] voxlpm: no instance started (no device on bus?)
icm42688p #0 on SPI bus 1 rotation 12
icm42688p #1 on SPI bus 2 rotation 12
icp20100 #0 on I2C bus 4 (external) address 0x63
INFO [pwm_out] instance: 0, max rate: 100, default: 50, alt: 50
INFO [init] Mixer: /etc/mixers/quad_x.main.mix on /dev/pwm_output0
INFO [init] setting PWM_AUX_OUT none
ekf2 [617:237]
INFO [pwm_out] instance: 0, max rate: 800, default: 50, alt: 400
Starting Main GPS on /dev/ttyS0
Starting MAVLink on /dev/ttyS6
INFO [icp20100] Already configured.
INFO [mavlink] mode: Onboard, data rate: 46080 B/s on /dev/ttyS6 @ 921600B
WARN [mavlink] stream SCALED_PRESSURE not found
ERROR [mavlink] configure_streams_to_default() failed
INFO [logger] logger started (mode=all)

NuttShell (NSH) NuttX-11.0.0
nsh> INFO [rc_input] RC scan: SBUS RC input locked

[Alex Kushleyev]

@jcroughan , Are you able to see raw IMU data coming into GCS and have you performed the IMU calibration using GCS?

[jcroughan]

@Alex-Kushleyev I figured it why the orientation estimator wasn't running after reading through this post, where that user had the same (plus a few more) problems: https://forum.modalai.com/topic/1755/setup-problems-on-new-boards-sensors-and-qgc-connection-problems/18

Basically the problem is that the firmware as-shipped assumes you have at least one active mag, but that the built-in one isn't active. The documentation didn't say how to change the PX4 parameters to make it work without one. The default build does not start the built-in mag, so if you don't have an external plugged in (as in my use case), then EKF2 gets stuck while looking for the reading for one without any error printout, making it unclear what the problem is. I initially tried the version of activating it by adding the boot line to the etc folder of the sd card, and that worked once, but then it got deleted on the next reboot for unknown reasons. That is an easy enough fix with just a firmware rebuild where the magnetometer is started by default. That really is the version you should ship unless you are going to specify in your documentation that the as-shipped version will need an external mag to function. With the firmware as-is, you have to change two parameters on PX4 to make it work, specifically you have to turn off EKF2 magnetometer fusion and whether or not the system has a mag at all. Ie. EKF2_MAG_TYPE to NONE and SYS_HAS_MAG to Disabled. After doing that it will run.

Still haven't sorted out the issue with the telem lockout. I'll see if that got fixed with EKF2 now running normally.