APPS IMU stopped producing data mid flight

Rowan Dempster

During a recent flight the voxl-imu-server stopped producing data with this error:

May 09 19:49:48 m0054 voxl-imu-server[1750]: ERROR icm42688_fifo_read: buffer was empty 54514 times in a row
May 09 19:49:48 m0054 voxl-imu-server[1750]: ERROR icm42688_fifo_read: buffer was empty 54515 times in a row
May 09 19:49:48 m0054 voxl-imu-server[1750]: ERROR icm42688_fifo_read: buffer was empty 54516 times in a row
May 09 19:49:48 m0054 voxl-imu-server[1750]: ERROR icm42688_fifo_read: buffer was empty 54517 times in a row
May 09 19:49:48 m0054 voxl-imu-server[1750]: ERROR icm42688_fifo_read: buffer was empty 54518 times in a row
May 09 19:49:48 m0054 voxl-imu-server[1750]: ERROR icm42688_fifo_read: buffer was empty 54519 times in a row
May 09 19:49:48 m0054 voxl-imu-server[1750]: ERROR icm42688_fifo_read: buffer was empty 54520 times in a row
May 09 19:49:48 m0054 voxl-imu-server[1750]: ERROR icm42688_fifo_read: buffer was empty 54521 times in a row
May 09 19:49:48 m0054 voxl-imu-server[1750]:  imu 0
May 09 19:49:48 m0054 voxl-imu-server[1750]: WARNING bad reading on imu 0

after restarting the service it failed again

May 09 19:51:14 m0054 voxl-imu-server[7425]: aux_imu3_enable:                 0
May 09 19:51:14 m0054 voxl-imu-server[7425]: aux_imu3_spi_bus:                5
May 09 19:51:14 m0054 voxl-imu-server[7425]: aux_imu3_sample_rate_hz:         1000.0
May 09 19:51:14 m0054 voxl-imu-server[7425]: aux_imu3_lp_cutoff_freq_hz:      92.0
May 09 19:51:14 m0054 voxl-imu-server[7425]: aux_imu3_fifo_poll_rate_hz:      100.0
May 09 19:51:14 m0054 voxl-imu-server[7425]: =================================================================
May 09 19:51:14 m0054 voxl-imu-server[7425]: thread is locked to cores: 0 1 2 3
May 09 19:51:14 m0054 voxl-imu-server[7425]: Detected M0054 VOXL2
May 09 19:51:14 m0054 voxl-imu-server[7425]: Initializing IMU0
May 09 19:51:14 m0054 systemd[1]: voxl-imu-server.service: Failed with result 'exit-code'.

after rebooting the voxl2 it was fine again, but I'm sure it will happen again given enough time, already seen it twice.

Any ideas what is up here or just a faulty voxl2? We have dozens of other voxl2s that haven't seen this issue.