voxl-logger seg fault on shutdown

[jon]

Hi, in order to log a lot of different configurable data & cleanly parse the output, I wrote a python wrapper that calls voxl-logger as a subprocess and terminates it by sending a SIGINT (emulating user ctrl+c).

This issue only happens sometimes, maybe 30% of the time or something. I don't recall ever seeing it when starting voxl-logger natively in the terminal.

Running: stdbuf -oL -eL voxl-logger -c hires_small_color -i imu_apps -i imu_vectornav -B baro_vectornav -g gnss1_vectornav -I ins_vectornav
Welcome to voxl-logger
58127 MB left in /data/voxl-logger/
starting new log: /data/voxl-logger/log0044/

connected to ch 0 /run/mpa/hires_small_color/
connected to ch 1 /run/mpa/imu_apps/
^C
Stopping...

received SIGINT Ctrl-C
WARNING, _stop_helper_and_remove_pipe timed out joining read thread

Segmentation fault:
Fault thread: voxl-logger(tid: 26766)
Fault address: 0x7f9d719010
Address not mapped.

The main issue is that when this happens it fails to print the logger summary (channels & messages logged) which is important for me to quickly analyze the validity of the logged data. Also we've had issues with the logger not shutting down gracefully and only partially populating the final row of the csv it was writing to.

Do you guys have any idea why this might be happening? I would much prefer the logger to exit consistently and gracefully.

Thanks!

[Alex Kushleyev]

Hi @jon , do you always see the following warning right before the seg fault? (and never see it when there is no seg fault?)

@jon said in voxl-logger seg fault on shutdown:

WARNING, _stop_helper_and_remove_pipe timed out joining read thread

The way one would normally debug this is using gdb, but that would be tricky if you are spawning the logger from another (python) process and don't have a terminal to type in gdb (after crash) to investigate.. You could potentially attach gdb to an already running (spawned) logger.

Alex