voxl communications completely locking up
-
voxl-cpu-monitor doesn't appear to output anything other than "Init complete, entering main loop"
-
Maybe you're on an older release. voxl-perfmon is the older version of cpu-monitor
-
I wasn't able to run voxl-cpu-monitor but I made a simple bash script that blinks the voxl led every second and it stops blinking once the voxl stops responding. Also I tried tweaking the settings mentioned here https://docs.ros.org/en/ros2_documentation/dashing/Guides/DDS-tuning.html#cross-vendor-tuning, but that didn't help
-
@Chad-Sweet how do you recommend debugging this? Is there a system log I can view or any way to identify what is causing the lockup? Also what "release" has voxl-cpu-monitor? I don't think it's included in the latest image.
-
voxl-perfmon is here https://gitlab.com/voxl-public/utilities/voxl-utils/-/blob/master/bin/voxl-perfmon
it will let you know if CPU is maxing out which is the most likely scenario. Do you have a fan on the Snapdragon? that will help maximize CPU performance
If you're launching too much stuff at the same time, can you manually launch incrementally until you get it figured out?
-
Hi @Chad-Sweet We have a fan and the cpu doesn't appear to be overloaded. Is there any system log capabilities to help identify why the system is locking up and requiring a hard reboot?
-
If you run
dmesg -wHin a terminal it will show you all kernel messages and will continually update so you can see any last messages. -
@Eric-Katzfey We're having trouble identifying the errors in dmesg. Is there a watchdog timer that could automatically restart the voxl? This will help debugging uas in a remote location without needing a hard reset.
-
@benjamin-linne I'm not sure what you mean by a watchdog timer. Usually a watchdog requires a check-in from the threads / processes it is monitoring and will automatically reset the board if the check-in does not happen. You could certainly implement something like that on your own.
-
@Eric-Katzfey Could you point me to how I can setup watchdog timers on the voxl?
-
@benjamin-linne I'm sorry, but I'm really not sure what you are looking for. There are no generic watchdog timers that I am aware of. You would need to make something custom.
-
@Eric-Katzfey I'm looking for documentation on the hardware watchdog timers onboard the voxl's snapdragon microcontroller
-
Unfortunately, I'm not aware of any such documentation.
-
@Eric-Katzfey I was referring to the documentation for the specific registers that would need to be modified to enable and configure it since I'm familiar with embedded systems that allow that functionality. Another possibility is to add the /dev/watchdog kernel module, however that would require an update to the operating system. If this is possible, it would greatly improve the robustness since we are encountering many cpu lockups that require human intervention to perform hard reboots. For remote applications this is potentially a critical feature
-
The kernel build is posted at https://gitlab.com/voxl-public/system-image-build/voxl-build
