Sentinel segmentation fault after running voxl-uvc-server -v [camera id] -d
-
@Róbert-Dévényi The service just runs the application. If the application stops then the service stops. Are you familiar with Linux and systemd? It's just a standard Linux startup mechanism. But you want to stop all services, even for voxl-uvc-server, then try to start voxl-uvc-server on the command line and see if it still segfaults. There are ways to debug segfaults. But you would need to understand how to get Linux to generate a core file after the segfault and how to load the core into gdb.
@Eric-Katzfey I would appreciate it if you could tell me what commands to use to stop all the services and then restart voxl-uvc-server or any resource that helps with debugging segfaults in general.
-
@Róbert-Dévényi The service just runs the application. If the application stops then the service stops. Are you familiar with Linux and systemd? It's just a standard Linux startup mechanism. But you want to stop all services, even for voxl-uvc-server, then try to start voxl-uvc-server on the command line and see if it still segfaults. There are ways to debug segfaults. But you would need to understand how to get Linux to generate a core file after the segfault and how to load the core into gdb.
@Eric-Katzfey I reset all mpa services to factory settings and I keep getting the same segfault.
-
@Eric-Katzfey I reset all mpa services to factory settings and I keep getting the same segfault.
You can stop the systemd service for
voxl-uvc-serverusing the following command:systemctl stop voxl-uvc-server
this will stop the service / server but will not prevent it from auto-starting during next rebootThen you can run the
voxl-uvc-serverin foreground. Just run it with the normal arguments:voxl-uvc-server <your additional arguments here>Confirm that this still crashes.. then run the same executable in gdb:
> gdb /usr/bin/voxl-uvc-server (gdb) r <your additional arguments here> ... wait for crash... ... btbtcommand in gdb will print the backtrace of the call stack and will tell you which function the seg fault happened in (and you will see the call stack). Maybe this will help with debugging.Alex
-
You can stop the systemd service for
voxl-uvc-serverusing the following command:systemctl stop voxl-uvc-server
this will stop the service / server but will not prevent it from auto-starting during next rebootThen you can run the
voxl-uvc-serverin foreground. Just run it with the normal arguments:voxl-uvc-server <your additional arguments here>Confirm that this still crashes.. then run the same executable in gdb:
> gdb /usr/bin/voxl-uvc-server (gdb) r <your additional arguments here> ... wait for crash... ... btbtcommand in gdb will print the backtrace of the call stack and will tell you which function the seg fault happened in (and you will see the call stack). Maybe this will help with debugging.Alex
@Alex-Kushleyev Thank you very much hopefully I can figure it out.
-
You can stop the systemd service for
voxl-uvc-serverusing the following command:systemctl stop voxl-uvc-server
this will stop the service / server but will not prevent it from auto-starting during next rebootThen you can run the
voxl-uvc-serverin foreground. Just run it with the normal arguments:voxl-uvc-server <your additional arguments here>Confirm that this still crashes.. then run the same executable in gdb:
> gdb /usr/bin/voxl-uvc-server (gdb) r <your additional arguments here> ... wait for crash... ... btbtcommand in gdb will print the backtrace of the call stack and will tell you which function the seg fault happened in (and you will see the call stack). Maybe this will help with debugging.Alex
@Alex-Kushleyev I can probably show you the backtrace log so that you can take a look at what causes the segfault.
-
@Alex-Kushleyev I can probably show you the backtrace log so that you can take a look at what causes the segfault.
@Róbert-Dévényi @Eric-Katzfey @Alex-Kushleyev I am getting the same seg fault with a boson+ camera. I determined that the seg fault occurs when the uvc_start_streaming function is called (I am running a modified version of voxl-uvc-server). I have the stack trace from gdb. If I figure out the cause of the problem I will post it here.
(gdb) run Starting program: /usr/bin/voxl-uvc-server [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1". VOXL UVC Server loading config file VOXL UVC Server parsed argument voxl-uvc-server starting Image resolution 640x512, 30 fps chosen [New Thread 0x7ff7a7e1c0 (LWP 2189)] UVC initialized Device found [New Thread 0x7ff727d1c0 (LWP 2190)] Device opened Found desired frame format: 1 uvc_get_stream_ctrl_format_size succeeded for format NV12 bmHint: 0001 bFormatIndex: 2 bFrameIndex: 1 dwFrameInterval: 333333 wKeyFrameRate: 1 wPFrameRate: 0 wCompQuality: 0 wCompWindowSize: 0 wDelay: 0 dwMaxVideoFrameSize: 491520 dwMaxPayloadTransferSize: 983044 bInterfaceNumber: 1 [New Thread 0x7ff6a7c1c0 (LWP 2191)] [New Thread 0x7ff627b1c0 (LWP 2192)] [New Thread 0x7ff5a7a1c0 (LWP 2193)] Mavlink Onboard server Connected [New Thread 0x7fe70ef1c0 (LWP 2194)] Streaming starting [New Thread 0x7feffff1c0 (LWP 2195)] Thread 3 "voxl-uvc-server" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ff727d1c0 (LWP 2190)] __memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:185 185 ../sysdeps/aarch64/multiarch/../memcpy.S: No such file or directory. (gdb) bt #0 __memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:185 #1 0x0000007ff7ed32a0 in _uvc_process_payload () from /usr/lib/libuvc.so.0 Backtrace stopped: previous frame identical to this frame (corrupt stack?) -
@Róbert-Dévényi @Eric-Katzfey @Alex-Kushleyev I am getting the same seg fault with a boson+ camera. I determined that the seg fault occurs when the uvc_start_streaming function is called (I am running a modified version of voxl-uvc-server). I have the stack trace from gdb. If I figure out the cause of the problem I will post it here.
(gdb) run Starting program: /usr/bin/voxl-uvc-server [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1". VOXL UVC Server loading config file VOXL UVC Server parsed argument voxl-uvc-server starting Image resolution 640x512, 30 fps chosen [New Thread 0x7ff7a7e1c0 (LWP 2189)] UVC initialized Device found [New Thread 0x7ff727d1c0 (LWP 2190)] Device opened Found desired frame format: 1 uvc_get_stream_ctrl_format_size succeeded for format NV12 bmHint: 0001 bFormatIndex: 2 bFrameIndex: 1 dwFrameInterval: 333333 wKeyFrameRate: 1 wPFrameRate: 0 wCompQuality: 0 wCompWindowSize: 0 wDelay: 0 dwMaxVideoFrameSize: 491520 dwMaxPayloadTransferSize: 983044 bInterfaceNumber: 1 [New Thread 0x7ff6a7c1c0 (LWP 2191)] [New Thread 0x7ff627b1c0 (LWP 2192)] [New Thread 0x7ff5a7a1c0 (LWP 2193)] Mavlink Onboard server Connected [New Thread 0x7fe70ef1c0 (LWP 2194)] Streaming starting [New Thread 0x7feffff1c0 (LWP 2195)] Thread 3 "voxl-uvc-server" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ff727d1c0 (LWP 2190)] __memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:185 185 ../sysdeps/aarch64/multiarch/../memcpy.S: No such file or directory. (gdb) bt #0 __memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:185 #1 0x0000007ff7ed32a0 in _uvc_process_payload () from /usr/lib/libuvc.so.0 Backtrace stopped: previous frame identical to this frame (corrupt stack?)@AndriiHlyvko , we have gotten another report about this recently. It seem there is a bug in libuvc which has been fixed, but we are using older version of the library.
Inserting the following check at the line below has been reported to fix the issue:
if (strmh->got_bytes + data_len > strmh->cur_ctrl.dwMaxVideoFrameSize) { UVC_DEBUG("Transfer too large and will overflow the buffer. Dropping data!"); _uvc_swap_buffers(strmh); return; }here: https://github.com/modalai/libuvc/blob/3fb2d2fd31f215f76bd5d6782785385fd0927ce5/src/stream.c#L757
It seems the fix in the official libuvc release is here:
https://github.com/libuvc/libuvc/blob/master/src/stream.c#L779
https://github.com/libuvc/libuvc/blob/master/src/stream.c#L787We have not fixed this officially, please look at the code snippet i provided official fix and implement something similar until we fix it.
Alex
-
@AndriiHlyvko , we have gotten another report about this recently. It seem there is a bug in libuvc which has been fixed, but we are using older version of the library.
Inserting the following check at the line below has been reported to fix the issue:
if (strmh->got_bytes + data_len > strmh->cur_ctrl.dwMaxVideoFrameSize) { UVC_DEBUG("Transfer too large and will overflow the buffer. Dropping data!"); _uvc_swap_buffers(strmh); return; }here: https://github.com/modalai/libuvc/blob/3fb2d2fd31f215f76bd5d6782785385fd0927ce5/src/stream.c#L757
It seems the fix in the official libuvc release is here:
https://github.com/libuvc/libuvc/blob/master/src/stream.c#L779
https://github.com/libuvc/libuvc/blob/master/src/stream.c#L787We have not fixed this officially, please look at the code snippet i provided official fix and implement something similar until we fix it.
Alex
Thanks @Alex-Kushleyev . Do you know if there are any differences between using the voxl fork of libuvc vs compiling the master libuvc? I was thinking of just building the master libuvc but I am not sure if modalai did any changes to it.
-
Thanks @Alex-Kushleyev . Do you know if there are any differences between using the voxl fork of libuvc vs compiling the master libuvc? I was thinking of just building the master libuvc but I am not sure if modalai did any changes to it.
@AndriiHlyvko , you can see the changes that we made to the original fork : https://github.com/modalai/libuvc/commits/modalai-master (just the three top commits).
You may definitely try to use the latest master from the official repo, but we have not tested it. I believe there was some discussion recently about moving our code to latest, but we have not done it yet.
Alex
-
@AndriiHlyvko , you can see the changes that we made to the original fork : https://github.com/modalai/libuvc/commits/modalai-master (just the three top commits).
You may definitely try to use the latest master from the official repo, but we have not tested it. I believe there was some discussion recently about moving our code to latest, but we have not done it yet.
Alex
@Alex-Kushleyev just tested with latest master. The seg fault is gone and it seems to be working fine. One new problem that I have after moving to libuvc master is that the video stream from the boson+ looks wrong (the video is green
). Its only wrong for boson+ and not the boson. I'll keep debugging the issue. I was thinking maybe the boson+ sends incomplete frames in the uvc callback but I'm not sure. -
@Alex-Kushleyev just tested with latest master. The seg fault is gone and it seems to be working fine. One new problem that I have after moving to libuvc master is that the video stream from the boson+ looks wrong (the video is green
). Its only wrong for boson+ and not the boson. I'll keep debugging the issue. I was thinking maybe the boson+ sends incomplete frames in the uvc callback but I'm not sure.@Alex-Kushleyev figured it out. I had a bug that I introduced. I broke the mapping between UVC_FRAME_FORMAT enum and the voxl IMAGE_FORMAT enum in my fork of the uvc-server. Right now uvc streaming is working well with the boson+
-
@Alex-Kushleyev figured it out. I had a bug that I introduced. I broke the mapping between UVC_FRAME_FORMAT enum and the voxl IMAGE_FORMAT enum in my fork of the uvc-server. Right now uvc streaming is working well with the boson+
@AndriiHlyvko , that's great!
If you are willing to help out and share your changes on top of libuvc master, it should help us get this updated more quickly
Thank you
Alex
-
@AndriiHlyvko , that's great!
If you are willing to help out and share your changes on top of libuvc master, it should help us get this updated more quickly
Thank you
Alex
@Alex-Kushleyev sure. Should I make a merge request for it and tag you in it?
-
@Alex-Kushleyev sure. Should I make a merge request for it and tag you in it?
@AndriiHlyvko , yes and please tag @Eric-Katzfey who will probably review and merge.
Thank you!
Alex
-
@AndriiHlyvko , yes and please tag @Eric-Katzfey who will probably review and merge.
Thank you!
Alex
-
@AndriiHlyvko , Thank you! I will ask @Eric-Katzfey to review at his convenience.
-
@AndriiHlyvko , Thank you! I will ask @Eric-Katzfey to review at his convenience.
I just ran into the same issue while testing Boson (in my case I was testing the 16-bit Boson streaming mode). The libuvc issue caused a crash, since i did not use a fixed version of libuvc. However, what actually caused the buffer overflow was the fact that Boson was reporting smaller buffer requirements for RAW16 during query (it was reporting
width * height * 1.5, just like for YUV, but actual frame size waswidth * height * 2, so uvc pipeline allocated smaller buffer and overflow happened with the full frame came in.Was this the same issue with your Boson tests or did you experience something else?
Thanks!
Alex
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login