ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Sentinel segmentation fault after running voxl-uvc-server -v [camera id] -d

    Sentinel
    4
    30
    1702
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Eric KatzfeyE
      Eric Katzfey ModalAI Team @Róbert Dévényi
      last edited by

      @Róbert-Dévényi Nothing should be causing a segfault. I'm just offering suggestions on how to debug what is going on.

      Róbert DévényiR 1 Reply Last reply Reply Quote 0
      • Eric KatzfeyE
        Eric Katzfey ModalAI Team @Róbert Dévényi
        last edited by

        @Róbert-Dévényi The service just runs the application.

        Róbert DévényiR 1 Reply Last reply Reply Quote 0
        • Róbert DévényiR
          Róbert Dévényi @Eric Katzfey
          last edited by

          @Eric-Katzfey so I stopped all services except for voxl-uvc-server:
          Scanning services...

          Service Name | Enabled | Running | CPU Usage

          docker-autorun | Disabled | Not Running |
          modallink-relink | Disabled | Not Running |
          voxl-camera-server | Enabled | Not Running |
          voxl-cpu-monitor | Enabled | Not Running |
          voxl-dfs-server | Disabled | Not Running |
          voxl-feature-tracker | Disabled | Not Running |
          voxl-flow-server | Disabled | Not Running |
          voxl-imu-server | Enabled | Not Running |
          voxl-lepton-server | Disabled | Not Running |
          voxl-mavcam-manager | Enabled | Not Running |
          voxl-mavlink-server | Enabled | Not Running |
          voxl-modem | Enabled | Not Running |
          voxl-neopixel-manager | Disabled | Not Running |
          voxl-open-vins-server | Disabled | Not Running |
          voxl-portal | Enabled | Not Running |
          voxl-px4-imu-server | Disabled | Not Running |
          voxl-px4 | Enabled | Not Running |
          voxl-qvio-server | Enabled | Not Running |
          voxl-rangefinder-server | Disabled | Not Running |
          voxl-remote-id | Disabled | Not Running |
          voxl-softap | Disabled | Not Running |
          voxl-static-ip | Disabled | Not Running |
          voxl-streamer | Enabled | Not Running |
          voxl-tag-detector | Disabled | Not Running |
          voxl-tflite-server | Disabled | Not Running |
          voxl-time-sync | Disabled | Not Running |
          voxl-uvc-server | Enabled | Running | 0.0%
          voxl-vision-hub | Enabled | Not Running |
          voxl-wait-for-fs | Enabled | Completed |

          And I am still getting the exact same segfault.

          Róbert DévényiR 1 Reply Last reply Reply Quote 0
          • Róbert DévényiR
            Róbert Dévényi @Róbert Dévényi
            last edited by

            loading config file
            Enabling debug messages

            width: 640
            height: 480
            fps: 30
            pipe_name: uvc

            voxl-uvc-server starting
            Image resolution 640x480, 30 fps chosen
            Vendor ID 0x534d chosen
            UVC initialized
            Device 534d:0000 found
            Device opened
            uvc_get_stream_ctrl_format_size succeeded for format YUYV
            Streaming starting

            Segmentation fault:
            Fault thread: voxl-uvc-server(tid: 16852)
            Fault address: 0x7f7dda3000
            Access to this address is not allowed.
            Segmentation fault

            1 Reply Last reply Reply Quote 0
            • Róbert DévényiR
              Róbert Dévényi @Eric Katzfey
              last edited by

              @Eric-Katzfey I don't know if it helps but whenever I run voxl-uvc-server, the segfault happens and after that the voxl-uvc-server service stops running, but is still enabled.

              Eric KatzfeyE 1 Reply Last reply Reply Quote 0
              • Eric KatzfeyE
                Eric Katzfey ModalAI Team @Róbert Dévényi
                last edited by

                @Róbert-Dévényi The service just runs the application. If the application stops then the service stops. Are you familiar with Linux and systemd? It's just a standard Linux startup mechanism. But you want to stop all services, even for voxl-uvc-server, then try to start voxl-uvc-server on the command line and see if it still segfaults. There are ways to debug segfaults. But you would need to understand how to get Linux to generate a core file after the segfault and how to load the core into gdb.

                Róbert DévényiR 2 Replies Last reply Reply Quote 0
                • Róbert DévényiR
                  Róbert Dévényi @Eric Katzfey
                  last edited by

                  @Eric-Katzfey I would appreciate it if you could tell me what commands to use to stop all the services and then restart voxl-uvc-server or any resource that helps with debugging segfaults in general.

                  1 Reply Last reply Reply Quote 0
                  • Róbert DévényiR
                    Róbert Dévényi @Eric Katzfey
                    last edited by

                    @Eric-Katzfey I reset all mpa services to factory settings and I keep getting the same segfault.

                    Alex KushleyevA 1 Reply Last reply Reply Quote 0
                    • Alex KushleyevA
                      Alex Kushleyev ModalAI Team @Róbert Dévényi
                      last edited by Alex Kushleyev

                      @Róbert-Dévényi ,

                      You can stop the systemd service for voxl-uvc-server using the following command:

                      systemctl stop voxl-uvc-server
                      this will stop the service / server but will not prevent it from auto-starting during next reboot

                      Then you can run the voxl-uvc-server in foreground. Just run it with the normal arguments:

                      voxl-uvc-server <your additional arguments here>
                      

                      Confirm that this still crashes.. then run the same executable in gdb:

                      > gdb /usr/bin/voxl-uvc-server
                      (gdb) r <your additional arguments here>
                      ...
                      wait for crash...
                      ...
                      bt
                      

                      bt command in gdb will print the backtrace of the call stack and will tell you which function the seg fault happened in (and you will see the call stack). Maybe this will help with debugging.

                      Alex

                      Róbert DévényiR 2 Replies Last reply Reply Quote 0
                      • Róbert DévényiR
                        Róbert Dévényi @Alex Kushleyev
                        last edited by

                        @Alex-Kushleyev Thank you very much hopefully I can figure it out.

                        1 Reply Last reply Reply Quote 0
                        • Róbert DévényiR
                          Róbert Dévényi @Alex Kushleyev
                          last edited by

                          @Alex-Kushleyev I can probably show you the backtrace log so that you can take a look at what causes the segfault.

                          A 1 Reply Last reply Reply Quote 0
                          • A
                            AndriiHlyvko @Róbert Dévényi
                            last edited by

                            @Róbert-Dévényi @Eric-Katzfey @Alex-Kushleyev I am getting the same seg fault with a boson+ camera. I determined that the seg fault occurs when the uvc_start_streaming function is called (I am running a modified version of voxl-uvc-server). I have the stack trace from gdb. If I figure out the cause of the problem I will post it here.

                            (gdb) run
                            Starting program: /usr/bin/voxl-uvc-server 
                            [Thread debugging using libthread_db enabled]
                            Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
                            VOXL UVC Server
                            loading config file
                            VOXL UVC Server parsed argument
                            voxl-uvc-server starting
                            Image resolution 640x512, 30 fps chosen
                            [New Thread 0x7ff7a7e1c0 (LWP 2189)]
                            UVC initialized
                            Device found
                            [New Thread 0x7ff727d1c0 (LWP 2190)]
                            Device opened
                            Found desired frame format: 1
                            uvc_get_stream_ctrl_format_size succeeded for format NV12
                            bmHint: 0001
                            bFormatIndex: 2
                            bFrameIndex: 1
                            dwFrameInterval: 333333
                            wKeyFrameRate: 1
                            wPFrameRate: 0
                            wCompQuality: 0
                            wCompWindowSize: 0
                            wDelay: 0
                            dwMaxVideoFrameSize: 491520
                            dwMaxPayloadTransferSize: 983044
                            bInterfaceNumber: 1
                            [New Thread 0x7ff6a7c1c0 (LWP 2191)]
                            [New Thread 0x7ff627b1c0 (LWP 2192)]
                            [New Thread 0x7ff5a7a1c0 (LWP 2193)]
                            Mavlink Onboard server Connected 
                            [New Thread 0x7fe70ef1c0 (LWP 2194)]
                            Streaming starting
                            [New Thread 0x7feffff1c0 (LWP 2195)]
                            
                            Thread 3 "voxl-uvc-server" received signal SIGSEGV, Segmentation fault.
                            [Switching to Thread 0x7ff727d1c0 (LWP 2190)]
                            __memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:185
                            185     ../sysdeps/aarch64/multiarch/../memcpy.S: No such file or directory.
                            (gdb) bt
                            #0  __memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:185
                            #1  0x0000007ff7ed32a0 in _uvc_process_payload () from /usr/lib/libuvc.so.0
                            Backtrace stopped: previous frame identical to this frame (corrupt stack?)
                            
                            Alex KushleyevA 1 Reply Last reply Reply Quote 0
                            • Alex KushleyevA
                              Alex Kushleyev ModalAI Team @AndriiHlyvko
                              last edited by Alex Kushleyev

                              @AndriiHlyvko , we have gotten another report about this recently. It seem there is a bug in libuvc which has been fixed, but we are using older version of the library.

                              Inserting the following check at the line below has been reported to fix the issue:

                              if (strmh->got_bytes + data_len > strmh->cur_ctrl.dwMaxVideoFrameSize) {
                                  UVC_DEBUG("Transfer too large and will overflow the buffer. Dropping data!");
                                  _uvc_swap_buffers(strmh);
                                  return;
                                }
                              

                              here: https://github.com/modalai/libuvc/blob/3fb2d2fd31f215f76bd5d6782785385fd0927ce5/src/stream.c#L757

                              It seems the fix in the official libuvc release is here:
                              https://github.com/libuvc/libuvc/blob/master/src/stream.c#L779
                              https://github.com/libuvc/libuvc/blob/master/src/stream.c#L787

                              We have not fixed this officially, please look at the code snippet i provided official fix and implement something similar until we fix it.

                              Alex

                              A 1 Reply Last reply Reply Quote 0
                              • A
                                AndriiHlyvko @Alex Kushleyev
                                last edited by

                                Thanks @Alex-Kushleyev . Do you know if there are any differences between using the voxl fork of libuvc vs compiling the master libuvc? I was thinking of just building the master libuvc but I am not sure if modalai did any changes to it.

                                Alex KushleyevA 1 Reply Last reply Reply Quote 0
                                • Alex KushleyevA
                                  Alex Kushleyev ModalAI Team @AndriiHlyvko
                                  last edited by

                                  @AndriiHlyvko , you can see the changes that we made to the original fork : https://github.com/modalai/libuvc/commits/modalai-master (just the three top commits).

                                  You may definitely try to use the latest master from the official repo, but we have not tested it. I believe there was some discussion recently about moving our code to latest, but we have not done it yet.

                                  Alex

                                  A 1 Reply Last reply Reply Quote 0
                                  • A
                                    AndriiHlyvko @Alex Kushleyev
                                    last edited by

                                    @Alex-Kushleyev just tested with latest master. The seg fault is gone and it seems to be working fine. One new problem that I have after moving to libuvc master is that the video stream from the boson+ looks wrong (the video is green 😕 ). Its only wrong for boson+ and not the boson. I'll keep debugging the issue. I was thinking maybe the boson+ sends incomplete frames in the uvc callback but I'm not sure.

                                    A 1 Reply Last reply Reply Quote 0
                                    • A
                                      AndriiHlyvko @AndriiHlyvko
                                      last edited by

                                      @Alex-Kushleyev figured it out. I had a bug that I introduced. I broke the mapping between UVC_FRAME_FORMAT enum and the voxl IMAGE_FORMAT enum in my fork of the uvc-server. Right now uvc streaming is working well with the boson+

                                      Alex KushleyevA 1 Reply Last reply Reply Quote 0
                                      • Alex KushleyevA
                                        Alex Kushleyev ModalAI Team @AndriiHlyvko
                                        last edited by

                                        @AndriiHlyvko , that's great!

                                        If you are willing to help out and share your changes on top of libuvc master, it should help us get this updated more quickly 🙂

                                        Thank you

                                        Alex

                                        A 1 Reply Last reply Reply Quote 0
                                        • A
                                          AndriiHlyvko @Alex Kushleyev
                                          last edited by

                                          @Alex-Kushleyev sure. Should I make a merge request for it and tag you in it?

                                          Alex KushleyevA 1 Reply Last reply Reply Quote 0
                                          • Alex KushleyevA
                                            Alex Kushleyev ModalAI Team @AndriiHlyvko
                                            last edited by

                                            @AndriiHlyvko , yes and please tag @Eric-Katzfey who will probably review and merge.

                                            Thank you!

                                            Alex

                                            A 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Powered by NodeBB | Contributors