ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    CAM_MISSING_STALLED

    GPS-denied Navigation (VIO)
    2
    19
    985
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Alex KushleyevA
      Alex Kushleyev ModalAI Team @Jetson Nano
      last edited by

      @Jetson-Nano , it seems like the tracking camera stops sending images - if this is happening during flight, perhaps there is some issue linked to vibration which is causing the camera image transmission from camera to the ISP to break. Please inspect camera flex cable and connection to VOXL2 board.

      Also, does voxl-camera-server stop with an error when this happens? you can check journal logs for the camera server process.

      Alex

      Jetson NanoJ 1 Reply Last reply Reply Quote 0
      • Jetson NanoJ
        Jetson Nano @Alex Kushleyev
        last edited by Jetson Nano

        @Alex-Kushleyev The Flex connectors seem to be fit in place. Maybe there could be possibility of the connectors losing connection points on the high speed connector points in the flex cables. Is there any way we can brace the connectors tight in place? Would you recommend something?

        Alex KushleyevA 1 Reply Last reply Reply Quote 0
        • Alex KushleyevA
          Alex Kushleyev ModalAI Team @Jetson Nano
          last edited by

          @Jetson-Nano, Normally, the camera connectors would not wiggle out of place (while staying inserted) if they are inserted correctly and they are not damaged. That is to say, if the connector stays physically inserted, vibration should not cause it to have bad contact.

          It is possible that the flex cable is damaged or has a fine crack in the copper traces. You can test it out by gently moving the flex cable while the camera is streaming (but do not pull hard or do not try to disconnect the connector by pulling on the flex).

          Alex

          Jetson NanoJ 1 Reply Last reply Reply Quote 0
          • Jetson NanoJ
            Jetson Nano @Alex Kushleyev
            last edited by Jetson Nano

            @Alex-Kushleyev But, the camera server is stopping. When the there is no vibrations, the drone is on ground and not even armed I am getting this error. When I run camera server it shows either killed or aborted. If there is damage in the flex cable it should not work even in the ground when it is stationary. It is stopping exactly after 5 sec of starting the server.

            If I configure the camera server only for tracking it is working fine, when I configure along with the ToF it is showing the error.

            Jetson NanoJ 1 Reply Last reply Reply Quote 0
            • Jetson NanoJ
              Jetson Nano @Jetson Nano
              last edited by

              @Alex-Kushleyev Here is the output for voxl-camera-server -d 0 only with ToF

              voxl-camera-server -d 0
              =================================================================
              configuration for 1 cameras:
              
              cam #0
                  name:                tof
                  sensor type:         pmd-tof
                  isEnabled:           1
                  camId:               0
                  camId2:              -1
                  fps:                 5
                  en_rotate:           0
                  en_rotate2:          0
              
                  en_preview:          1
                  pre_width:           224
                  pre_height:          1557
                  en_raw_preview:      1
              
                  en_small_video:      0
                  small_video_width:   -1
                  small_video_height:  -1
              
                  en_large_video:      0
                  large_video_width:   -1
                  large_video_height:  -1
              
                  en_snapshot:         0
                  snap_width:          -1
                  snap_height:         -1
              
                  ae_mode:             off
                  standby_enabled:     0
                  decimator:           5
                  independent_exposure:0
              
              fsync_en:            1
              fsync_gpio:          111
              =================================================================
              DEBUG:   Attempting to open the hal module
              DEBUG:   SUCCESS: Camera module opened on attempt 0
              DEBUG:   ----------- Number of cameras: 1
              
              DEBUG:   Cam idx: 0, Cam slot: 0, Slave Address: 0x007A, Sensor Id: 0x003D
              DEBUG:   ------ voxl-camera-server: Starting 1 cameras
              Starting Camera: tof (id #0)
              Connected to cpu-monitor
              DEBUG:   Checking Gain limits for Camera: tof
              Using gain limits min: 54 max: 8000
              DEBUG:   cam ID 0 checking for fmt:   38  w:  224 h: 1557 o:   0
              DEBUG:   	==> option   0: format= 34 width= 176 height= 144 is_input=0
              DEBUG:   	==> option   4: format= 34 width= 176 height= 144 is_input=1
              DEBUG:   	==> option   8: format= 35 width= 176 height= 144 is_input=0
              DEBUG:   	==> option  12: format= 35 width= 176 height= 144 is_input=1
              DEBUG:   	==> option  16: format= 33 width= 176 height= 144 is_input=0
              DEBUG:   	==> option  20: format= 37 width= 224 height=1557 is_input=0
              DEBUG:   	==> option  24: format= 38 width= 224 height=1557 is_input=0
              DEBUG:   	==> option  28: format= 32 width= 224 height=1557 is_input=0
              DEBUG:   	==> option  32: format= 36 width= 224 height=1557 is_input=0
              VERBOSE: Successfully found configuration match for camera 0: 224x1557
              VERBOSE: Adding preview stream for camera: 0
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              VERBOSE: Opened GBM fd
              gbm_create_device(156): Info: backend name is: msm_drm
              VERBOSE: Created GBM device
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Converted gralloc flags 0x20000 to GBM flags 0x400000
              VERBOSE: Dumping GBM flags
              DEBUG:   	Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI	
              DEBUG:   Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20000 format=GBM_FORMAT_RAW12
              DEBUG:   Successfully set up pipeline for stream: PREVIEW
              VERBOSE: Entered thread: cam0-request(tid: 5577)
              VERBOSE: Entered thread: cam0-result(tid: 5578)
              DEBUG:   Started Camera: tof
              
              ------ voxl-camera-server: Started 1 of 1 cameras
              
              ------ voxl-camera-server: Camera server is now running
              VERBOSE: Found Royale module config: imagerType 2, illuminationConfig.dutyCycle: 2 temp_sensor_type: 1
              VERBOSE: Found Royale module config: maxImgW 224 maxImgH 172 frameTxMode 1 camName CUSTOM
              VERBOSE: Found Royale module config: tempLimitSoft 60.000000 tempLimitHard 65.000000 autoExpoSupported yes
              VERBOSE: useCaseList size is 11
              VERBOSE: Usecase name MODE9_5
              VERBOSE: Found Royale usecase: MODE9_5 - phases: 9 fps: 5
              VERBOSE: Found Royale usecase: MODE9_5 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE9_5 - exposure group[1] = mod1
              VERBOSE: Found Royale usecase: MODE9_5 - exposure group[2] = mod2
              VERBOSE: Found Royale usecase: MODE9_5 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE9_5 - exp_limit[1] = (1 1200)
              VERBOSE: Found Royale usecase: MODE9_5 - exp_limit[2] = (1 1200)
              VERBOSE: Found Royale usecase: MODE9_5 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE9_5 - exp_time[1] = 1200
              VERBOSE: Found Royale usecase: MODE9_5 - exp_time[2] = 1200
              VERBOSE: Usecase name MODE9_1
              VERBOSE: Found Royale usecase: MODE9_1 - phases: 9 fps: 1
              VERBOSE: Found Royale usecase: MODE9_1 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE9_1 - exposure group[1] = mod1
              VERBOSE: Found Royale usecase: MODE9_1 - exposure group[2] = mod2
              VERBOSE: Found Royale usecase: MODE9_1 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE9_1 - exp_limit[1] = (1 1200)
              VERBOSE: Found Royale usecase: MODE9_1 - exp_limit[2] = (1 1200)
              VERBOSE: Found Royale usecase: MODE9_1 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE9_1 - exp_time[1] = 1200
              VERBOSE: Found Royale usecase: MODE9_1 - exp_time[2] = 1200
              VERBOSE: Usecase name MODE9_2
              VERBOSE: Found Royale usecase: MODE9_2 - phases: 9 fps: 2
              VERBOSE: Found Royale usecase: MODE9_2 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE9_2 - exposure group[1] = mod1
              VERBOSE: Found Royale usecase: MODE9_2 - exposure group[2] = mod2
              VERBOSE: Found Royale usecase: MODE9_2 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE9_2 - exp_limit[1] = (1 1200)
              VERBOSE: Found Royale usecase: MODE9_2 - exp_limit[2] = (1 1200)
              VERBOSE: Found Royale usecase: MODE9_2 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE9_2 - exp_time[1] = 1200
              VERBOSE: Found Royale usecase: MODE9_2 - exp_time[2] = 1200
              VERBOSE: Usecase name MODE9_3
              VERBOSE: Found Royale usecase: MODE9_3 - phases: 9 fps: 3
              VERBOSE: Found Royale usecase: MODE9_3 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE9_3 - exposure group[1] = mod1
              VERBOSE: Found Royale usecase: MODE9_3 - exposure group[2] = mod2
              VERBOSE: Found Royale usecase: MODE9_3 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE9_3 - exp_limit[1] = (1 1200)
              VERBOSE: Found Royale usecase: MODE9_3 - exp_limit[2] = (1 1200)
              VERBOSE: Found Royale usecase: MODE9_3 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE9_3 - exp_time[1] = 1200
              VERBOSE: Found Royale usecase: MODE9_3 - exp_time[2] = 1200
              VERBOSE: Usecase name MODE9_10
              VERBOSE: Found Royale usecase: MODE9_10 - phases: 9 fps: 10
              VERBOSE: Found Royale usecase: MODE9_10 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE9_10 - exposure group[1] = mod1
              VERBOSE: Found Royale usecase: MODE9_10 - exposure group[2] = mod2
              VERBOSE: Found Royale usecase: MODE9_10 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE9_10 - exp_limit[1] = (1 650)
              VERBOSE: Found Royale usecase: MODE9_10 - exp_limit[2] = (1 650)
              VERBOSE: Found Royale usecase: MODE9_10 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE9_10 - exp_time[1] = 650
              VERBOSE: Found Royale usecase: MODE9_10 - exp_time[2] = 650
              VERBOSE: Usecase name MODE9_15
              VERBOSE: Found Royale usecase: MODE9_15 - phases: 9 fps: 15
              VERBOSE: Found Royale usecase: MODE9_15 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE9_15 - exposure group[1] = mod1
              VERBOSE: Found Royale usecase: MODE9_15 - exposure group[2] = mod2
              VERBOSE: Found Royale usecase: MODE9_15 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE9_15 - exp_limit[1] = (1 450)
              VERBOSE: Found Royale usecase: MODE9_15 - exp_limit[2] = (1 450)
              VERBOSE: Found Royale usecase: MODE9_15 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE9_15 - exp_time[1] = 450
              VERBOSE: Found Royale usecase: MODE9_15 - exp_time[2] = 450
              VERBOSE: Usecase name MODE5_5
              VERBOSE: Found Royale usecase: MODE5_5 - phases: 5 fps: 5
              VERBOSE: Found Royale usecase: MODE5_5 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE5_5 - exposure group[1] = mod
              VERBOSE: Found Royale usecase: MODE5_5 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE5_5 - exp_limit[1] = (1 250)
              VERBOSE: Found Royale usecase: MODE5_5 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE5_5 - exp_time[1] = 250
              VERBOSE: Usecase name MODE5_10
              VERBOSE: Found Royale usecase: MODE5_10 - phases: 5 fps: 10
              VERBOSE: Found Royale usecase: MODE5_10 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE5_10 - exposure group[1] = mod
              VERBOSE: Found Royale usecase: MODE5_10 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE5_10 - exp_limit[1] = (1 250)
              VERBOSE: Found Royale usecase: MODE5_10 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE5_10 - exp_time[1] = 250
              VERBOSE: Usecase name MODE5_15
              VERBOSE: Found Royale usecase: MODE5_15 - phases: 5 fps: 15
              VERBOSE: Found Royale usecase: MODE5_15 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE5_15 - exposure group[1] = mod
              VERBOSE: Found Royale usecase: MODE5_15 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE5_15 - exp_limit[1] = (1 250)
              VERBOSE: Found Royale usecase: MODE5_15 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE5_15 - exp_time[1] = 250
              VERBOSE: Usecase name MODE5_30
              VERBOSE: Found Royale usecase: MODE5_30 - phases: 5 fps: 30
              VERBOSE: Found Royale usecase: MODE5_30 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE5_30 - exposure group[1] = mod
              VERBOSE: Found Royale usecase: MODE5_30 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE5_30 - exp_limit[1] = (1 250)
              VERBOSE: Found Royale usecase: MODE5_30 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE5_30 - exp_time[1] = 250
              VERBOSE: Usecase name MODE5_45
              VERBOSE: Found Royale usecase: MODE5_45 - phases: 5 fps: 45
              VERBOSE: Found Royale usecase: MODE5_45 - exposure group[0] = gray
              VERBOSE: Found Royale usecase: MODE5_45 - exposure group[1] = mod
              VERBOSE: Found Royale usecase: MODE5_45 - exp_limit[0] = (100 100)
              VERBOSE: Found Royale usecase: MODE5_45 - exp_limit[1] = (1 250)
              VERBOSE: Found Royale usecase: MODE5_45 - exp_time[0] = 100
              VERBOSE: Found Royale usecase: MODE5_45 - exp_time[1] = 250
              DEBUG:   Configuring TOF module for A65
              VERBOSE: sid: 0x00AC, addr: 0x0000, data: 0x0050
              VERBOSE: sid: 0x00AC, addr: 0x0001, data: 0x004D
              VERBOSE: sid: 0x00AC, addr: 0x0002, data: 0x0044
              VERBOSE: sid: 0x00AC, addr: 0x0003, data: 0x0054
              VERBOSE: sid: 0x00AC, addr: 0x0004, data: 0x0045
              VERBOSE: sid: 0x00AC, addr: 0x0005, data: 0x0043
              VERBOSE: sid: 0x00AC, addr: 0x0006, data: 0x0007
              VERBOSE: sid: 0x00AC, addr: 0x0007, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0008, data: 0x004B
              VERBOSE: sid: 0x00AC, addr: 0x0009, data: 0x0086
              VERBOSE: sid: 0x00AC, addr: 0x000A, data: 0x0051
              VERBOSE: sid: 0x00AC, addr: 0x000B, data: 0x0014
              VERBOSE: sid: 0x00AC, addr: 0x000C, data: 0x002A
              VERBOSE: sid: 0x00AC, addr: 0x000D, data: 0x00EB
              VERBOSE: sid: 0x00AC, addr: 0x000E, data: 0x0001
              VERBOSE: sid: 0x00AC, addr: 0x000F, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0010, data: 0x0003
              VERBOSE: sid: 0x00AC, addr: 0x0011, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0012, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0013, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0014, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0015, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0016, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0017, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0018, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x0019, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x001A, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x001B, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x001C, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x001D, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x001E, data: 0x0000
              VERBOSE: sid: 0x00AC, addr: 0x001F, data: 0x0071
              VERBOSE: sid: 0x00AC, addr: 0x0020, data: 0x0050
              VERBOSE: sid: 0x00AC, addr: 0x0021, data: 0x004D
              VERBOSE: sid: 0x00AC, addr: 0x0022, data: 0x0044
              VERBOSE: sid: 0x00AC, addr: 0x0023, data: 0x0054
              VERBOSE: sid: 0x00AC, addr: 0x0024, data: 0x004F
              VERBOSE: sid: 0x00AC, addr: 0x0025, data: 0x0046
              VERBOSE: sid: 0x00AC, addr: 0x0026, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x0027, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x0028, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x0029, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x002A, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x002B, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x002C, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x002D, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x002E, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x002F, data: 0x005F
              VERBOSE: sid: 0x00AC, addr: 0x0030, data: 0x0041
              VERBOSE: sid: 0x00AC, addr: 0x0031, data: 0x0043
              VERBOSE: sid: 0x00AC, addr: 0x0032, data: 0x0033
              VERBOSE: sid: 0x00AC, addr: 0x0033, data: 0x0030
              VERBOSE: sid: 0x00AC, addr: 0x0034, data: 0x002D
              VERBOSE: sid: 0x00AC, addr: 0x0035, data: 0x0030
              VERBOSE: sid: 0x00AC, addr: 0x0036, data: 0x0039
              VERBOSE: sid: 0x00AC, addr: 0x0037, data: 0x0044
              VERBOSE: sid: 0x00AC, addr: 0x0038, data: 0x0042
              VERBOSE: sid: 0x00AC, addr: 0x0039, data: 0x002D
              VERBOSE: sid: 0x00AC, addr: 0x003A, data: 0x0041
              VERBOSE: sid: 0x00AC, addr: 0x003B, data: 0x0030
              VERBOSE: sid: 0x00AC, addr: 0x003C, data: 0x0032
              VERBOSE: sid: 0x00AC, addr: 0x003D, data: 0x0034
              VERBOSE: sid: 0x00AC, addr: 0x003E, data: 0x002D
              VERBOSE: sid: 0x00AC, addr: 0x003F, data: 0x0031
              VERBOSE: sid: 0x00AC, addr: 0x0040, data: 0x0041
              VERBOSE: sid: 0x00AC, addr: 0x0041, data: 0x0033
              VERBOSE: sid: 0x00AC, addr: 0x0042, data: 0x0030
              VERBOSE: sid: 0x007A, addr: 0xA0A5, data: 0x0B12
              VERBOSE: sid: 0x007A, addr: 0xA007, data: 0x0001
              VERBOSE: sid: 0x007A, addr: 0xA00A, data: 0x0007
              VERBOSE: sid: 0x007A, addr: 0xA014, data: 0x0007
              VERBOSE: sid: 0x007A, addr: 0xA01C, data: 0x131A
              VERBOSE: sid: 0x007A, addr: 0xA063, data: 0x01C5
              VERBOSE: sid: 0x007A, addr: 0xA065, data: 0x0206
              VERBOSE: sid: 0x007A, addr: 0xA066, data: 0x01C5
              VERBOSE: sid: 0x007A, addr: 0xA068, data: 0x0206
              VERBOSE: sid: 0x007A, addr: 0xA007, data: 0x0001
              VERBOSE: sid: 0x007A, addr: 0xA00A, data: 0x0007
              VERBOSE: sid: 0x007A, addr: 0xA014, data: 0x0007
              VERBOSE: sid: 0x007A, addr: 0xA01C, data: 0x131A
              VERBOSE: sid: 0x007A, addr: 0xA063, data: 0x01C5
              VERBOSE: sid: 0x007A, addr: 0xA065, data: 0x0206
              VERBOSE: sid: 0x007A, addr: 0xA066, data: 0x01C5
              VERBOSE: sid: 0x007A, addr: 0xA068, data: 0x0206
              VERBOSE: sid: 0x007A, addr: 0xA084, data: 0x1513
              VERBOSE: sid: 0x007A, addr: 0x9838, data: 0x0000
              VERBOSE: sid: 0x007A, addr: 0x9835, data: 0x0192
              VERBOSE: sid: 0x007A, addr: 0xB402, data: 0x0080
              VERBOSE: sid: 0x007A, addr: 0xA800, data: 0x00E3
              VERBOSE: sid: 0x007A, addr: 0xA801, data: 0x06D6
              VERBOSE: sid: 0x007A, addr: 0xA802, data: 0x00E0
              VERBOSE: sid: 0x007A, addr: 0xA803, data: 0x0604
              VERBOSE: sid: 0x007A, addr: 0xA804, data: 0x00E0
              VERBOSE: sid: 0x007A, addr: 0xA805, data: 0x0F0D
              VERBOSE: sid: 0x007A, addr: 0xA806, data: 0x0A16
              VERBOSE: sid: 0x007A, addr: 0xA807, data: 0x0890
              VERBOSE: sid: 0x007A, addr: 0xA808, data: 0x0442
              VERBOSE: sid: 0x007A, addr: 0xA809, data: 0x0A02
              VERBOSE: sid: 0x007A, addr: 0xA80A, data: 0x00B0
              VERBOSE: sid: 0x007A, addr: 0xA80B, data: 0x0C40
              VERBOSE: sid: 0x007A, addr: 0xA80C, data: 0x0040
              VERBOSE: sid: 0x007A, addr: 0xA80D, data: 0x0200
              VERBOSE: sid: 0x007A, addr: 0xA80E, data: 0x088E
              VERBOSE: sid: 0x007A, addr: 0xA80F, data: 0x0472
              VERBOSE: sid: 0x007A, addr: 0xA810, data: 0x0888
              VERBOSE: sid: 0x007A, addr: 0xA811, data: 0x0459
              VERBOSE: sid: 0x007A, addr: 0xA812, data: 0x0A0E
              VERBOSE: sid: 0x007A, addr: 0xA813, data: 0x0B06
              VERBOSE: sid: 0x007A, addr: 0xA814, data: 0x0890
              VERBOSE: sid: 0x007A, addr: 0xA815, data: 0x0450
              VERBOSE: sid: 0x007A, addr: 0xA816, data: 0x0888
              VERBOSE: sid: 0x007A, addr: 0xA817, data: 0x0D59
              VERBOSE: sid: 0x007A, addr: 0xA818, data: 0x0890
              VERBOSE: sid: 0x007A, addr: 0xA819, data: 0x0C51
              VERBOSE: sid: 0x007A, addr: 0xA81A, data: 0x0230
              VERBOSE: sid: 0x007A, addr: 0xA81B, data: 0x0398
              VERBOSE: sid: 0x007A, addr: 0xAD26, data: 0x053B
              VERBOSE: sid: 0x007A, addr: 0xAD27, data: 0x003D
              VERBOSE: sid: 0x007A, addr: 0xAD28, data: 0x001F
              VERBOSE: sid: 0x007A, addr: 0xAD29, data: 0x0848
              VERBOSE: sid: 0x007A, addr: 0xAD2A, data: 0x0811
              VERBOSE: sid: 0x007A, addr: 0xAD2B, data: 0x0DBB
              VERBOSE: sid: 0x007A, addr: 0xB3A5, data: 0x5712
              VERBOSE: sid: 0x007A, addr: 0xB3A6, data: 0x0002
              DEBUG:   [ROYALE_INFO] {ROYALE_PROCESSING}: Spectre backend : ng_arm
              DEBUG:   setup accessLevel 3, ret 0
              DEBUG:   Royale getUseCases returned No error
              DEBUG:   List of usecases:
              DEBUG:   	Usecase 0: MODE9_5
              DEBUG:   	Usecase 1: MODE9_1
              DEBUG:   	Usecase 2: MODE9_2
              DEBUG:   	Usecase 3: MODE9_3
              DEBUG:   	Usecase 4: MODE9_10
              DEBUG:   	Usecase 5: MODE9_15
              DEBUG:   	Usecase 6: MODE5_5
              DEBUG:   	Usecase 7: MODE5_10
              DEBUG:   	Usecase 8: MODE5_15
              DEBUG:   	Usecase 9: MODE5_30
              DEBUG:   	Usecase 10: MODE5_45
              DEBUG:   Attempting to set use case 'MODE9_5'
              DEBUG:   Changed to usecase MODE9_5 exposure time limits [ 1 - 1200 ], cur exposure_time 1200, exposure mode 1
              DEBUG:   Successfully set Royale exposure mode to AUTOMATICVERBOSE: 
              Setting use case: mode=9, fps = 5
              DEBUG:   Royale getUseCases returned No error
              DEBUG:   List of usecases:
              DEBUG:   	Usecase 0: MODE9_5
              DEBUG:   	Usecase 1: MODE9_1
              DEBUG:   	Usecase 2: MODE9_2
              DEBUG:   	Usecase 3: MODE9_3
              DEBUG:   	Usecase 4: MODE9_10
              DEBUG:   	Usecase 5: MODE9_15
              DEBUG:   	Usecase 6: MODE5_5
              DEBUG:   	Usecase 7: MODE5_10
              DEBUG:   	Usecase 8: MODE5_15
              DEBUG:   	Usecase 9: MODE5_30
              DEBUG:   	Usecase 10: MODE5_45
              DEBUG:   Attempting to set use case 'MODE9_5'
              DEBUG:   Changed to usecase MODE9_5 exposure time limits [ 1 - 1200 ], cur exposure_time 1200, exposure mode 1
              VERBOSE: startCapture E
              VERBOSE: sid: 0x007A, addr: 0x9005, data: 0x0002
              VERBOSE: sid: 0x007A, addr: 0xB420, data: 0x1000
              VERBOSE: sid: 0x007A, addr: 0xB420, data: 0x1040
              VERBOSE: sid: 0x007A, addr: 0xB400, data: 0x0001
              VERBOSE: sid: 0x007A, addr: 0x9000, data: 0x0005
              VERBOSE: sid: 0x007A, addr: 0x9005, data: 0x0000
              VERBOSE: sid: 0x007A, addr: 0x982F, data: 0x0000
              VERBOSE: sid: 0x007A, addr: 0x9830, data: 0x0000
              VERBOSE: sid: 0x007A, addr: 0x982F, data: 0x0001
              VERBOSE: startCapture X
              VERBOSE: TOF interface created!
              VERBOSE: setting hal3 exposure for tof     2259763   200
              VERBOSE: added request for preview stream
              VERBOSE: Sending request for frame 0 for camera tof for 1 streams
              VERBOSE: Received result from HAl3 for frame number 0
              VERBOSE: Received 0 buffers from camera tof, partial result:1
              VERBOSE: finished sending request for frame 0 for camera tof
              VERBOSE: returning from SendOneCaptureRequest for frame 1 for camera tof
              VERBOSE: setting hal3 exposure for tof     2259763   200
              VERBOSE: added request for preview stream
              VERBOSE: Sending request for frame 1 for camera tof for 1 streams
              VERBOSE: finished sending request for frame 1 for camera tof
              VERBOSE: returning from SendOneCaptureRequest for frame 2 for camera tof
              VERBOSE: setting hal3 exposure for tof     2259763   200
              VERBOSE: added request for preview stream
              VERBOSE: Sending request for frame 2 for camera tof for 1 streams
              VERBOSE: finished sending request for frame 2 for camera tof
              VERBOSE: returning from SendOneCaptureRequest for frame 3 for camera tof
              VERBOSE: setting hal3 exposure for tof     2259763   200
              VERBOSE: added request for preview stream
              VERBOSE: Sending request for frame 3 for camera tof for 1 streams
              VERBOSE: finished sending request for frame 3 for camera tof
              VERBOSE: returning from SendOneCaptureRequest for frame 4 for camera tof
              VERBOSE: setting hal3 exposure for tof     2259763   200
              VERBOSE: added request for preview stream
              VERBOSE: Sending request for frame 4 for camera tof for 1 streams
              VERBOSE: finished sending request for frame 4 for camera tof
              VERBOSE: returning from SendOneCaptureRequest for frame 5 for camera tof
              VERBOSE: setting hal3 exposure for tof     2259763   200
              VERBOSE: added request for preview stream
              VERBOSE: Sending request for frame 5 for camera tof for 1 streams
              VERBOSE: Received result from HAl3 for frame number 1
              VERBOSE: Received 0 buffers from camera tof, partial result:1
              VERBOSE: Received result from HAl3 for frame number 2
              VERBOSE: Received 0 buffers from camera tof, partial result:1
              VERBOSE: Received result from HAl3 for frame number 3
              VERBOSE: Received 0 buffers from camera tof, partial result:1
              VERBOSE: Received result from HAl3 for frame number 0
              VERBOSE: Received 0 buffers from camera tof, partial result:2
              VERBOSE: Received metadata for frame 0 from camera tof
              VERBOSE: 	Timestamp: 786183245685
              VERBOSE: 	Gain: 0
              VERBOSE: 	Exposure: 0
              VERBOSE: Received result from HAl3 for frame number 4
              VERBOSE: Received 0 buffers from camera tof, partial result:1
              DEBUG:   Value of standby_active is: 0 
              DEBUG:   Value of standby_active is: 0 
              DEBUG:   Value of standby_active is: 0 
              DEBUG:   Value of standby_active is: 0 
              DEBUG:   Value of standby_active is: 0 
              Aborted
              
              
              Alex KushleyevA 1 Reply Last reply Reply Quote 0
              • Alex KushleyevA
                Alex Kushleyev ModalAI Team @Jetson Nano
                last edited by

                @Jetson-Nano , Yes it seems like TOF sensor is not working properly. Can you please repeat the test with only TOF and enable some debugging:

                echo 0xf > /sys/kernel/debug/camera_ife/ife_csid_debug
                

                Then start dmesg -w in a separate terminal and restart camera server. Under normal conditions, you should be seeing messages like below (SOF stands for Start of Frame and EOF stands for End of Frame). This would also show up any errors in transmission between TOF camera and VOXL2.

                CAM_INFO: CAM-ISP: cam_ife_csid_irq: 5210 CSID RDI:0 SOF received
                CAM_INFO: CAM-ISP: cam_ife_csid_irq: 5219 CSID RDI:0 EOF received
                
                Jetson NanoJ 1 Reply Last reply Reply Quote 0
                • Jetson NanoJ
                  Jetson Nano @Alex Kushleyev
                  last edited by Jetson Nano

                  @Alex-Kushleyev thanks for reverting, meanwhile running this test, should I have physically other cameras(ie. ov7251 tracking) disconnected from the system or just have them disabled in the camera config file?

                  Alex KushleyevA 1 Reply Last reply Reply Quote 0
                  • Alex KushleyevA
                    Alex Kushleyev ModalAI Team @Jetson Nano
                    last edited by

                    @Jetson-Nano , just disabled in the config file is fine, but you can try to see if physically disconnecting them would make a difference. May be a good test to do that.

                    Jetson NanoJ 1 Reply Last reply Reply Quote 0
                    • Jetson NanoJ
                      Jetson Nano @Alex Kushleyev
                      last edited by Jetson Nano

                      @Alex-Kushleyev I did the test, with ToF enabled in config with others disabled and physically also removed the other cameras(tracking, hires) and got the following output,

                      [  500.965614] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x40d00
                      [  513.797938] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x40d00
                      [  563.809259] CAM_ERR: CAM-MEM: cam_mem_mgr_create_debug_fs: 126 failed to create dentry
                      [  563.837666] CAM_INFO: CAM-HFI: cam_hfi_init: 878 Init IO1 : [0x10c00000 0xcf300000] IO2 [0xe0200000 0x1ed00000]
                      [  563.847825] CAM_INFO: CAM-ICP: cam_icp_mgr_hw_open: 3879 FW download done successfully
                      [  563.847979] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079,OEM_IMAGE_VERSION_STRING=CRM,BUILD_TIME: Oct 17 2019 05:49:19,CACHE_ENABLED at icphostinterface.c:636 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                      [  563.847983] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :ELF variant: CACHE-ENABLED:T480:API_V2:USE_CDM_1_1: , API version: 0x2000049 at icphostinterface.c:637 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                      [  564.015546] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.60
                      [  564.015606] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.79
                      [  564.042685] CAM_ERR: CAM-CCI: cam_cci_irq: 238 Base:        pK-error, M0_Q1 NACK ERROR: 0x10000000
                      [  564.042909] CAM_ERR: CAM-CCI: cam_cci_read: 1329 ERROR with Slave 0x7a:
                      [  564.042926] CAM_ERR: CAM-CCI: cam_cci_read_bytes: 1613 failed to read rc:-22
                      [  564.042935] CAM_ERR: CAM-SENSOR: cam_cci_i2c_read: 35 rc = -22
                      [  564.042945] CAM_WARN: CAM-SENSOR: cam_sensor_match_id: 707 read id: 0x0 expected id 0x3d:
                      [  564.043137] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.79
                      [  564.047327] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.60
                      [  564.076125] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.60
                      [  564.076278] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.79
                      [  564.093922] CAM_ERR: CAM-CCI: cam_cci_irq: 238 Base:        pK-error, M0_Q1 NACK ERROR: 0x10000000
                      [  564.093988] CAM_ERR: CAM-CCI: cam_cci_read: 1329 ERROR with Slave 0x20:
                      [  564.094000] CAM_ERR: CAM-CCI: cam_cci_read_bytes: 1613 failed to read rc:-22
                      [  564.094008] CAM_ERR: CAM-SENSOR: cam_cci_i2c_read: 35 rc = -22
                      [  564.094016] CAM_WARN: CAM-SENSOR: cam_sensor_match_id: 707 read id: 0x0 expected id 0x214:
                      [  564.094138] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.79
                      [  564.098341] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.60
                      [  564.129852] qcom,camera ac50000.qcom,cci:qcom,cam-sensor3: Linked as a consumer to regulator.55
                      [  564.132377] qcom,camera ac50000.qcom,cci:qcom,cam-sensor3: Linked as a consumer to regulator.60
                      [  564.132522] qcom,camera ac50000.qcom,cci:qcom,cam-sensor3: Linked as a consumer to regulator.79
                      [  564.156581] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 731 [Applying Alternative Address] : Sensor Index   : 391549108
                      [  564.156594] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 732 [Applying Alternative Address] : Slave Address  : 0xE2
                      [  564.156602] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 733 [Applying Alternative Address] : Slave sensor id: 0x7750
                      [  564.156610] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 734 [Applying Alternative Address] : cci_client sid : 0x71
                      [  564.156830] CAM_ERR: CAM-CCI: cam_cci_irq: 258 Base:        pK-error, M1_Q0 NACK ERROR: 0x20000000
                      [  564.156882] CAM_ERR: CAM-CCI: cam_cci_wait: 270 failed rc -22
                      [  564.156891] CAM_ERR: CAM-CCI: cam_cci_transfer_end: 345 failed rc -22
                      [  564.156899] CAM_ERR: CAM-CCI: cam_cci_data_queue: 870 Slave: 0xe0 failed rc -22
                      [  564.156907] CAM_ERR: CAM-CCI: cam_cci_i2c_write: 1450 failed rc: -22
                      [  564.156916] CAM_ERR: CAM-SENSOR: cam_cci_i2c_write_table_cmd: 123 Failed rc = -22
                      [  564.156924] CAM_ERR: CAM-SENSOR: cam_sensor_set_alt_id: 768 [Applying Alternative Address] : Probe : camera_io_dev_write failed: rc=-22
                      [  564.157075] CAM_ERR: CAM-CCI: cam_cci_irq: 264 Base:        pK-error, M1_Q1 NACK ERROR: 0x40000000
                      [  564.157195] CAM_ERR: CAM-CCI: cam_cci_read: 1329 ERROR with Slave 0xe0:
                      [  564.157203] CAM_ERR: CAM-CCI: cam_cci_read_bytes: 1613 failed to read rc:-22
                      [  564.157210] CAM_ERR: CAM-SENSOR: cam_cci_i2c_read: 35 rc = -22
                      [  564.157219] CAM_WARN: CAM-SENSOR: cam_sensor_match_id: 707 read id: 0x0 expected id 0x7750:
                      [  564.157384] qcom,camera ac50000.qcom,cci:qcom,cam-sensor3: Dropping the link to regulator.79
                      [  564.161659] qcom,camera ac50000.qcom,cci:qcom,cam-sensor3: Dropping the link to regulator.60
                      [  564.161850] qcom,camera ac50000.qcom,cci:qcom,cam-sensor3: Dropping the link to regulator.55
                      [  564.214462] CAM_WARN: CAM-CRM: cam_req_mgr_close: 160 release invoked associated userspace process has died
                      [  593.797336] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x40d00
                      [  593.797362] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x40d00
                      
                      Jetson NanoJ 1 Reply Last reply Reply Quote 0
                      • Jetson NanoJ
                        Jetson Nano @Jetson Nano
                        last edited by

                        @Jetson-Nano I tried doing the same above process of checking for Hires IMX214 cam also, found the following using dmesg -w. I configured C-8 configuration upon J7-U port.

                        [ 1414.886039] cam_csid_evt_bottom_half_handler: 12 callbacks suppressed
                        [ 1414.886043] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status RDI0: 0
                        [ 1414.886046] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status RDI1: 0
                        [ 1414.886049] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status RDI2: 0
                        [ 1414.886053] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status RDI3: 0
                        [ 1414.886056] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status TOP: 0
                        [ 1414.886060] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status RX: 38000
                        [ 1414.886063] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status IPP: 0
                        [ 1414.886066] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status PPP: 0
                        [ 1414.886070] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status UDI0: 0
                        [ 1414.886073] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4621 status UDI1: 0
                        [ 1414.886081] CAM_ERR: CAM-ISP: cam_csid_evt_bottom_half_handler: 4616 idx 1 err 5 phy 3  lane type:0 ln num:4 ln cfg:0x3210 cnt 2
                        [ 1416.258821] msm_vidc:   err : ffffffff: .....: DBLP Set: status 0
                        [ 1420.000213] CAM_ERR: CAM-CRM: __cam_req_mgr_process_sof_freeze: 1800 SOF freeze for session 8520192 link 0xb50105
                        [ 1420.000250] CAM_INFO: CAM-ISP: cam_ife_csid_sof_irq_debug: 4250 SOF freeze: CSID SOF irq enabled
                        [ 1420.000265] CAM_INFO: CAM-ISP: cam_ife_csid_sof_irq_debug: 4250 SOF freeze: CSID SOF irq enabled
                        
                        Alex KushleyevA 1 Reply Last reply Reply Quote 0
                        • Alex KushleyevA
                          Alex Kushleyev ModalAI Team @Jetson Nano
                          last edited by

                          @Jetson-Nano , SOF freeze condition is when the ISP never gets a frame and times out. Something strange is going on.

                          Are any cameras working at all?

                          Your original post stated that your cameras would stop working in mid flight, but now it seems they dont work at all?

                          Alex

                          Jetson NanoJ 1 Reply Last reply Reply Quote 0
                          • Jetson NanoJ
                            Jetson Nano @Alex Kushleyev
                            last edited by Jetson Nano

                            @Alex-Kushleyev Based on the above guidelines you mentioned I tried debugging the system messages along with pasting bin needed for respective ports. The ToF Also I made some interesting observation. ToF when plugged and seated firmly at certain angle works for me. Initially I tested with only ToF with required bin files in place. I receive the following output,

                            ToF is seated on J6L port

                             643.046323] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  643.046350] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  643.046369] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  643.046388] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  643.046406] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  643.046425] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  643.046443] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  643.046461] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  643.046480] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  670.810479] ufshcd-qcom 1d84000.ufshc: ufshcd_ioctl: User buffer is NULL!
                            [  693.797692] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x40d00
                            [  700.336130] hrtimer: interrupt took 105417 ns
                            [  721.252079] ufshcd-qcom 1d84000.ufshc: ufshcd_ioctl: User buffer is NULL!
                            [  747.535072] CAM_ERR: CAM-MEM: cam_mem_mgr_create_debug_fs: 126 failed to create dentry
                            [  747.564180] CAM_INFO: CAM-HFI: cam_hfi_init: 878 Init IO1 : [0x10c00000 0xcf300000] IO2 [0xe0200000 0x1ed00000]
                            [  747.574301] CAM_INFO: CAM-ICP: cam_icp_mgr_hw_open: 3879 FW download done successfully
                            [  747.574309] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079,OEM_IMAGE_VERSION_STRING=CRM,BUILD_TIME: Oct 17 2019 05:49:19,CACHE_ENABLED at icphostinterface.c:636 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                            [  747.574315] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :ELF variant: CACHE-ENABLED:T480:API_V2:USE_CDM_1_1: , API version: 0x2000049 at icphostinterface.c:637 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                            [  747.667487] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.60
                            [  747.667526] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.79
                            [  747.696358] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 918 Probe success,slot:2,slave_addr:0x7a,sensor_id:0x3d
                            [  747.696481] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.79
                            [  747.700211] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.60
                            [  747.744388] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.60
                            [  747.744579] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.79
                            [  747.767225] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 997 CAM_ACQUIRE_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                            [  748.124228] CAM_ERR: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 710 Successfully acquire single IFE[6 -1] with [0 pix] [0 pd] [1 rdi] ports for ctx:0
                            [  748.124972] CAM_ERR: CAM-CRM: cam_req_mgr_cb_add_req: 2742 req 1 not found in in_q
                            [  748.125219] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 1137 START_DEV: CSIPHY_IDX: 2, Device_slot: 0, Datarate: 288000000, Settletime: 2800000000
                            [  748.133931] CAM_INFO: CAM-ISP: cam_vfe_bus_ver3_init_hw: 3659 Overriding clock gating at bus input
                            [  748.133934] CAM_INFO: CAM-ISP: cam_vfe_top_ver3_init_hw: 246 Disable clock gating at IFE top
                            [  748.134108] CAM_ERR: CAM-ISP: cam_ife_mgr_start_hw: 4510 ->Config HW, 00000000a3bb1d21
                            [  748.134411] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1089 CAM_START_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                            [  748.146857] CAM_ERR: CAM-ISP: cam_ife_csid_irq: 4927 CSID:6 UNBOUNDED_FRAME
                            [  762.859114] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  762.859143] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                            [  835.472101] perf: interrupt took too long (3965 > 3960), lowering kernel.perf_event_max_sample_rate to 50250
                            

                            Further upon I tested with a Tracking Camera ov7251 attached, where I'm receiving data from the sensor and feed from sensors could be viewed upon the portal.

                            Here ToF is seated as earlier on J6 L port and Tracking ov7251 on J7 L port

                            [    5.505291] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.79
                            [    5.526921] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 731 [Applying Alternative Address] : Sensor Index   : 391385268
                            [    5.526926] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 732 [Applying Alternative Address] : Slave Address  : 0xE2
                            [    5.526930] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 733 [Applying Alternative Address] : Slave sensor id: 0x7750
                            [    5.526934] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 734 [Applying Alternative Address] : cci_client sid : 0x71
                            [    5.527197] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 770 [Applying Alternative Address] : Probe : Alternate address updated
                            [    5.527203] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 997 CAM_ACQUIRE_DEV Success, sensor_id:0x7750,sensor_slave_addr:0xe2
                            [    5.556310] CAM_ERR: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 710 Successfully acquire single IFE[6 -1] with [0 pix] [0 pd] [1 rdi] ports for ctx:0
                            [    5.557998] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 1137 START_DEV: CSIPHY_IDX: 2, Device_slot: 0, Datarate: 384000000, Settletime: 2800000000
                            [    5.567355] CAM_INFO: CAM-ISP: cam_vfe_bus_ver3_init_hw: 3659 Overriding clock gating at bus input
                            [    5.567362] CAM_INFO: CAM-ISP: cam_vfe_top_ver3_init_hw: 246 Disable clock gating at IFE top
                            [    5.567662] CAM_ERR: CAM-ISP: cam_ife_mgr_start_hw: 4510 ->Config HW, 00000000b07f5b5e
                            [    5.568263] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1089 CAM_START_DEV Success, sensor_id:0x7750,sensor_slave_addr:0xe2
                            [    5.710704] android_work: sent uevent USB_STATE=CONNECTED
                            [    5.720776] configfs-gadget gadget: high-speed config #1: c
                            [    5.721016] android_work: sent uevent USB_STATE=CONFIGURED
                            [    5.760593] CAM_ERR: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 710 Successfully acquire single IFE[5 -1] with [0 pix] [0 pd] [1 rdi] ports for ctx:1
                            [    5.761075] CAM_ERR: CAM-CRM: cam_req_mgr_cb_add_req: 2742 req 1 not found in in_q
                            [    5.761251] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 1137 START_DEV: CSIPHY_IDX: 0, Device_slot: 0, Datarate: 288000000, Settletime: 2800000000
                            [    5.769634] CAM_INFO: CAM-ISP: cam_vfe_bus_ver3_init_hw: 3659 Overriding clock gating at bus input
                            [    5.769638] CAM_INFO: CAM-ISP: cam_vfe_top_ver3_init_hw: 246 Disable clock gating at IFE top
                            [    5.769647] CAM_ERR: CAM-ISP: cam_ife_mgr_start_hw: 4510 ->Config HW, 00000000b07f5b5e
                            [    5.769921] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1089 CAM_START_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                            [    5.783209] CAM_ERR: CAM-ISP: cam_ife_csid_irq: 4927 CSID:5 UNBOUNDED_FRAME
                            [    6.030729] spi_geni 98c000.spi: tx_fifo 16 rx_fifo 16 tx_width 32
                            [    6.215921] Started adbd. 
                            [    6.374976] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
                            [    6.387472] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
                            [    6.391291] QTI:Netlink Query to Kernel Success
                            [   10.090752] devfreq-qcom-fw 18590000.qcom,devfreq-l3:qcom,cdsp-cdsp-l3-lat: Successfully started CDSP L3 governor
                            [   11.435643] netlink: 'hostapd': attribute type 213 has an invalid length.
                            [   11.489516] RTW: assoc success
                            [   11.489998] RTW: set group key camid:0, addr:00:c0:ca:b3:a3:e9, kid:1, type:AES
                            [   28.000258] CAM_INFO: CAM-CRM: cam_req_mgr_process_flush_req: 2177 Last request id to flush is 115
                            [   28.000507] CAM_INFO: CAM-ISP: __cam_isp_ctx_flush_req_in_top_state: 3046 Last request id to flush is 115
                            [   28.000960] CAM_INFO: CAM-ISP: __cam_isp_ctx_flush_req_in_top_state: 3069 Stop HW complete. Reset HW next.
                            [   28.005985] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1117 CAM_STOP_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                            [   28.006005] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 982 STOP_DEV: CSIPHY_IDX: 0, Device_slot: 0, Datarate: 288000000, Settletime: 2800000000
                            [   28.012879] CAM_INFO: CAM-ISP: cam_ife_mgr_release_hw: 4755 Release HW success ctx id: 1
                            [   28.014448] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.79
                            [   28.017762] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.60
                            [   28.018601] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1047 CAM_RELEASE_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                            [   28.066082] CAM_INFO: CAM-CRM: cam_req_mgr_process_flush_req: 2177 Last request id to flush is 679
                            [   28.066153] CAM_INFO: CAM-ISP: __cam_isp_ctx_flush_req_in_top_state: 3046 Last request id to flush is 679
                            [   28.066283] CAM_INFO: CAM-ISP: __cam_isp_ctx_flush_req_in_top_state: 3069 Stop HW complete. Reset HW next.
                            [   28.071811] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1117 CAM_STOP_DEV Success, sensor_id:0x7750,sensor_slave_addr:0xe2
                            [   28.071831] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 982 STOP_DEV: CSIPHY_IDX: 2, Device_slot: 0, Datarate: 384000000, Settletime: 2800000000
                            [   28.078536] CAM_INFO: CAM-ISP: cam_ife_mgr_release_hw: 4755 Release HW success ctx id: 0
                            [   28.080413] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.79
                            [   28.083593] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.60
                            [   28.084953] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1047 CAM_RELEASE_DEV Success, sensor_id:0x7750,sensor_slave_addr:0xe2
                            [   28.092411] rpmh_rsc_send_data: 1 callbacks suppressed
                            [   28.092417] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x30080
                            [   28.102533] CAM_WARN: CAM-CRM: cam_req_mgr_close: 160 release invoked associated userspace process has died
                            [   28.173983] CAM_ERR: CAM-MEM: cam_mem_mgr_create_debug_fs: 126 failed to create dentry
                            [   28.208650] CAM_INFO: CAM-HFI: cam_hfi_init: 878 Init IO1 : [0x10c00000 0xcf300000] IO2 [0xe0200000 0x1ed00000]
                            [   28.218782] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079,OEM_IMAGE_VERSION_STRING=CRM,BUILD_TIME: Oct 17 2019 05:49:19,CACHE_ENABLED at icphostinterface.c:636 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                            [   28.218793] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :ELF variant: CACHE-ENABLED:T480:API_V2:USE_CDM_1_1: , API version: 0x2000049 at icphostinterface.c:637 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                            [   28.218982] CAM_INFO: CAM-ICP: cam_icp_mgr_hw_open: 3879 FW download done successfully
                            [   28.385352] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.60
                            [   28.385413] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.79
                            [   28.410828] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 918 Probe success,slot:0,slave_addr:0x7a,sensor_id:0x3d
                            [   28.410999] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.79
                            [   28.415312] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.60
                            [   28.448500] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.60
                            [   28.448594] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.79
                            [   28.471853] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 731 [Applying Alternative Address] : Sensor Index   : 391385268
                            [   28.471863] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 732 [Applying Alternative Address] : Slave Address  : 0xE2
                            [   28.471870] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 733 [Applying Alternative Address] : Slave sensor id: 0x7750
                            [   28.471876] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 734 [Applying Alternative Address] : cci_client sid : 0x71
                            [   28.472189] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 770 [Applying Alternative Address] : Probe : Alternate address updated
                            [   28.472454] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 918 Probe success,slot:2,slave_addr:0xe2,sensor_id:0x7750
                            [   28.472681] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.79
                            [   28.476332] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.60
                            [   28.537077] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.60
                            [   28.537166] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.79
                            [   28.559234] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 997 CAM_ACQUIRE_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                            [   28.681627] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.60
                            [   28.681728] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.79
                            [   28.702666] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 731 [Applying Alternative Address] : Sensor Index   : 391385268
                            [   28.702672] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 732 [Applying Alternative Address] : Slave Address  : 0xE2
                            [   28.702676] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 733 [Applying Alternative Address] : Slave sensor id: 0x7750
                            [   28.702680] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 734 [Applying Alternative Address] : cci_client sid : 0x71
                            [   28.702943] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 770 [Applying Alternative Address] : Probe : Alternate address updated
                            [   28.702949] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 997 CAM_ACQUIRE_DEV Success, sensor_id:0x7750,sensor_slave_addr:0xe2
                            [   28.737746] CAM_ERR: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 710 Successfully acquire single IFE[6 -1] with [0 pix] [0 pd] [1 rdi] ports for ctx:2
                            [   28.739627] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 1137 START_DEV: CSIPHY_IDX: 2, Device_slot: 0, Datarate: 384000000, Settletime: 2800000000
                            [   28.748886] CAM_INFO: CAM-ISP: cam_vfe_bus_ver3_init_hw: 3659 Overriding clock gating at bus input
                            [   28.748893] CAM_INFO: CAM-ISP: cam_vfe_top_ver3_init_hw: 246 Disable clock gating at IFE top
                            [   28.757430] CAM_ERR: CAM-ISP: cam_ife_mgr_start_hw: 4510 ->Config HW, 00000000b07f5b5e
                            [   28.757854] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1089 CAM_START_DEV Success, sensor_id:0x7750,sensor_slave_addr:0xe2
                            [   29.029065] CAM_ERR: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 710 Successfully acquire single IFE[5 -1] with [0 pix] [0 pd] [1 rdi] ports for ctx:3
                            [   29.029714] CAM_ERR: CAM-CRM: cam_req_mgr_cb_add_req: 2742 req 1 not found in in_q
                            [   29.030119] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 1137 START_DEV: CSIPHY_IDX: 0, Device_slot: 0, Datarate: 288000000, Settletime: 2800000000
                            [   29.039005] CAM_INFO: CAM-ISP: cam_vfe_bus_ver3_init_hw: 3659 Overriding clock gating at bus input
                            [   29.039012] CAM_INFO: CAM-ISP: cam_vfe_top_ver3_init_hw: 246 Disable clock gating at IFE top
                            [   29.039031] CAM_ERR: CAM-ISP: cam_ife_mgr_start_hw: 4510 ->Config HW, 00000000b07f5b5e
                            [   29.039547] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1089 CAM_START_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                            
                            Jetson NanoJ 1 Reply Last reply Reply Quote 0
                            • Jetson NanoJ
                              Jetson Nano @Jetson Nano
                              last edited by Jetson Nano

                              @Alex-Kushleyev I have also attached the below output where ToF sensor fails sometimes during midflight, we tried reattaching the flex connectors to interposers and works fine for a while and shuts off. I tried to troubleshoot on the following note,

                              Case 1: Only Tracking enabled with ToF attached - I copied the required .bin files to /usr/lib/camera/. I restart the voxl-camera-server and the Tracking camera works fine on ground and midflight, without being frozen or feed being stalled from server.

                              Case 2: Only ToF enabled, with Tracking attached - I copied the required bin files to the destination. I restarted the voxl-camera-server, after a while data starts getting published. But the senosr doesn't always work. On a scale of comparison based on testing, the sensor stops publishing data 2 out of 5 run of tests.

                              Tried increasing the FPS of the sensor from 5 to 15, to check if would help. It helped improve mapping quality, but not well to solve the issue of losing feed.

                              Case 3: ToF + Tracking ov7251 attached and enabled - Configured the voxl-configure-cameras to configure 5. Then I restart the systemctl restart voxl-camera-server, the camera server restarts and feed is viewed in portal since cameras are publishing data. Sometimes upon restarting or running voxl-camera-server only, the cameras are detected and the server gets aborted abruptly or a message of gracefully exited appears.

                              Sometimes, the server goes of randomly on ground or mid-air and comes back again. On running dmesg -w on a terminal, I get the following output.

                              [  827.840180] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.58
                              [  827.842571] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.55
                              [  827.844131] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.60
                              [  827.844205] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.79
                              [  827.870672] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 918 Probe success,slot:0,slave_addr:0x7a,sensor_id:0x3d
                              [  827.870946] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.79
                              [  827.874552] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.60
                              [  827.874719] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.55
                              [  827.874859] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.58
                              [  827.905837] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.59
                              [  827.908159] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.55
                              [  827.910035] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.60
                              [  827.910156] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.79
                              [  827.932850] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 731 [Applying Alternative Address] : Sensor Index   : 1465270452
                              [  827.932859] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 732 [Applying Alternative Address] : Slave Address  : 0xE2
                              [  827.932866] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 733 [Applying Alternative Address] : Slave sensor id: 0x7750
                              [  827.932872] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 734 [Applying Alternative Address] : cci_client sid : 0x71
                              [  827.933224] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 770 [Applying Alternative Address] : Probe : Alternate address updated
                              [  827.933477] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 918 Probe success,slot:2,slave_addr:0xe2,sensor_id:0x7750
                              [  827.933651] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.79
                              [  827.937888] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.60
                              [  827.938082] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.55
                              [  827.938221] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.59
                              [  828.000369] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.58
                              [  828.002330] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.55
                              [  828.004189] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.60
                              [  828.004269] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.79
                              [  828.028279] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 997 CAM_ACQUIRE_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                              [  828.478535] CAM_ERR: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 710 Successfully acquire single IFE[6 -1] with [0 pix] [0 pd] [1 rdi] ports for ctx:8
                              [  828.479181] CAM_ERR: CAM-CRM: cam_req_mgr_cb_add_req: 2742 req 1 not found in in_q
                              [  828.479457] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 1137 START_DEV: CSIPHY_IDX: 0, Device_slot: 0, Datarate: 288000000, Settletime: 2800000000
                              [  828.487975] CAM_INFO: CAM-ISP: cam_vfe_bus_ver3_init_hw: 3659 Overriding clock gating at bus input
                              [  828.487979] CAM_INFO: CAM-ISP: cam_vfe_top_ver3_init_hw: 246 Disable clock gating at IFE top
                              [  828.488207] CAM_ERR: CAM-ISP: cam_ife_mgr_start_hw: 4510 ->Config HW, 00000000851a209a
                              [  828.488561] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1089 CAM_START_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                              [  841.529716] RTW: wlan0- hw port(0) mac_addr =00:c0:ca:b3:a3:e9
                              [  862.477578] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.79
                              [  862.481705] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.60
                              [  862.481785] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.55
                              [  862.481857] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.58
                              [  862.490614] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 5
                              [  862.490620] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 1
                              [  862.490623] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 2
                              [  862.490626] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 4
                              [  862.490630] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 3
                              [  862.490634] CAM_ERR: CAM-ISP: __cam_isp_ctx_release_dev_in_top_state: 3789 releasing hw
                              [  862.490658] CAM_INFO: CAM-ISP: cam_ife_mgr_release_hw: 4755 Release HW success ctx id: 8
                              [  862.490757] CAM_WARN: CAM-CRM: cam_req_mgr_close: 160 release invoked associated userspace process has died
                              [  862.490838] CAM-REQ-MGR_UTIL cam_req_mgr_util_free_hdls:99 CAM_WARN: CAM-CRM: cam_req_mgr_util_free_hdls: 99 Dev handle = f10102 session_handle = 6e0200
                              [  862.493693] ion_buffer_destroy: 1 callbacks suppressed
                              [  862.493695] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  862.493706] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  862.493715] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  862.493803] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  862.493839] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  862.494021] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  862.494253] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  862.494262] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  862.494282] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  862.494288] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  863.270637] RTW: wlan0- hw port(0) mac_addr =00:c0:ca:b3:a3:e9
                              [  870.888214] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                              [  870.888244] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
                              [  875.098507] CAM_ERR: CAM-MEM: cam_mem_mgr_create_debug_fs: 126 failed to create dentry
                              [  875.124437] CAM_INFO: CAM-HFI: cam_hfi_init: 878 Init IO1 : [0x10c00000 0xcf300000] IO2 [0xe0200000 0x1ed00000]
                              [  875.134626] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079,OEM_IMAGE_VERSION_STRING=CRM,BUILD_TIME: Oct 17 2019 05:49:19,CACHE_ENABLED at icphostinterface.c:636 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                              [  875.134631] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :ELF variant: CACHE-ENABLED:T480:API_V2:USE_CDM_1_1: , API version: 0x2000049 at icphostinterface.c:637 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                              [  875.134702] CAM_INFO: CAM-ICP: cam_icp_mgr_hw_open: 3879 FW download done successfully
                              [  875.357708] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.58
                              [  875.359880] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.55
                              [  875.362038] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.60
                              [  875.362116] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.79
                              [  875.387690] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 918 Probe success,slot:0,slave_addr:0x7a,sensor_id:0x3d
                              [  875.387887] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.79
                              [  875.390781] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.60
                              [  875.390890] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.55
                              [  875.390989] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.58
                              [  875.410081] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.59
                              [  875.412357] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.55
                              [  875.414556] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.60
                              [  875.414677] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.79
                              [  875.438443] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 731 [Applying Alternative Address] : Sensor Index   : 1465270452
                              [  875.438452] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 732 [Applying Alternative Address] : Slave Address  : 0xE2
                              [  875.438459] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 733 [Applying Alternative Address] : Slave sensor id: 0x7750
                              [  875.438464] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 734 [Applying Alternative Address] : cci_client sid : 0x71
                              [  875.438825] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 770 [Applying Alternative Address] : Probe : Alternate address updated
                              [  875.439083] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 918 Probe success,slot:2,slave_addr:0xe2,sensor_id:0x7750
                              [  875.439587] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.79
                              [  875.443944] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.60
                              [  875.444426] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.55
                              [  875.444571] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.59
                              [  875.511578] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.58
                              [  875.513201] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.55
                              [  875.515282] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.60
                              [  875.515350] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.79
                              [  875.538928] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 997 CAM_ACQUIRE_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                              [  876.005639] CAM_ERR: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 710 Successfully acquire single IFE[6 -1] with [0 pix] [0 pd] [1 rdi] ports for ctx:9
                              [  876.006492] CAM_ERR: CAM-CRM: cam_req_mgr_cb_add_req: 2742 req 1 not found in in_q
                              [  876.006720] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 1137 START_DEV: CSIPHY_IDX: 0, Device_slot: 0, Datarate: 288000000, Settletime: 2800000000
                              [  876.015994] CAM_INFO: CAM-ISP: cam_vfe_bus_ver3_init_hw: 3659 Overriding clock gating at bus input
                              [  876.015997] CAM_INFO: CAM-ISP: cam_vfe_top_ver3_init_hw: 246 Disable clock gating at IFE top
                              [  876.016401] CAM_ERR: CAM-ISP: cam_ife_mgr_start_hw: 4510 ->Config HW, 00000000851a209a
                              [  876.016793] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1089 CAM_START_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                              [  876.019097] CAM_INFO: CAM-CRM: __cam_req_mgr_find_dev_name: 209 Skip Frame: req: 2 not ready on link: 0xa10104 for pd: 2 dev: cam-sensor open_req count: 4
                              [  880.848318] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.79
                              [  880.852399] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.60
                              [  880.852461] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.55
                              [  880.852506] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.58
                              [  880.865126] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 2
                              [  880.865140] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 3
                              [  880.865150] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 4
                              [  880.865159] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 5
                              [  880.865168] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 1
                              [  880.865179] CAM_ERR: CAM-ISP: __cam_isp_ctx_release_dev_in_top_state: 3789 releasing hw
                              [  880.865285] CAM_INFO: CAM-ISP: cam_ife_mgr_release_hw: 4755 Release HW success ctx id: 9
                              [  880.865409] CAM_WARN: CAM-CRM: cam_req_mgr_close: 160 release invoked associated userspace process has died
                              [  880.865699] CAM-REQ-MGR_UTIL cam_req_mgr_util_free_hdls:99 CAM_WARN: CAM-CRM: cam_req_mgr_util_free_hdls: 99 Dev handle = df0102 session_handle = a60200
                              [  880.871851] ion_buffer_destroy: 1 callbacks suppressed
                              [  880.871858] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  880.871999] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  880.873664] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  880.873691] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  880.873866] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  880.873889] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  880.873988] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  880.874057] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  880.879726] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  880.879944] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  885.149685] RTW: wlan0- hw port(0) mac_addr =00:c0:ca:b3:a3:e9
                              [  890.823665] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x40d00
                              [  906.997270] RTW: wlan0- hw port(0) mac_addr =00:c0:ca:b3:a3:e9
                              [  920.855588] CAM_ERR: CAM-MEM: cam_mem_mgr_create_debug_fs: 126 failed to create dentry
                              [  920.884159] CAM_INFO: CAM-HFI: cam_hfi_init: 878 Init IO1 : [0x10c00000 0xcf300000] IO2 [0xe0200000 0x1ed00000]
                              [  920.894375] CAM_INFO: CAM-ICP: cam_icp_mgr_hw_open: 3879 FW download done successfully
                              [  920.894383] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079,OEM_IMAGE_VERSION_STRING=CRM,BUILD_TIME: Oct 17 2019 05:49:19,CACHE_ENABLED at icphostinterface.c:636 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                              [  920.894389] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2572 FW_DBG:CICP_FW_E : HFI  :ELF variant: CACHE-ENABLED:T480:API_V2:USE_CDM_1_1: , API version: 0x2000049 at icphostinterface.c:637 QC_IMAGE_VERSION_STRING=CICP.FW.1.0-00079 OEM_IMAGE_VERSION_STRING=CRM
                              [  921.122596] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.58
                              [  921.124189] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.55
                              [  921.126333] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.60
                              [  921.126411] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.79
                              [  921.153915] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 918 Probe success,slot:0,slave_addr:0x7a,sensor_id:0x3d
                              [  921.154186] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.79
                              [  921.159914] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.60
                              [  921.160560] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.55
                              [  921.160703] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.58
                              [  921.188632] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.59
                              [  921.191511] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.55
                              [  921.193717] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.60
                              [  921.193854] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Linked as a consumer to regulator.79
                              [  921.218411] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 731 [Applying Alternative Address] : Sensor Index   : 1465270452
                              [  921.218420] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 732 [Applying Alternative Address] : Slave Address  : 0xE2
                              [  921.218427] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 733 [Applying Alternative Address] : Slave sensor id: 0x7750
                              [  921.218432] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 734 [Applying Alternative Address] : cci_client sid : 0x71
                              [  921.218777] CAM_INFO: CAM-SENSOR: cam_sensor_set_alt_id: 770 [Applying Alternative Address] : Probe : Alternate address updated
                              [  921.219067] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 918 Probe success,slot:2,slave_addr:0xe2,sensor_id:0x7750
                              [  921.219268] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.79
                              [  921.223412] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.60
                              [  921.223568] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.55
                              [  921.223668] qcom,camera ac50000.qcom,cci:qcom,cam-sensor2: Dropping the link to regulator.59
                              [  921.284538] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.58
                              [  921.285965] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.55
                              [  921.288211] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.60
                              [  921.288298] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Linked as a consumer to regulator.79
                              [  921.311468] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 997 CAM_ACQUIRE_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                              [  921.859799] CAM_ERR: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 710 Successfully acquire single IFE[6 -1] with [0 pix] [0 pd] [1 rdi] ports for ctx:1
                              [  921.861359] CAM_ERR: CAM-CRM: cam_req_mgr_cb_add_req: 2742 req 1 not found in in_q
                              [  921.861782] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 1137 START_DEV: CSIPHY_IDX: 0, Device_slot: 0, Datarate: 288000000, Settletime: 2800000000
                              [  921.872367] CAM_INFO: CAM-ISP: cam_vfe_bus_ver3_init_hw: 3659 Overriding clock gating at bus input
                              [  921.872375] CAM_INFO: CAM-ISP: cam_vfe_top_ver3_init_hw: 246 Disable clock gating at IFE top
                              [  921.872758] CAM_ERR: CAM-ISP: cam_ife_mgr_start_hw: 4510 ->Config HW, 00000000851a209a
                              [  921.873590] CAM_INFO: CAM-SENSOR: cam_sensor_driver_cmd: 1089 CAM_START_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a
                              [  926.700983] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.79
                              [  926.704609] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.60
                              [  926.704746] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.55
                              [  926.704858] qcom,camera ac4f000.qcom,cci:qcom,cam-sensor0: Dropping the link to regulator.58
                              [  926.716260] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 2
                              [  926.716270] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 3
                              [  926.716278] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 4
                              [  926.716286] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 5
                              [  926.716293] CAM_ERR: CAM-SYNC: cam_sync_signal: 203 Error: accessing an uninitialized sync obj = 1
                              [  926.716302] CAM_ERR: CAM-ISP: __cam_isp_ctx_release_dev_in_top_state: 3789 releasing hw
                              [  926.716346] CAM_INFO: CAM-ISP: cam_ife_mgr_release_hw: 4755 Release HW success ctx id: 1
                              [  926.716439] CAM_WARN: CAM-CRM: cam_req_mgr_close: 160 release invoked associated userspace process has died
                              [  926.716536] CAM-REQ-MGR_UTIL cam_req_mgr_util_free_hdls:99 CAM_WARN: CAM-CRM: cam_req_mgr_util_free_hdls: 99 Dev handle = bb0102 session_handle = a10200
                              [  926.723046] ion_buffer_destroy: 1 callbacks suppressed
                              [  926.723050] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  926.723184] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  926.723315] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  926.723534] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  926.728238] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  926.728261] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  926.728311] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  926.728332] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  926.728425] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  926.728641] ION client likely missing a call to dma_buf_kunmap or dma_buf_vunmap
                              [  928.697163] RTW: wlan0- hw port(0) mac_addr =00:c0:ca:b3:a3:e9
                              

                              Sometimes, when we tweak the cables slightly or power the system after a while, the camera server including the ToF and tracking work fine, even though CAM ERR message appear on terminal line system call messages.

                              Through these I'm not able to actually conclude if it's a hardware issue or software issue in the voxl-camera-server while running multiple camera sensors simultanelously through the server.

                              Would be great if you could help out.

                              Jetson NanoJ 1 Reply Last reply Reply Quote 0
                              • Jetson NanoJ
                                Jetson Nano @Jetson Nano
                                last edited by

                                @Alex-Kushleyev Is there something we could do to tweak the way the data is being published through the voxl-camera-server pipeline?

                                Jetson NanoJ 1 Reply Last reply Reply Quote 0
                                • Jetson NanoJ
                                  Jetson Nano @Jetson Nano
                                  last edited by

                                  @Alex-Kushleyev could you look into?

                                  Alex KushleyevA 1 Reply Last reply Reply Quote 0
                                  • Alex KushleyevA
                                    Alex Kushleyev ModalAI Team @Jetson Nano
                                    last edited by

                                    Hi @Jetson-Nano ,

                                    Thank you for sending the logs. Unfortunately, I do not see anything specific that would indicate an error in the last few logs.

                                    Do you have more than one VOXL2 you could use for testing? also, do you have another TOF sensor you could use? It seems the issue is tied to the TOF sensor but it is not clear whether it is on the VOXL2 side (maybe connector issue) or the TOF sensor side.

                                    What you are describing is not normal behavior - the camera server from any recent official release should be pretty stable when used with these standard configurations (tracking camera, TOF, etc).

                                    Since the tracking camera by itself seems to be working fine, please focus on testing TOF only (you can disconnect the tracking camera) and make sure you can get that to work reliably, testing on a different VOXL2 or a different TOF module if possible (to rule out one of the two being an issue)

                                    Also, which SDK are you using?

                                    Alex

                                    Jetson NanoJ 2 Replies Last reply Reply Quote 0
                                    • Jetson NanoJ
                                      Jetson Nano @Alex Kushleyev
                                      last edited by

                                      @Alex-Kushleyev Thanks for reverting. Yes, I have another VOXL2 board. I'll test and share the results. I'm currently testing with SDKs 1.1.1 and 1.2 releases.

                                      1 Reply Last reply Reply Quote 0
                                      • Jetson NanoJ
                                        Jetson Nano @Alex Kushleyev
                                        last edited by Jetson Nano

                                        @Alex-Kushleyev I tested the ToF with another VOXL2 board I have. Modified the camera config as only for ToF. On ground tests, the camera server seems to work fine. On terminal system logger publishes similar type of CAM_INFO, CAM_ERR dmesg -w messages as earlier.

                                        We used the same interposer as used in earlier VOXL2 compute.

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post
                                        Powered by NodeBB | Contributors