sysimg 4.0.0 voxl-camera-server crashing tracking



  • With a fresh install on one of our platforms of System Image 4.0.0, the camera server crashes and QVIO becomes unresponsive. The system is configured with Stereo, Tracking, and TOF.

    voxl-camera-server:

    Jan 01 00:00:07 apq8096 systemd[1]: Started voxl-camera-server.
    Jan 01 00:00:10 apq8096 bash[2543]: ------ voxl-camera-server: Camera server is now running
    Nov 16 21:25:39 apq8096 bash[2543]: ERROR:   Recieved Fatal error from camera: tof
    Nov 16 21:25:39 apq8096 bash[2543]: ERROR:   Sending request 397, ErrorCode: -ENODEV
    Nov 16 21:25:39 apq8096 bash[2543]: WARNING: Thread: tracking result thread recieved ESTOP
    Nov 16 21:25:39 apq8096 bash[2543]: WARNING: Thread: tof request thread recieved ESTOP
    Nov 16 21:25:39 apq8096 bash[2543]: WARNING: Thread: stereo_child result thread recieved ESTOP
    Nov 16 21:25:39 apq8096 bash[2543]: WARNING: Thread: tof result thread recieved ESTOP
    Nov 16 21:25:39 apq8096 bash[2543]: WARNING: Thread: stereo result thread recieved ESTOP
    Nov 16 21:25:39 apq8096 bash[2543]: WARNING: Thread: stereo request thread recieved ESTOP
    Nov 16 21:25:39 apq8096 bash[2543]: WARNING: Thread: tracking request thread recieved ESTOP
    Nov 16 21:25:39 apq8096 bash[2543]: WARNING: Thread: stereo_child request thread recieved ESTOP
    Nov 16 21:25:39 apq8096 bash[2543]: ------ voxl-camera-server INFO: Camera server is now stopping
    Nov 16 21:25:39 apq8096 bash[2543]: WARNING: Deleting buffers: 3 of 16 still in use
    Nov 16 21:25:40 apq8096 bash[2543]: WARNING: Deleting buffers: 5 of 16 still in use
    Nov 16 21:25:47 apq8096 bash[2543]: WARNING: Deleting buffers: 1179656 of 1179668 still in use
    Nov 16 21:25:47 apq8096 bash[2543]: Segmentation fault:
    Nov 16 21:25:47 apq8096 bash[2543]: Fault thread: voxl-camera-ser(tid: 2543)
    Nov 16 21:25:47 apq8096 bash[2543]: Fault address: 0xa
    Nov 16 21:25:47 apq8096 bash[2543]: Address not mapped.
    Nov 16 21:25:49 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Main process exited, code=killed, status=11/SEGV[[0m
    Nov 16 21:25:49 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Unit entered failed state.[[0m
    Nov 16 21:25:49 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Failed with result 'signal'.[[0m
    

    voxl-qvio-server:

    -- Logs begin at Thu 1970-01-01 00:00:04 UTC, end at Wed 2022-11-16 21:29:40 UTC. --
    Jan 01 00:00:07 apq8096 systemd[1]: Started voxl-qvio-server.
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: setting scheduler
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: INFO: set FIFO priority successfully!
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: loading config file
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: =================================================================
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: imu_name:                         imu0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: cam_name:                         tracking
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: lens_cal_path:                    /data/modalai/opencv_tracking_intrinsics.yml
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: cam_extrinsics_name:              tracking
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: use_second_cam_if_stereo:         0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: odr_hz:                           30.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: use_camera_height_bootstrap:      1
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: camera_height_off_ground_m:        0.085
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: enable_init_while_moving:         0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: cam_imu_timeshift_s:              -0.002
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: cam_imu_timeshift_s_uncertainty:   0.001
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: T_cam_wrt_imu_uncertainty:         0.001  0.001  0.001
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: R_cam_to_imu_uncertainty:          0.005  0.005  0.005
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: accl_fsr_ms2:                     156.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: gyro_fsr_rad:                     34.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: accl_noise_std_dev:                0.316
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: gyro_noise_std_dev:                0.010
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: cam_noise_std_dev:                100.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: min_std_pixel_noise:               0.500
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: fail_high_pixel_noise_points:      1.665
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: limited_imu_bw_trigger:           25.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: gps_imu_time_alignment_s:          0.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: T_gps_wrt_imu:                     0.000  0.000  0.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: enable_mapping:                   1
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: enable_gps_vel:                   0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: show_extra_points_on_overlay;     0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: auto-reset params:
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: en_auto_reset:                    1
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: auto_reset_max_velocity:          10.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: auto_reset_max_v_cov_instant:      0.100
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: auto_reset_max_v_cov:              0.010
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: auto_reset_max_v_cov_timeout_s:    0.500
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: auto_reset_min_features:          3
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: auto_reset_min_feature_timeout_s:  1.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: =================================================================
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: loading extrinsics config file
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: #0:
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: parent:                imu0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: child:                 tracking
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: T_child_wrt_parent:    0.109  -0.012   0.001
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: RPY_parent_to_child: 180.0    45.0    90.0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: R_child_to_parent:     0.000  -0.707   0.707
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: -1.000  -0.000  -0.000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: 0.000  -0.707  -0.707
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: tbc:   0.109 -0.012  0.001
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: ombc:  2.504 -2.504  1.037 (axis angle)
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: ombc: 180.000 45.000 90.000 (RPY deg)
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: loading /data/modalai/opencv_tracking_intrinsics.yml
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: using fisheye camera intrinsics:
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: distortion: 0.008810 -0.003841 0.000000 0.000000
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: focal lengths: 269.295927 268.516850
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: principle points: 340.767010 221.351666
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: WARNING failed to init in time, trying again
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: LIBMV: AR INFO:
                                                    FIT:FeatureName=>SensorFusion, TSBDrift=>0.00059 (0.004)
                                                    FIT:FeatureName=>SensorFusion, TSBDrift0=>-0.000 (0.003)
                                                    FIT:FeatureName=>SensorFusion, TSBDrift1=>0.000 (0.003)
                                                    FIT:FeatureName=>SensorFusion, TSBDrift2=>-0.000 (0.002)
                                                    FIT:FeatureName=>SensorFusion, RSBDrift=>0.00062
                                                    FIT:FeatureName=>SensorFusion, AccelBias0=>-0.00278
                                                    FIT:FeatureName=>SensorFusion, AccelBias1=>-0.00121
                                                    FIT:FeatureName=>SensorFusion, AccelBias2=>0.12533
                                                    FIT:FeatureName=>SensorFusion, GyroBias0=>-0.00470
                                                    FIT:FeatureName=>SensorFusion, GyroBias1=>-0.00426
                                                    FIT:FeatureName=>SensorFusion, GyroBias2=>-0.00017
                                                    FIT:FeatureName=>SensorFusion, GravityBias0=>0.30783
                                                    FIT:FeatureName=>SensorFusion, GravityBias1=>0.16675
                                                    FIT:FeatureName=>SensorFusion, GravityBias2=>-9.78417
                                                    FIT:FeatureName=>SensorFusion, FrameCount=>7
                                                    FIT:FeatureName=>SensorFusion, ImuCount=>304
                                                    FIT:FeatureName=>SensorFusion, AvgTimeFrameUpdate=>5.84ms
                                                    FIT:FeatureName=>SensorFusion, NumResets=>0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: LIBMV: AR INFO:
                                                    TrckLnDst:6 4 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: LIBMV: AR INFO:
                                                    Distribution of num of consec keyframes w/ GN failing all features:0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: LIBMV: AR INFO:
                                                    Distribution of num of consec frames w/ increasing depth uncertainty:2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: waiting for imu
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: waiting for cam
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: connected to imu server
    Jan 01 00:00:10 apq8096 voxl-qvio-server[2539]: connected to camera server
    Nov 16 21:25:48 apq8096 voxl-qvio-server[2539]: LIBMV: AR INFO:
                                                    FIT:FeatureName=>SensorFusion, TSBDrift=>0.76085 (0.011)
                                                    FIT:FeatureName=>SensorFusion, TSBDrift0=>0.596 (0.008)
                                                    FIT:FeatureName=>SensorFusion, TSBDrift1=>-0.458 (0.006)
                                                    FIT:FeatureName=>SensorFusion, TSBDrift2=>-0.114 (0.005)
                                                    FIT:FeatureName=>SensorFusion, RSBDrift=>0.30760
                                                    FIT:FeatureName=>SensorFusion, AccelBias0=>0.12854
                                                    FIT:FeatureName=>SensorFusion, AccelBias1=>0.19535
                                                    FIT:FeatureName=>SensorFusion, AccelBias2=>-0.10591
                                                    FIT:FeatureName=>SensorFusion, GyroBias0=>-0.00679
                                                    FIT:FeatureName=>SensorFusion, GyroBias1=>-0.00544
                                                    FIT:FeatureName=>SensorFusion, GyroBias2=>-0.00180
                                                    FIT:FeatureName=>SensorFusion, GravityBias0=>0.49261
                                                    FIT:FeatureName=>SensorFusion, GravityBias1=>0.21229
                                                    FIT:FeatureName=>SensorFusion, GravityBias2=>-9.78607
                                                    FIT:FeatureName=>SensorFusion, FrameCount=>2375
                                                    FIT:FeatureName=>SensorFusion, ImuCount=>180880
                                                    FIT:FeatureName=>SensorFusion, AvgTimeFrameUpdate=>178.39ms
                                                    FIT:FeatureName=>SensorFusion, NumResets=>1
    Nov 16 21:25:48 apq8096 voxl-qvio-server[2539]: LIBMV: AR INFO:
                                                    TrckLnDst:182 46 36 69 45 37 32 43 20 22 21 16 16 11 8 11 4 8 7 2 4 2 6 4 2 2 5 3 2 2 3 0 2 3 4 1 1 1 4 1 2 1 0 1 5 1 5 1 0 3 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 0 1 0 1 0 1 1 2 0 1 2 1 1 0 0 1 0 0 0 2 0 0 0 2 0 1 0 1 0 0 0 1 0 0 1 0 0 1 1 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 108
    Nov 16 21:25:48 apq8096 voxl-qvio-server[2539]: LIBMV: AR INFO:
                                                    Distribution of num of consec keyframes w/ GN failing all features:42 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    Nov 16 21:25:48 apq8096 voxl-qvio-server[2539]: LIBMV: AR INFO:
                                                    Distribution of num of consec frames w/ increasing depth uncertainty:2354 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    Nov 16 21:25:48 apq8096 voxl-qvio-server[2539]: WARNING: disconnected from camera server, resetting VIO
    Nov 16 21:25:48 apq8096 voxl-qvio-server[2539]: client "voxl-portal0" connected to overlay
    

    Here are our versions etc.

    Service Name         |  Version  |  Enabled  |   Running   |  CPU Usage
    ---------------------------------------------------------------------------
     docker-autorun       |    1.2.4  | Disabled  | Not Running |
     docker-daemon        |    1.2.4  | Disabled  | Not Running |
     modallink-relink     |   0.16.1  | Disabled  | Not Running |
     voxl-camera-server   |    1.3.5  |  Enabled  |   Running   |    12.7
     voxl-cpu-monitor     |    0.3.0  |  Enabled  |   Running   |     0.0
     voxl-dfs-server      |    0.3.1  | Disabled  | Not Running |
     voxl-imu-server      |    1.0.3  |  Enabled  |   Running   |     0.0
     voxl-mavlink-server  |    0.2.0  |  Enabled  |   Running   |     1.8
     voxl-modem           |   0.16.1  | Disabled  | Not Running |
     voxl-portal          |    0.4.2  |  Enabled  |   Running   |     0.0
     voxl-qvio-server     |    0.8.2  |  Enabled  |   Running   |     9.1
     voxl-streamer        |    0.4.1  | Disabled  | Not Running |
     voxl-tag-detector    |    0.0.4  | Disabled  | Not Running |
     voxl-tflite-server   |    0.3.1  | Disabled  | Not Running |
     voxl-time-sync       |    1.2.2  | Disabled  | Not Running |
     voxl-vision-px4      |    1.4.0  |  Enabled  |   Running   |     0.0
     voxl-wait-for-fs     |    1.2.2  |  Enabled  |  Completed  |
    

    This happens with or without the portal. Other units seem OK and this unit was OK in a different environment. Trying to isolate the issue.


  • Dev Team

    camera server errors are very likely hardware related. usually a sensor is not fully plugged in



  • Resurrecting this topic. We have two drones with identical hardware builds.
    Only camera's plugged in are tracking cameras to J4.
    Both are running the latest system image.

    Drone 1's camera server never crashes while flying Indoors/outdoors.
    Drone 2's camera server sometimes crashes and seems to be triggered by exposure changes when outside in bright environments. It is running a slightly newer version of voxl-camera-server.

    The failing drone has this in the logs from logcat | grep "camera":

    voxl:~$ logcat | grep camera
    
    
    01-26 20:06:14.004  5732  5732 V HAL     : loaded HAL id=camera path=/usr/lib/hw/camera.msm8996.so hmi=(nil) handle=0x64450
    01-26 20:06:14.005  5732  5732 I QCamera : <MCI><INFO> get_num_of_cameras: 2767: dev_info[id=0,name='video2']
    01-26 20:06:14.005  5732  5732 I QCamera : <MCI><INFO> sort_camera_info: 2611: Camera id: 0 facing: 1, type: 1 is_yuv: 0
    01-26 20:06:14.005  5732  5732 I QCamera : <MCI><INFO> sort_camera_info: 2617: Number of cameras 1 sorted 1
    01-26 20:06:14.005  5732  5732 I QCamera : <MCI><INFO> get_num_of_cameras: 2785: num_cameras=1
    01-26 20:06:14.006  5732  5732 I QCamera : <HAL><INFO> QCamera2Factory: 142: 1 camera devices detected!
    01-26 20:06:14.107  5732  5732 E QCamera : <HAL><ERROR> initStaticMetadata: 9846: static int qcamera::QCamera3HardwareInterface::initStaticMetadata(uint32_t)@9846: from gCamCapability QCAMERA3_EXP_TIME_RANGE= (19425, 1020461640)
    01-26 20:06:14.107  5732  5732 E QCamera : <HAL><ERROR> initStaticMetadata: 9877: static int qcamera::QCamera3HardwareInterface::initStaticMetadata(uint32_t)@9877: cameraId 0 raw_dim[0].w 640 raw_dim[0].h 480
    01-26 20:06:14.107  5732  5732 I QCamera : <HAL><INFO> getCamInfo: 10272: camera 0 resource cost is 100
    01-26 20:06:14.107  5732  5732 I QCamera : <HAL><INFO> cameraDeviceOpen: 414: Open camera id 0 API version 768
    01-26 20:06:14.117  5732  5732 I QCamera : <HAL><INFO> openCamera: 851: [KPI Perf]: E PROFILE_OPEN_CAMERA camera id 0
    01-26 20:06:14.197  5732  5732 I QCamera : <HAL><INFO> openCamera: 861: [KPI Perf]: X PROFILE_OPEN_CAMERA camera id 0, rc: 0
    01-26 20:06:14.197  5732  5732 E QCamera : <HAL><ERROR> initParameters: 8374: int qcamera::QCamera3HardwareInterface::initParameters()@8374 HAL params are initialized
    
    
    01-26 20:07:55.283  5732  5789 E QCamera : <MCI><ERROR> mm_camera_event_notify: 192: Camera Event DAEMON DIED received
    01-26 20:07:55.284  5732  5788 E QCamera : <HAL><ERROR> camEvtHandle: 803: Fatal, camera daemon died
    
    

    journalctl on the drone with crashed voxl-camera-server shows:

    Jan 01 00:00:52 apq8096 systemd[1]: Started voxl-camera-server.
    Jan 26 20:01:17 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
    Jan 26 20:01:23 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
    Jan 26 20:01:24 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
    Jan 26 20:01:35 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
    Jan 26 20:01:37 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
    Jan 26 20:01:42 apq8096 bash[2966]: ERROR:   Recieved Fatal error from camera: tracking
    Jan 26 20:01:42 apq8096 bash[2966]: ERROR:   Sending request 7188, ErrorCode: -ENODEV
    Jan 26 20:01:42 apq8096 bash[2966]: WARNING: Thread: tracking request thread recieved ESTOP
    Jan 26 20:01:42 apq8096 bash[2966]: WARNING: Thread: tracking result thread recieved ESTOP
    Jan 26 20:01:42 apq8096 bash[2966]: ------ voxl-camera-server INFO: Camera server is now stopping
    Jan 26 20:01:42 apq8096 bash[2966]: WARNING: Deleting buffers: 24 of 32 still in use
    Jan 26 20:01:42 apq8096 bash[2966]: Segmentation fault:
    Jan 26 20:01:42 apq8096 bash[2966]: Fault thread: voxl-camera-ser(tid: 2966)
    Jan 26 20:01:42 apq8096 bash[2966]: Fault address: 0x8
    Jan 26 20:01:42 apq8096 bash[2966]: Address not mapped.
    Jan 26 20:01:43 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Main process exited, code=killed, status=11/SEGV[[0m
    Jan 26 20:01:43 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Unit entered failed state.[[0m
    Jan 26 20:01:43 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Failed with result 'signal'.[[0m
    

    They have not been crashed... and the failure happens regularly when in bright environments. I matched up software versions as well and the crash still happens, so it has to be some kind of hardware issue and the below can be disregarded.

    Hardware problems again with expansion or something weird in this specific camera/cable?

    Drone 1 package versions (a bit older/stable repo):

    libvoxl-cci-direct - 0.1.5
    libvoxl-cutils - 0.1.1
    voxl-camera-calibration - 0.2.3
    voxl-camera-server - 1.3.5
    voxl-cpu-monitor - 0.3.0
    voxl-docker-support - 1.2.4
    voxl-gphoto2-server - 0.0.10
    voxl-jpeg-turbo - 2.1.3-4
    voxl-libgphoto2 - 0.0.4
    voxl-libuvc - 1.0.7
    voxl-logger - 0.3.4
    voxl-mavlink - 0.1.0
    voxl-mavlink-server - 0.2.0
    voxl-modem - 0.16.1
    voxl-mongoose - 7.7.0-1
    voxl-mpa-to-ros - 0.3.6
    voxl-mpa-tools - 0.7.6
    voxl-opencv - 4.5.5-1
    voxl-portal - 0.4.2
    voxl-qvio-server - 0.9.0
    voxl-streamer - 0.4.1
    voxl-suite - 0.9.4
    voxl-system-image - 4.0-r0
    voxl-tag-detector - 0.0.4
    voxl-tflite-server - 0.3.1
    voxl-utils - 1.2.2
    voxl-uvc-server - 0.1.3
    voxl-vision-px4 - 1.4.0
    voxl-vpn - 0.0.6
    

    Drone 2 package versions (a bit newer, dev repos):

    libvoxl-cci-direct - 0.1.5
    libvoxl-cutils - 0.1.1
    voxl-camera-calibration - 0.3.0
    voxl-camera-server - 1.3.7
    voxl-cpu-monitor - 0.4.1
    voxl-docker-support - 1.2.5
    voxl-gphoto2-server - 0.0.10
    voxl-jpeg-turbo - 2.1.3-4
    voxl-lepton-server - 1.0.1
    voxl-libgphoto2 - 0.0.4
    voxl-libuvc - 1.0.7
    voxl-logger - 0.3.4
    voxl-mavlink - 0.1.0
    voxl-mavlink-server - 0.2.0
    voxl-modem - 0.16.2
    voxl-mongoose - 7.7.0-1
    voxl-mpa-to-ros - 0.3.6
    voxl-mpa-tools - 0.8.4
    voxl-opencv - 4.5.5-1
    voxl-portal - 0.4.3
    voxl-qvio-server - 0.9.0
    voxl-streamer - 0.4.3
    voxl-suite - 1.0.0
    voxl-system-image - 4.0-r0
    voxl-tag-detector - 0.0.4
    voxl-tflite-server - 0.3.1
    voxl-utils - 1.2.3
    voxl-uvc-server - 0.1.3
    voxl-vision-px4 - 1.4.1
    voxl-vpn - 0.0.6
    

  • Dev Team

    These are usually hardware issues. Some possibilities

    1. crimped flex cable
    2. exposed copper on the flex cable
    3. loose or damaged connector

Log in to reply