Navigation

    ModalAI Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    1. Home
    2. Steve Turner
    • Profile
    • Following
    • Followers
    • Topics
    • Posts
    • Best
    • Groups

    Steve Turner

    @Steve Turner

    sturner@darkhive.ai

    12
    Reputation
    86
    Posts
    50
    Profile views
    1
    Followers
    0
    Following
    Joined Last Online
    Location CA

    Steve Turner Follow

    Best posts made by Steve Turner

    • RE: Tilting the VOXL CAM forward

      @Sem-Andeweg

      I don't work for ModalAI but can try to help.

      You need to adjust the IMU0, and IMU1 to body RPY at minimum, which stands for Roll, Pitch, Yaw.

      If you look at the Seeker example at https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/blob/master/misc_files/usr/share/modalai/extrinsic_configs/seeker_v1_voxlcam.conf#L64

      you'll see the VOXL is rotated 90 degrees "up" and yawed 90 degrees right. From there the tracking camera is then rotated "down" 45 degrees to account for the VOXL CAM being mounted on the front of the Seeker and its orientation relative to the VOXL inside the camera housing.

      Hope this helps a bit, definitely watch the video at https://docs.modalai.com/configure-extrinsics/

      posted in VOXL-CAM
      Steve Turner
      Steve Turner
    • RE: Vision failure everytime during takeoff

      Is this a unit direct from ModalAI? If it is, it should be tuned or you could load default parameters from https://gitlab.com/voxl-public/flight-core-px4/px4-parameters/-/tree/master/platforms for your version of PX4 and platform.

      Then from there you load a helper through QGroundControl parameters page: https://gitlab.com/voxl-public/flight-core-px4/px4-parameters/-/tree/master/helpers

      That should get you close to a decent tune.

      If this is a custom drone, you need to configure your extrinsics, calibrate your camera, and validate QVIO first before attempting to fly which it looks like you are attempting to do. The VOXL portal qVIO overlay will show you points selected as well as distance traveled which you can validate against manually moving the drone around prior to any attempts to fly. The camera needs to have areas of contrasting features in view -- so reflective floors or some repetitive patterns can be a challenge for qVIO.

      https://docs.modalai.com/configure-extrinsics/

      https://docs.modalai.com/calibrate-cameras/

      posted in VOXL 2
      Steve Turner
      Steve Turner
    • RE: Opkg install voxl-streamer

      Late to the party on this -- but I have been able to build ipk's off of an M1 recently.

      I followed: https://docs.modalai.com/install-voxl-docker/

      And then also downloaded the image from the developer portal as laid out: https://docs.modalai.com/voxl-cross/

      To load:
      docker load --input voxl-cross_V1.8.tar

      To run:
      voxl-docker -i voxl-cross:V1.8

      Then the docs in all of the Modal repo's work well.

      I do have buildkit etc enabled as laid out here: https://medium.com/nerd-for-tech/developing-on-apple-m1-silicon-with-virtual-environments-4f5f0765fd2f but you shouldn't need any of that. Let me know if that helps you. Worked just fine for me -- no special rosetta2 terminal or Docker version.

      I'll see if I can contribute something to the docs when I have a moment.

      posted in Ask your questions right here!
      Steve Turner
      Steve Turner
    • RE: How to integrate my ROS/algo in VOXL2 ?

      https://docs.modalai.com/setup-ros-on-voxl-2/

      May help you with setting up ROS on VOXL2.

      https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-to-ros may also help you bridge the Modal Pipe Architecture into ROS.

      posted in Feature Requests
      Steve Turner
      Steve Turner
    • RE: Voxl2 + flight core in custom frame

      Extrinsics and QVIO server configuration will determine camera orientation and which IMU to use respectively.

      https://docs.modalai.com/configure-extrinsics/

      https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/blob/master/server/config_file.h#L51

      You will need to measure from the IMU (Parent) to child (camera) for translation in meters and also the roll, pitch, and yaw in degrees. I recommend watching the video on the configure extrinsics page.

      posted in VOXL 2
      Steve Turner
      Steve Turner
    • RE: Get camera Frame from libmodal pipe client

      https://gitlab.com/voxl-public/voxl-sdk/services/voxl-portal/-/blob/master/src/video_manager.cpp#L77

      Is an example of allocating a buffer and copying the frame into it. There are some other good examples in that file as well for processing frames and converting them to various formats within the callback. Hope that helps!

      posted in Modal Pipe Architecture (MPA)
      Steve Turner
      Steve Turner
    • RE: VOXL2 Downward pointing camera

      @rohitpillai

      VOXL2 supports many camera configurations including front and rear stereo, a hires camera, and a tracking camera.

      The drone development kits for VOXL2 such as the Sentinel (https://www.modalai.com/products/sentinel-development-drone?variant=40446488805427) and VOXL2 Flight Deck include the tracking camera at 45 degrees. You could adapt this to point straight down.

      You could also put any camera on a gimbal or statically using the MIPI connectors for these "standard" cameras ModalAI provides (https://docs.modalai.com/voxl2-camera-configs/) or even interface over USB. https://www.modalai.com/pages/voxl-cameras

      posted in Ask your questions right here!
      Steve Turner
      Steve Turner
    • RE: TOF camera

      https://forum.modalai.com/topic/1328/camera-servers-docker-build-royale-331-spectre-4-7 may have some answers for you on voxl-camera-server which is something ModalAI is working on WRT the build.

      posted in Ask your questions right here!
      Steve Turner
      Steve Turner
    • RE: QVIO Output Interpretation

      In case anyone else comes on this thread --

      https://gitlab.com/voxl-public/voxl-sdk/core-libs/libmodal-pipe/-/blob/master/library/include/modal_pipe_interfaces.h#L848 defines the VIO messages within MPA.

      float T_imu_wrt_vio[3]; contains the translation in x, y, z and is what is printed in the overlay within VOXL Portal.

      • Positive X = Forward
      • Positive Y = Left
      • Positive Z = Increase in Altitude

      If you then look in the Odometry messages used by PX4 or run voxl-inspect-pose these coordinates will be in FRD/NED frame of reference which is:
      Positive X = Forward
      Positive Y = Right
      Positive Z = Decrease Altitude
      (I think I've got this right -- hopefully someone from Modal can confirm)

      https://gitlab.com/voxl-public/voxl-sdk/services/voxl-vision-px4/-/blob/master/src/vio_manager.c#L350-393 walks through the publication of the odometry MAVLINK message to PX4 and rotation into appropriate frame for processing.

      ========
      TLDR; Most of my issues for this platform were solved by disabling the QVIO server, keeping voxl-vision-px4 running, and dialing in PID for PX4 without any external inputs from vision or GPS. Once tuned, you can focus on calibrating cameras and etc/modalai/extrinsics.conf to ensure consistency in VIO inputs.

      Shoutout to @wilkinsaf for tips and tricks on tuning.

      posted in Ask your questions right here!
      Steve Turner
      Steve Turner
    • RE: Is there any the off-the-shelf AI perception models available for VOXL2?

      I don't work for Modal but,
      Questions 1,2,3:

      I recommend checking out https://docs.modalai.com/voxl-tflite-server/#voxl-2 and following along with deploying the TF Lite server on your VOXL2.

      The source is available here https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server and includes many built-in models.

      Questions 4:
      Yes for both cameras, but likely will need some adaptation particularly for thermal.

      posted in Ask your questions right here!
      Steve Turner
      Steve Turner

    Latest posts made by Steve Turner

    • RE: voxl-mavlink-server external FC Bus Error

      Zach,

      Thanks for the quick response!
      I can run both on the dev branch, or roll back to staging.

      I thought that voxl-vision-hub would talk to the voxl-mavlink-server which handles all communication with PX4.

      From there voxl-mavlink-server forwards between visionhub, px4, and QGC once it connections.

      I'll try running the new config tool again to see if that cleans anything up.

      Anything else to look for for the bus error?

      posted in Ask your questions right here!
      Steve Turner
      Steve Turner
    • voxl-mavlink-server external FC Bus Error

      We were running a VOXL-1 based system off of staging.

      Aftern an opkg update and opkg upgrade it looks like the MAVLINK server is broken. We're investigating now, just wanted to get this out there.

      voxl:~$ voxl-mavlink-server
      loading our own config file
      =================================================================
      Parameters as loaded from config file:
      primary_static_gcs_ip:        192.168.8.10
      secondary_static_gcs_ip:      NULL
      autopilot_uart_backup_bus:    12
      autopilot_uart_bus:           5
      autopilot_uart_baudrate:      921600
      udp_mtu:                      0
      en_external_ap_timesync:      1
      =================================================================
      Successfully opened primary mavparser on bus 5
      Starting primary thread for bus 5
      Successfully opened backup mavparser on bus 12
      Starting backup thread for bus 12
      got first packet on bus 5, stopping the other bus: 12
      Adding primary GCS IP address from conf file to list: 192.168.8.10
      Added new UDP connection to 192.168.8.10
      Not connecting to secondary gcs ip: NULL
      Added new UDP connection to 10.147.20.136
      CONNECTED to GCS at 10.147.20.136
      Init complete, entering main loop
      Detected Autopilot Mavlink SYSID 1
      thread for bus 12 trying to close
      thread for bus 12 Successfully closed
      Bus error
      

      If I rollback the mavlink server to previous versions it seems happy but I can't connect to voxl-vision-hub through the mavlink server from something like QGC.

      posted in Ask your questions right here!
      Steve Turner
      Steve Turner
    • RE: Voxl2 + flight core in custom frame

      Will be the VOXL2 IMU's.

      https://docs.modalai.com/configure-extrinsics/#voxl2 recommends using imu_apps for configuration.

      posted in VOXL 2
      Steve Turner
      Steve Turner
    • RE: Reconfiguring VIO

      Use cam name = "stereo" and there should be an option for using either the left or right stereo camera.

      As Chad pointed out you will need to also configure extrinsics if they aren't already for the stereo camera.
      https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/blob/master/server/config_file.h#L155

      I have a PR to fix an issue with using a stereo camera here with the QVIO Server: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/merge_requests/10 that you will need as well I believe.

      posted in Ask your questions right here!
      Steve Turner
      Steve Turner
    • RE: Voxl2 + flight core in custom frame

      Extrinsics and QVIO server configuration will determine camera orientation and which IMU to use respectively.

      https://docs.modalai.com/configure-extrinsics/

      https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/blob/master/server/config_file.h#L51

      You will need to measure from the IMU (Parent) to child (camera) for translation in meters and also the roll, pitch, and yaw in degrees. I recommend watching the video on the configure extrinsics page.

      posted in VOXL 2
      Steve Turner
      Steve Turner
    • RE: Fusing optical flow together with VIO

      Yes, you can. You may need to evaluate the performance of both at the same time and make tweaks.

      posted in Ask your questions right here!
      Steve Turner
      Steve Turner
    • RE: VIO flight capabilities

      You can change the QVIO server to use a stereo camera if you want in the front, but these are much narrower FOV.

      https://docs.modalai.com/configure-extrinsics/ would need to be adjusted to accommodate any camera location changes as well. This is not something supported out of the box with the current VOXL software and would require some work on your part to make it fit your platform and camera configuration.

      posted in GPS-denied Navigation (VIO)
      Steve Turner
      Steve Turner
    • RE: sysimg 4.0.0 voxl-camera-server crashing tracking

      Resurrecting this topic. We have two drones with identical hardware builds.
      Only camera's plugged in are tracking cameras to J4.
      Both are running the latest system image.

      Drone 1's camera server never crashes while flying Indoors/outdoors.
      Drone 2's camera server sometimes crashes and seems to be triggered by exposure changes when outside in bright environments. It is running a slightly newer version of voxl-camera-server.

      The failing drone has this in the logs from logcat | grep "camera":

      voxl:~$ logcat | grep camera
      
      
      01-26 20:06:14.004  5732  5732 V HAL     : loaded HAL id=camera path=/usr/lib/hw/camera.msm8996.so hmi=(nil) handle=0x64450
      01-26 20:06:14.005  5732  5732 I QCamera : <MCI><INFO> get_num_of_cameras: 2767: dev_info[id=0,name='video2']
      01-26 20:06:14.005  5732  5732 I QCamera : <MCI><INFO> sort_camera_info: 2611: Camera id: 0 facing: 1, type: 1 is_yuv: 0
      01-26 20:06:14.005  5732  5732 I QCamera : <MCI><INFO> sort_camera_info: 2617: Number of cameras 1 sorted 1
      01-26 20:06:14.005  5732  5732 I QCamera : <MCI><INFO> get_num_of_cameras: 2785: num_cameras=1
      01-26 20:06:14.006  5732  5732 I QCamera : <HAL><INFO> QCamera2Factory: 142: 1 camera devices detected!
      01-26 20:06:14.107  5732  5732 E QCamera : <HAL><ERROR> initStaticMetadata: 9846: static int qcamera::QCamera3HardwareInterface::initStaticMetadata(uint32_t)@9846: from gCamCapability QCAMERA3_EXP_TIME_RANGE= (19425, 1020461640)
      01-26 20:06:14.107  5732  5732 E QCamera : <HAL><ERROR> initStaticMetadata: 9877: static int qcamera::QCamera3HardwareInterface::initStaticMetadata(uint32_t)@9877: cameraId 0 raw_dim[0].w 640 raw_dim[0].h 480
      01-26 20:06:14.107  5732  5732 I QCamera : <HAL><INFO> getCamInfo: 10272: camera 0 resource cost is 100
      01-26 20:06:14.107  5732  5732 I QCamera : <HAL><INFO> cameraDeviceOpen: 414: Open camera id 0 API version 768
      01-26 20:06:14.117  5732  5732 I QCamera : <HAL><INFO> openCamera: 851: [KPI Perf]: E PROFILE_OPEN_CAMERA camera id 0
      01-26 20:06:14.197  5732  5732 I QCamera : <HAL><INFO> openCamera: 861: [KPI Perf]: X PROFILE_OPEN_CAMERA camera id 0, rc: 0
      01-26 20:06:14.197  5732  5732 E QCamera : <HAL><ERROR> initParameters: 8374: int qcamera::QCamera3HardwareInterface::initParameters()@8374 HAL params are initialized
      
      
      01-26 20:07:55.283  5732  5789 E QCamera : <MCI><ERROR> mm_camera_event_notify: 192: Camera Event DAEMON DIED received
      01-26 20:07:55.284  5732  5788 E QCamera : <HAL><ERROR> camEvtHandle: 803: Fatal, camera daemon died
      
      

      journalctl on the drone with crashed voxl-camera-server shows:

      Jan 01 00:00:52 apq8096 systemd[1]: Started voxl-camera-server.
      Jan 26 20:01:17 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
      Jan 26 20:01:23 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
      Jan 26 20:01:24 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
      Jan 26 20:01:35 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
      Jan 26 20:01:37 apq8096 bash[2966]: ERROR:   Recieved "Buffer" error from camera: tracking
      Jan 26 20:01:42 apq8096 bash[2966]: ERROR:   Recieved Fatal error from camera: tracking
      Jan 26 20:01:42 apq8096 bash[2966]: ERROR:   Sending request 7188, ErrorCode: -ENODEV
      Jan 26 20:01:42 apq8096 bash[2966]: WARNING: Thread: tracking request thread recieved ESTOP
      Jan 26 20:01:42 apq8096 bash[2966]: WARNING: Thread: tracking result thread recieved ESTOP
      Jan 26 20:01:42 apq8096 bash[2966]: ------ voxl-camera-server INFO: Camera server is now stopping
      Jan 26 20:01:42 apq8096 bash[2966]: WARNING: Deleting buffers: 24 of 32 still in use
      Jan 26 20:01:42 apq8096 bash[2966]: Segmentation fault:
      Jan 26 20:01:42 apq8096 bash[2966]: Fault thread: voxl-camera-ser(tid: 2966)
      Jan 26 20:01:42 apq8096 bash[2966]: Fault address: 0x8
      Jan 26 20:01:42 apq8096 bash[2966]: Address not mapped.
      Jan 26 20:01:43 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Main process exited, code=killed, status=11/SEGV[[0m
      Jan 26 20:01:43 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Unit entered failed state.[[0m
      Jan 26 20:01:43 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Failed with result 'signal'.[[0m
      

      They have not been crashed... and the failure happens regularly when in bright environments. I matched up software versions as well and the crash still happens, so it has to be some kind of hardware issue and the below can be disregarded.

      Hardware problems again with expansion or something weird in this specific camera/cable?

      Drone 1 package versions (a bit older/stable repo):

      libvoxl-cci-direct - 0.1.5
      libvoxl-cutils - 0.1.1
      voxl-camera-calibration - 0.2.3
      voxl-camera-server - 1.3.5
      voxl-cpu-monitor - 0.3.0
      voxl-docker-support - 1.2.4
      voxl-gphoto2-server - 0.0.10
      voxl-jpeg-turbo - 2.1.3-4
      voxl-libgphoto2 - 0.0.4
      voxl-libuvc - 1.0.7
      voxl-logger - 0.3.4
      voxl-mavlink - 0.1.0
      voxl-mavlink-server - 0.2.0
      voxl-modem - 0.16.1
      voxl-mongoose - 7.7.0-1
      voxl-mpa-to-ros - 0.3.6
      voxl-mpa-tools - 0.7.6
      voxl-opencv - 4.5.5-1
      voxl-portal - 0.4.2
      voxl-qvio-server - 0.9.0
      voxl-streamer - 0.4.1
      voxl-suite - 0.9.4
      voxl-system-image - 4.0-r0
      voxl-tag-detector - 0.0.4
      voxl-tflite-server - 0.3.1
      voxl-utils - 1.2.2
      voxl-uvc-server - 0.1.3
      voxl-vision-px4 - 1.4.0
      voxl-vpn - 0.0.6
      

      Drone 2 package versions (a bit newer, dev repos):

      libvoxl-cci-direct - 0.1.5
      libvoxl-cutils - 0.1.1
      voxl-camera-calibration - 0.3.0
      voxl-camera-server - 1.3.7
      voxl-cpu-monitor - 0.4.1
      voxl-docker-support - 1.2.5
      voxl-gphoto2-server - 0.0.10
      voxl-jpeg-turbo - 2.1.3-4
      voxl-lepton-server - 1.0.1
      voxl-libgphoto2 - 0.0.4
      voxl-libuvc - 1.0.7
      voxl-logger - 0.3.4
      voxl-mavlink - 0.1.0
      voxl-mavlink-server - 0.2.0
      voxl-modem - 0.16.2
      voxl-mongoose - 7.7.0-1
      voxl-mpa-to-ros - 0.3.6
      voxl-mpa-tools - 0.8.4
      voxl-opencv - 4.5.5-1
      voxl-portal - 0.4.3
      voxl-qvio-server - 0.9.0
      voxl-streamer - 0.4.3
      voxl-suite - 1.0.0
      voxl-system-image - 4.0-r0
      voxl-tag-detector - 0.0.4
      voxl-tflite-server - 0.3.1
      voxl-utils - 1.2.3
      voxl-uvc-server - 0.1.3
      voxl-vision-px4 - 1.4.1
      voxl-vpn - 0.0.6
      
      posted in Ask your questions right here!
      Steve Turner
      Steve Turner
    • RE: EV data causing the drone to crash when in stabilize mode

      https://github.com/PX4/PX4-Autopilot/issues/20950

      @Sem-Andeweg Just cross-posting here too for anyone that comes back to it. I'm interested in these issues as well.

      posted in GPS-denied Navigation (VIO)
      Steve Turner
      Steve Turner
    • RE: EV data causing the drone to crash when in stabilize mode

      Sem, (I don't work for modal!)

      A few ideas..

      Have you tried disabling the QVIO service and adjusting the EKF2_AID_MASK values to ignore the vision inputs across the modes you've tested with PX4?

      See how the platform behaves and that should either point to VIO or rule it out. In stabilized mode and without GPS and VIO inputs the platform should be relatively stable.

      posted in GPS-denied Navigation (VIO)
      Steve Turner
      Steve Turner