# Configuring extrinsics on Starling (Voxl2) Source: https://forum.modalai.com/topic/3470/configuring-extrinsics-on-starling-voxl2 Category: General Questions (https://forum.modalai.com/category/2/general-questions) Posted: 2024-05-22 17:21:00 UTC by riteshsharma Replies: 21 · Views: 6689 ## riteshsharma · 2024-05-22 17:21:00 UTC Hi, I recently ran "apt update" and "apt upgrade" on my starling drone. Since then the drone is not holding position and keep climbing. I believe it is something to do with extrinsic configuration. Before, the option for extrinsic configuration used to show "voxl-configure-extrinsics factory_starling_v1" but now the option is gone. Can you please help me how to configure extrinsics given the information below: ![Capture2.PNG](https://forum.modalai.com/assets/uploads/files/1716398348118-capture2.png) ![Capture1.PNG](https://forum.modalai.com/assets/uploads/files/1716398366306-capture1.png) ## Reply by tom (ModalAI staff) · 2024-05-22 19:12:41 UTC @riteshsharma I would re-flash your drone with the latest VOXL 2 SDK 1.2.0 via. it's `install.sh` script It seems as though your packages are all mismatched now as you appear to have switched to the dev branch of voxl-suite but are on an old system image ## Reply by riteshsharma · 2024-05-22 19:55:38 UTC (in reply to tom) @tom Hi Tom, I re-flashed with the latest VOXL 2 SDK 1.2.0 via install.sh script. Now, to configure extrinsics what should I choose? ![Capture1.PNG](https://forum.modalai.com/assets/uploads/files/1716407732718-capture1.png) I dont see any option saying startling_v1_voxl2 or as such. ## Reply by tom (ModalAI staff) · 2024-05-22 21:25:59 UTC (in reply to riteshsharma) @riteshsharma If you ran the `install.sh` script then after flashing the script would've also ran `voxl-configure-sku` and `voxl-configure-mpa` which would setup your extrinsics based off of the SKU you selected (in your case starling) ## Reply by tom (ModalAI staff) · 2024-05-22 21:27:53 UTC (in reply to tom) @tom But also if you'd like to know, the correct option for your starling is `starling_v2_voxl2` ## Reply by riteshsharma · 2024-05-22 22:31:51 UTC (in reply to tom) @tom Thank you. I tried running ```voxl-configure-sku``` and ```voxl-configure-mpa``` and flew the drone. It just do not hold position and qvio goes to -1. I then tried running ```voxl-configure-extrinsics starling_v2_voxl2``` and flew the drone again. The result was the same. The drone is not holding its position. ``` voxl2:~$ voxl-version -------------------------------------------------------------------------------- system-image: 1.7.6-M0054-14.1a-perf kernel: #1 SMP PREEMPT Wed Mar 20 19:48:52 UTC 2024 4.19.125 -------------------------------------------------------------------------------- hw platform: M0054 mach.var: 1.0 -------------------------------------------------------------------------------- voxl-suite: 1.2.0 -------------------------------------------------------------------------------- Packages: Repo: http://voxl-packages.modalai.com/ ./dists/qrb5165/sdk-1.2/binary-arm64/ Last Updated: 2024-05-22 21:32:55 List: kernel-module-voxl-fsync-mod-4.19.125 1.0-r0 kernel-module-voxl-gpio-mod-4.19.125 1.0-r0 kernel-module-voxl-platform-mod-4.19.125 1.0-r0 libfc-sensor 1.0.5 libmodal-cv 0.4.0 libmodal-exposure 0.1.0 libmodal-journal 0.2.2 libmodal-json 0.4.3 libmodal-pipe 2.10.0 libqrb5165-io 0.4.5 libvoxl-cci-direct 0.2.1 libvoxl-cutils 0.1.1 modalai-slpi 1.1.12 mv-voxl 0.1-r0 qrb5165-bind 0.1-r0 qrb5165-dfs-server 0.2.0 qrb5165-imu-server 1.0.1 qrb5165-rangefinder-server 0.1.1 qrb5165-slpi-test-sig 01-r0 qrb5165-system-tweaks 0.2.6 qrb5165-tflite 2.8.0-2 voxl-bind-spektrum 0.1.1 voxl-camera-calibration 0.5.4 voxl-camera-server 1.9.1 voxl-ceres-solver 2:1.14.0-10 voxl-configurator 0.5.6 voxl-cpu-monitor 0.4.8 voxl-docker-support 1.3.1 voxl-elrs 0.1.3 voxl-esc 1.4.0 voxl-feature-tracker 0.3.2 voxl-flow-server 0.3.3 voxl-fsync-mod 1.0-r0 voxl-gphoto2-server 0.0.10 voxl-gpio-mod 1.0-r0 voxl-jpeg-turbo 2.1.3-5 voxl-lepton-server 1.2.0 voxl-libgphoto2 0.0.4 voxl-libuvc 1.0.7 voxl-logger 0.4.0 voxl-mavcam-manager 0.5.3 voxl-mavlink 0.1.1 voxl-mavlink-server 1.4.0 voxl-microdds-agent 2.4.1-0 voxl-modem 1.0.9 voxl-mongoose 7.7.0-1 voxl-mpa-to-ros 0.3.7 voxl-mpa-to-ros2 0.0.3 voxl-mpa-tools 1.1.5 voxl-neopixel-manager 0.0.3 voxl-open-vins 0.4.4 voxl-open-vins-server 0.2.18 voxl-opencv 4.5.5-2 voxl-platform-mod 1.0-r0 voxl-portal 0.6.5 voxl-px4 1.14.0-2.0.68 voxl-px4-imu-server 0.1.2 voxl-px4-params 0.3.8 voxl-qvio-server 1.0.2 voxl-remote-id 0.0.9 voxl-ros2-foxy 0.0.1 voxl-streamer 0.7.4 voxl-suite 1.2.0 voxl-tag-detector 0.0.4 voxl-tflite-server 0.3.2 voxl-utils 1.3.9 voxl-uvc-server 0.1.6 voxl-vision-hub 1.7.4 voxl2-system-image 1.7.6-r0 voxl2-wlan 1.0-r0 -------------------------------------------------------------------------------- voxl2:~$ voxl-inspect-sku family code: MRB-D0005 (starling) compute board: 4 (voxl2) hw version: 2 cam config: 6 tx config: 8 SKU: MRB-D0005-4-V2-C6-T8 voxl2:~$ voxl-configure-extrinsics starling_v2_voxl2 wiping old extrinsics config file copying /usr/share/modalai/extrinsic_configs/starling_v2_voxl2.conf to /etc/modalai/extrinsics.conf loading and updating file with voxl-inspect-extrinsics -q done configuring extrinsics voxl2:~$ ``` ## Reply by Eric Katzfey (ModalAI staff) · 2024-05-23 15:57:15 UTC (in reply to riteshsharma) @riteshsharma Have you taken a look at: https://docs.modalai.com/flying-with-vio/#troubleshooting-vio? ## Reply by riteshsharma · 2024-06-10 19:42:43 UTC (in reply to Eric Katzfey) @Eric-Katzfey I troubleshooted using the instruction on the https://docs.modalai.com/flying-with-vio/#troubleshooting-vio. Still as soon as the drone takes off, the quality of qvio goes down to -1. I also tried to reflash the drone with the system image VOXL 2 SDK 1.2.0 and enabled every thing to factory settings, still the drone starts drifting and qvio falls below -1. But qvio at the ground always shows 100% quality, Do you have any other suggestion? ## Reply by Alex Kushleyev (ModalAI staff) · 2024-06-11 05:08:02 UTC (in reply to riteshsharma) @riteshsharma , which version of the SDK did you have before the upgrade? If you used `voxl-configure-extrinsics factory_starling_v1` before, we can check whether that matches the extrinsics that you have after the installation now. ## Reply by riteshsharma · 2024-06-12 20:57:24 UTC (in reply to Alex Kushleyev) @Moderator @Alex-Kushleyev Hi Alex, I actually don't remember exact version. I am sharing my flight logs(links: [https://drive.google.com/drive/folders/1v0FVU-SGO8WZukiOfFMfS7vYAGeJIq3E?usp=drive_link](link url)) and video of what happens when I try to fly in position mode. It just keeps climbing and had to press kill-switch. Just to let you know I am using starling drone in its factory setting (without any hardware damage) and re-flashed it with VOXL 2 SDK 1.2.0 using the option ```./install.sh -w``` My current configurations: ![Screenshot from 2024-06-12 15-54-03.png](https://forum.modalai.com/assets/uploads/files/1718225882336-screenshot-from-2024-06-12-15-54-03.png) ![Screenshot from 2024-06-12 15-52-10.png](https://forum.modalai.com/assets/uploads/files/1718225897649-screenshot-from-2024-06-12-15-52-10.png) ![Screenshot from 2024-06-12 15-55-47.png](https://forum.modalai.com/assets/uploads/files/1718225908125-screenshot-from-2024-06-12-15-55-47.png) Also, when i run voxl-health-check, it fail the test. however I can see all my camera server working in Voxl-portal ``` voxl2:~$ voxl-health-check SKU: MRB-D0005-4-V2-C6-T8 >> calOk: True >> compute: > imageSensor0 probe: CAM_START_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a result: True > imageSensor1 probe: error result: False > imageSensor2 probe: CAM_START_DEV Success, sensor_id:0x7750,sensor_slave_addr:0xe2 result: True >> voxl-camera-server: > result: False > running: True > mpaOk: True > tof_conf: False > tof_pc: True > tof_depth: False > tof_ir: False > tracking: False > hires_large_color: False > hires_large_grey: False > hires_large_encoded: False > hires_small_color: False > hires_small_grey: False > hires_small_encoded: False > tof_confPipe: error > tof_pcPipe: /run/mpa/tof_pc/info > tof_depthPipe: error > tof_irPipe: error > trackingPipe: error > hires_large_colorPipe: error > hires_large_greyPipe: error > hires_large_encodedPipe: error > hires_small_colorPipe: error > hires_small_greyPipe: error > hires_small_encodedPipe: error >> voxl-mavlink-server: > result: True > running: True >> voxl-px4: > result: True > running: True > error: False > execStart: /usr/bin/voxl-px4 > sensor_accel x: -0.10558 y: -0.07159 z: -9.90697 result: True > sensor_gyro x: 0.01312 y: 0.00221 z: -0.02396 result: True > sensor_baro temperature: 31.71471 pressure: 97145.28125 result: True > sensor_mag x: -0.26683 y: -0.28708 z: 0.62417 result: True > battery_status voltage_filtered_v: 8.62994 result: True > sensor_gps lon: 0.0 lat: 0.0 alt: -17000.0 vel_m_s: 0.0 satellites_used: 0 device_id: 11272245 result: True >> voxl-imu-server: > result: True > running: True > mpaPipe: /run/mpa/imu_apps/info > device: ICM42688 >> voxl-qvio-server: > result: True > running: True > mpaPipe: /run/mpa/qvio/info >> voxl-vision-hub: > result: True > running: True > mpaPipeVoaPcOut: /run/mpa/voa_pc_out/info > mpaPipeFixed: /run/mpa/vvhub_body_wrt_fixed/info > mpaPipeLocal: /run/mpa/vvhub_body_wrt_local/info > mpaPipeFixedPoseInput: /run/mpa/vvhub_fixed_pose_input Health Check: Fail voxl2:~$ ``` ## Reply by Alex Kushleyev (ModalAI staff) · 2024-06-14 21:30:47 UTC (in reply to riteshsharma) @riteshsharma , Can you please try the following test: - make sure the RC is off and QGC is not running, so there is no way to arm the drone (for safety) - start qvio server if not already running - connect to the drone using `voxl-portal` - look at the qvio overlay stream, which should show the image from tracking camera and detected features - while stationary, features should be locked, vio should show good quality (if not, please place the drone so that there are lots of good features with corners in the view of the camera) - slowly pick up the drone by hand and observe whether the overlay is showing that the features being tracked (the feature mark locations are not jumping, but correctly following the actual features) - if the above step looks good, you can carry the drone around the room and come back and put it in exact same location as you started - the final position should be close to the origin (0,0,0) if everything is working properly (within 10-20cm, depending on the environment, etc). You can check vio position using `voxl-inspect-qvio` ** If VIO blows up (features not tracked, jump around) as soon as you pick up the drone, then it is very likely that the imu to camera transform is still somehow incorrect. Alex ## Reply by riteshsharma · 2024-06-20 15:26:16 UTC (in reply to Alex Kushleyev) @Alex-Kushleyev I tried your suggestion. It works until I move the drone quickly or shake it abruptly. The quality goes to -1 and feature which were being tracked are not longer following actual features. Also, regarding camera transform, the starling drone( with silver motor) drone I have is in factory assembly. It would be great, if you could share the config file for camera transform. ## Reply by Alex Kushleyev (ModalAI staff) · 2024-06-20 16:49:29 UTC (in reply to riteshsharma) @riteshsharma , Just to clarify, if you don't move quickly or shake, you are able to move the drone around the room and come back to the same location without VIO "blowing up"? if this is the case, then the transform is probably correct. Are you able to share a video for a good run (had held) and a bad one (where you move quickly) - I am just trying to understand exactly what is going on. What i mean by "video" - output of the VIO overlay stream that shows the original image, features and vio stats. I will ask the team to point you to the exact transform for your vehicle, so you can confirm whether what is configured now matches what it should be. Alex ## Reply by Alex Kushleyev (ModalAI staff) · 2024-06-20 19:35:12 UTC (in reply to Alex Kushleyev) @riteshsharma , i found the exitrinsics file that was used if you previously used the following command on old SDK `voxl-configure-extrinsics factory_starling_v1` https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/blob/6fee5d13de91b80d7c9f93875d1dba49b8136955/misc_files/usr/share/modalai/extrinsic_configs/starling_v1.conf However, from later commit ([link](https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/commit/3ba45e3e9de29ee75cc9ff6cd598ee6a0794d84b)), you can see that `starling_v1` config file was later renamed to `starling_v1_voxl_flight`, which means it was used for VOXL1 with Flight Core, not VOXL2. If possible, can you please send a picture of your drone that shows the orientation of the tracking camera (you have not changed the camera or its orientation, right?). From the picture we can confirm the actual mechanical configuration of the vehicle and then check which extrinsics parameters should be used. Alex ## Reply by riteshsharma · 2024-06-24 16:36:56 UTC (in reply to Alex Kushleyev) @Alex-Kushleyev Hi Alex, I was able to take picture and prepare video as you asked. Please find here [Video & Images](https://drive.google.com/drive/folders/1sCWuzH8HgISuCofxJZNT2Upzjnmqk7hG?usp=sharing) ## Reply by Alex Kushleyev (ModalAI staff) · 2024-06-24 20:52:52 UTC (in reply to riteshsharma) @riteshsharma , thanks for the pictures and videos! From the videos you provided, i can conclude the following: - the transform between camera and IMU is specified correctly. If the transform was off by a 90 degree rotation (or multiple of 90), VIO would not track the features at all even in the slow test case - the slow test seems to work perfectly, as you would expect - aggressive shaking test shows VIO working for the first few seconds of fast motion, but then VIO blows up. In order to reset blown-up vio, i believe you need to be stationary for VIO to init (it is a qvio param, documented [here](https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/blob/master/server/config_file.h?ref_type=heads#L77) - you could check your qvio server params - the aggressive test is actually not shaking, but rotating, which is most difficult for VIO because many features are moving very fast and also getting out of frame, and features cannot be localized well due to small translational motion (compared to rotation). So far, i don't see anything that looks out of ordinary. I think your aggressive test is too much for VIO to handle, and I am not surprised. Now, for the next step, i suggest the following test: - manual flight (angle + thrust) - QVIO running and record screen capture, then we can check how QVIO did during the manual flight (without risking a crash in case QVIO is not working properly). It is possible that there is too much vibration in flight, which we can investigate using px4 log. Maybe something came loose (either camera or VOXL2 board). - if you do run this test, please post the px4 log and also qvio output video, it would help us figure out what is going on. Alex ## Reply by riteshsharma · 2024-06-26 16:36:19 UTC (in reply to Alex Kushleyev) @Alex-Kushleyev Hi Alex, I uploaded the qvio overlay video and logs at the link https://drive.google.com/drive/folders/105QyDrfBvA52eWwrdS9NHI0q9LHTPbMA?usp=drive_link for you to review. Can you please have a look at it? ## Reply by riteshsharma · 2024-07-10 19:15:54 UTC (in reply to riteshsharma) @Alex-Kushleyev Did you have a chance to look at the logs and the video? ## Reply by riteshsharma · 2024-07-22 19:31:50 UTC (in reply to riteshsharma) @riteshsharma Hi @admin @Moderator @Alex-Kushleyev Can you please help me in this regard? I am stuck at it and could not move further for almost a month now. ## Reply by Moderator (ModalAI staff) · 2024-07-27 16:13:30 UTC (in reply to riteshsharma) @riteshsharma For VIO to work, you need to fly slow and gently in more of an autonomous fashion. Quick jerky movements will cause it to fail ## Reply by Alex Kushleyev (ModalAI staff) · 2024-08-01 20:54:56 UTC (in reply to riteshsharma) @riteshsharma , I apologize for the delay in response. Another suggestion is to try to fly in more feature-rich environments. Your environment has a carpet which has very few unique features and the walls are plain white. Additionally, the room is very small, so even small motion / rotation causes features to go in and out of frame, which affects VIO performance. I do not see anything that would immediately stand out as wrong, just seems like a difficult environment for VIO to work. Alex ## Reply by riteshsharma · 2024-08-06 15:20:19 UTC (in reply to Alex Kushleyev) @Alex-Kushleyev Hi Alex, When I hold the drone in hand and move around the room, I see that VIO is at 100%. It only fails when I shake the drone or start flying it. I'm not sure what's going wrong.