Configuring extrinsics on Starling (Voxl2)
-
@Moderator @Alex-Kushleyev Hi Alex, I actually don't remember exact version.
I am sharing my flight logs(links: [https://drive.google.com/drive/folders/1v0FVU-SGO8WZukiOfFMfS7vYAGeJIq3E?usp=drive_link](link url)) and video of what happens when I try to fly in position mode. It just keeps climbing and had to press kill-switch. Just to let you know I am using starling drone in its factory setting (without any hardware damage) and re-flashed it with VOXL 2 SDK 1.2.0 using the option
./install.sh -wMy current configurations:
Also, when i run voxl-health-check, it fail the test. however I can see all my camera server working in Voxl-portal
voxl2:~$ voxl-health-check SKU: MRB-D0005-4-V2-C6-T8 >> calOk: True >> compute: > imageSensor0 probe: CAM_START_DEV Success, sensor_id:0x3d,sensor_slave_addr:0x7a result: True > imageSensor1 probe: error result: False > imageSensor2 probe: CAM_START_DEV Success, sensor_id:0x7750,sensor_slave_addr:0xe2 result: True >> voxl-camera-server: > result: False > running: True > mpaOk: True > tof_conf: False > tof_pc: True > tof_depth: False > tof_ir: False > tracking: False > hires_large_color: False > hires_large_grey: False > hires_large_encoded: False > hires_small_color: False > hires_small_grey: False > hires_small_encoded: False > tof_confPipe: error > tof_pcPipe: /run/mpa/tof_pc/info > tof_depthPipe: error > tof_irPipe: error > trackingPipe: error > hires_large_colorPipe: error > hires_large_greyPipe: error > hires_large_encodedPipe: error > hires_small_colorPipe: error > hires_small_greyPipe: error > hires_small_encodedPipe: error >> voxl-mavlink-server: > result: True > running: True >> voxl-px4: > result: True > running: True > error: False > execStart: /usr/bin/voxl-px4 > sensor_accel x: -0.10558 y: -0.07159 z: -9.90697 result: True > sensor_gyro x: 0.01312 y: 0.00221 z: -0.02396 result: True > sensor_baro temperature: 31.71471 pressure: 97145.28125 result: True > sensor_mag x: -0.26683 y: -0.28708 z: 0.62417 result: True > battery_status voltage_filtered_v: 8.62994 result: True > sensor_gps lon: 0.0 lat: 0.0 alt: -17000.0 vel_m_s: 0.0 satellites_used: 0 device_id: 11272245 result: True >> voxl-imu-server: > result: True > running: True > mpaPipe: /run/mpa/imu_apps/info > device: ICM42688 >> voxl-qvio-server: > result: True > running: True > mpaPipe: /run/mpa/qvio/info >> voxl-vision-hub: > result: True > running: True > mpaPipeVoaPcOut: /run/mpa/voa_pc_out/info > mpaPipeFixed: /run/mpa/vvhub_body_wrt_fixed/info > mpaPipeLocal: /run/mpa/vvhub_body_wrt_local/info > mpaPipeFixedPoseInput: /run/mpa/vvhub_fixed_pose_input Health Check: Fail voxl2:~$Can you please try the following test:
- make sure the RC is off and QGC is not running, so there is no way to arm the drone (for safety)
- start qvio server if not already running
- connect to the drone using
voxl-portal - look at the qvio overlay stream, which should show the image from tracking camera and detected features
- while stationary, features should be locked, vio should show good quality (if not, please place the drone so that there are lots of good features with corners in the view of the camera)
- slowly pick up the drone by hand and observe whether the overlay is showing that the features being tracked (the feature mark locations are not jumping, but correctly following the actual features)
- if the above step looks good, you can carry the drone around the room and come back and put it in exact same location as you started
- the final position should be close to the origin (0,0,0) if everything is working properly (within 10-20cm, depending on the environment, etc). You can check vio position using
voxl-inspect-qvio
** If VIO blows up (features not tracked, jump around) as soon as you pick up the drone, then it is very likely that the imu to camera transform is still somehow incorrect.
Alex
-
Can you please try the following test:
- make sure the RC is off and QGC is not running, so there is no way to arm the drone (for safety)
- start qvio server if not already running
- connect to the drone using
voxl-portal - look at the qvio overlay stream, which should show the image from tracking camera and detected features
- while stationary, features should be locked, vio should show good quality (if not, please place the drone so that there are lots of good features with corners in the view of the camera)
- slowly pick up the drone by hand and observe whether the overlay is showing that the features being tracked (the feature mark locations are not jumping, but correctly following the actual features)
- if the above step looks good, you can carry the drone around the room and come back and put it in exact same location as you started
- the final position should be close to the origin (0,0,0) if everything is working properly (within 10-20cm, depending on the environment, etc). You can check vio position using
voxl-inspect-qvio
** If VIO blows up (features not tracked, jump around) as soon as you pick up the drone, then it is very likely that the imu to camera transform is still somehow incorrect.
Alex
I tried your suggestion. It works until I move the drone quickly or shake it abruptly. The quality goes to -1 and feature which were being tracked are not longer following actual features.
Also, regarding camera transform, the starling drone( with silver motor) drone I have is in factory assembly. It would be great, if you could share the config file for camera transform.
-
I tried your suggestion. It works until I move the drone quickly or shake it abruptly. The quality goes to -1 and feature which were being tracked are not longer following actual features.
Also, regarding camera transform, the starling drone( with silver motor) drone I have is in factory assembly. It would be great, if you could share the config file for camera transform.
Just to clarify, if you don't move quickly or shake, you are able to move the drone around the room and come back to the same location without VIO "blowing up"? if this is the case, then the transform is probably correct.
Are you able to share a video for a good run (had held) and a bad one (where you move quickly) - I am just trying to understand exactly what is going on. What i mean by "video" - output of the VIO overlay stream that shows the original image, features and vio stats.
I will ask the team to point you to the exact transform for your vehicle, so you can confirm whether what is configured now matches what it should be.
Alex
-
Just to clarify, if you don't move quickly or shake, you are able to move the drone around the room and come back to the same location without VIO "blowing up"? if this is the case, then the transform is probably correct.
Are you able to share a video for a good run (had held) and a bad one (where you move quickly) - I am just trying to understand exactly what is going on. What i mean by "video" - output of the VIO overlay stream that shows the original image, features and vio stats.
I will ask the team to point you to the exact transform for your vehicle, so you can confirm whether what is configured now matches what it should be.
Alex
@riteshsharma , i found the exitrinsics file that was used if you previously used the following command on old SDK
voxl-configure-extrinsics factory_starling_v1
misc_files/usr/share/modalai/extrinsic_configs/starling_v1.conf · 6fee5d13de91b80d7c9f93875d1dba49b8136955 · voxl / VOXL SDK / Utilities / voxl-mpa-tools · GitLab
Tools and utilities specific to MPA.
GitLab (gitlab.com)
However, from later commit (link), you can see that
starling_v1config file was later renamed tostarling_v1_voxl_flight, which means it was used for VOXL1 with Flight Core, not VOXL2.If possible, can you please send a picture of your drone that shows the orientation of the tracking camera (you have not changed the camera or its orientation, right?). From the picture we can confirm the actual mechanical configuration of the vehicle and then check which extrinsics parameters should be used.
Alex
-
@riteshsharma , i found the exitrinsics file that was used if you previously used the following command on old SDK
voxl-configure-extrinsics factory_starling_v1
misc_files/usr/share/modalai/extrinsic_configs/starling_v1.conf · 6fee5d13de91b80d7c9f93875d1dba49b8136955 · voxl / VOXL SDK / Utilities / voxl-mpa-tools · GitLab
Tools and utilities specific to MPA.
GitLab (gitlab.com)
However, from later commit (link), you can see that
starling_v1config file was later renamed tostarling_v1_voxl_flight, which means it was used for VOXL1 with Flight Core, not VOXL2.If possible, can you please send a picture of your drone that shows the orientation of the tracking camera (you have not changed the camera or its orientation, right?). From the picture we can confirm the actual mechanical configuration of the vehicle and then check which extrinsics parameters should be used.
Alex
@Alex-Kushleyev
Hi Alex, I was able to take picture and prepare video as you asked. Please find here Video & Images -
@Alex-Kushleyev
Hi Alex, I was able to take picture and prepare video as you asked. Please find here Video & Images@riteshsharma , thanks for the pictures and videos!
From the videos you provided, i can conclude the following:
- the transform between camera and IMU is specified correctly. If the transform was off by a 90 degree rotation (or multiple of 90), VIO would not track the features at all even in the slow test case
- the slow test seems to work perfectly, as you would expect
- aggressive shaking test shows VIO working for the first few seconds of fast motion, but then VIO blows up. In order to reset blown-up vio, i believe you need to be stationary for VIO to init (it is a qvio param, documented here - you could check your qvio server params
- the aggressive test is actually not shaking, but rotating, which is most difficult for VIO because many features are moving very fast and also getting out of frame, and features cannot be localized well due to small translational motion (compared to rotation).
So far, i don't see anything that looks out of ordinary. I think your aggressive test is too much for VIO to handle, and I am not surprised.
Now, for the next step, i suggest the following test:
- manual flight (angle + thrust)
- QVIO running and record screen capture, then we can check how QVIO did during the manual flight (without risking a crash in case QVIO is not working properly). It is possible that there is too much vibration in flight, which we can investigate using px4 log. Maybe something came loose (either camera or VOXL2 board).
- if you do run this test, please post the px4 log and also qvio output video, it would help us figure out what is going on.
Alex
-
@riteshsharma , thanks for the pictures and videos!
From the videos you provided, i can conclude the following:
- the transform between camera and IMU is specified correctly. If the transform was off by a 90 degree rotation (or multiple of 90), VIO would not track the features at all even in the slow test case
- the slow test seems to work perfectly, as you would expect
- aggressive shaking test shows VIO working for the first few seconds of fast motion, but then VIO blows up. In order to reset blown-up vio, i believe you need to be stationary for VIO to init (it is a qvio param, documented here - you could check your qvio server params
- the aggressive test is actually not shaking, but rotating, which is most difficult for VIO because many features are moving very fast and also getting out of frame, and features cannot be localized well due to small translational motion (compared to rotation).
So far, i don't see anything that looks out of ordinary. I think your aggressive test is too much for VIO to handle, and I am not surprised.
Now, for the next step, i suggest the following test:
- manual flight (angle + thrust)
- QVIO running and record screen capture, then we can check how QVIO did during the manual flight (without risking a crash in case QVIO is not working properly). It is possible that there is too much vibration in flight, which we can investigate using px4 log. Maybe something came loose (either camera or VOXL2 board).
- if you do run this test, please post the px4 log and also qvio output video, it would help us figure out what is going on.
Alex
@Alex-Kushleyev Hi Alex,
I uploaded the qvio overlay video and logs at the link https://drive.google.com/drive/folders/105QyDrfBvA52eWwrdS9NHI0q9LHTPbMA?usp=drive_link for you to review. Can you please have a look at it?
-
@Alex-Kushleyev Hi Alex,
I uploaded the qvio overlay video and logs at the link https://drive.google.com/drive/folders/105QyDrfBvA52eWwrdS9NHI0q9LHTPbMA?usp=drive_link for you to review. Can you please have a look at it?
@Alex-Kushleyev Did you have a chance to look at the logs and the video?
-
@Alex-Kushleyev Did you have a chance to look at the logs and the video?
@riteshsharma Hi @admin @Moderator @Alex-Kushleyev Can you please help me in this regard? I am stuck at it and could not move further for almost a month now.
-
@riteshsharma Hi @admin @Moderator @Alex-Kushleyev Can you please help me in this regard? I am stuck at it and could not move further for almost a month now.
@riteshsharma For VIO to work, you need to fly slow and gently in more of an autonomous fashion. Quick jerky movements will cause it to fail
-
@riteshsharma Hi @admin @Moderator @Alex-Kushleyev Can you please help me in this regard? I am stuck at it and could not move further for almost a month now.
@riteshsharma , I apologize for the delay in response.
Another suggestion is to try to fly in more feature-rich environments. Your environment has a carpet which has very few unique features and the walls are plain white. Additionally, the room is very small, so even small motion / rotation causes features to go in and out of frame, which affects VIO performance.
I do not see anything that would immediately stand out as wrong, just seems like a difficult environment for VIO to work.
Alex
-
@riteshsharma , I apologize for the delay in response.
Another suggestion is to try to fly in more feature-rich environments. Your environment has a carpet which has very few unique features and the walls are plain white. Additionally, the room is very small, so even small motion / rotation causes features to go in and out of frame, which affects VIO performance.
I do not see anything that would immediately stand out as wrong, just seems like a difficult environment for VIO to work.
Alex
@Alex-Kushleyev Hi Alex, When I hold the drone in hand and move around the room, I see that VIO is at 100%. It only fails when I shake the drone or start flying it. I'm not sure what's going wrong.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login