Configuring extrinsics on Starling (Voxl2)

Alex Kushleyev

Just to clarify, if you don't move quickly or shake, you are able to move the drone around the room and come back to the same location without VIO "blowing up"? if this is the case, then the transform is probably correct.

Are you able to share a video for a good run (had held) and a bad one (where you move quickly) - I am just trying to understand exactly what is going on. What i mean by "video" - output of the VIO overlay stream that shows the original image, features and vio stats.

I will ask the team to point you to the exact transform for your vehicle, so you can confirm whether what is configured now matches what it should be.

Alex

Alex Kushleyev

@riteshsharma , i found the exitrinsics file that was used if you previously used the following command on old SDK voxl-configure-extrinsics factory_starling_v1

https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/blob/6fee5d13de91b80d7c9f93875d1dba49b8136955/misc_files/usr/share/modalai/extrinsic_configs/starling_v1.conf

However, from later commit (link), you can see that starling_v1 config file was later renamed to starling_v1_voxl_flight, which means it was used for VOXL1 with Flight Core, not VOXL2.

If possible, can you please send a picture of your drone that shows the orientation of the tracking camera (you have not changed the camera or its orientation, right?). From the picture we can confirm the actual mechanical configuration of the vehicle and then check which extrinsics parameters should be used.

Alex

riteshsharma

@Alex-Kushleyev
Hi Alex, I was able to take picture and prepare video as you asked. Please find here Video & Images

Alex Kushleyev

@riteshsharma , thanks for the pictures and videos!

From the videos you provided, i can conclude the following:

the transform between camera and IMU is specified correctly. If the transform was off by a 90 degree rotation (or multiple of 90), VIO would not track the features at all even in the slow test case
the slow test seems to work perfectly, as you would expect
aggressive shaking test shows VIO working for the first few seconds of fast motion, but then VIO blows up. In order to reset blown-up vio, i believe you need to be stationary for VIO to init (it is a qvio param, documented here - you could check your qvio server params
the aggressive test is actually not shaking, but rotating, which is most difficult for VIO because many features are moving very fast and also getting out of frame, and features cannot be localized well due to small translational motion (compared to rotation).

So far, i don't see anything that looks out of ordinary. I think your aggressive test is too much for VIO to handle, and I am not surprised.

Now, for the next step, i suggest the following test:

manual flight (angle + thrust)
QVIO running and record screen capture, then we can check how QVIO did during the manual flight (without risking a crash in case QVIO is not working properly). It is possible that there is too much vibration in flight, which we can investigate using px4 log. Maybe something came loose (either camera or VOXL2 board).
if you do run this test, please post the px4 log and also qvio output video, it would help us figure out what is going on.

Alex

riteshsharma

@Alex-Kushleyev Hi Alex,

I uploaded the qvio overlay video and logs at the link https://drive.google.com/drive/folders/105QyDrfBvA52eWwrdS9NHI0q9LHTPbMA?usp=drive_link for you to review. Can you please have a look at it?

riteshsharma

@Alex-Kushleyev Did you have a chance to look at the logs and the video?

riteshsharma

@riteshsharma Hi @admin @Moderator @Alex-Kushleyev Can you please help me in this regard? I am stuck at it and could not move further for almost a month now.

Moderator

@riteshsharma For VIO to work, you need to fly slow and gently in more of an autonomous fashion. Quick jerky movements will cause it to fail

Alex Kushleyev

@riteshsharma , I apologize for the delay in response.

Another suggestion is to try to fly in more feature-rich environments. Your environment has a carpet which has very few unique features and the walls are plain white. Additionally, the room is very small, so even small motion / rotation causes features to go in and out of frame, which affects VIO performance.

I do not see anything that would immediately stand out as wrong, just seems like a difficult environment for VIO to work.

Alex

riteshsharma

@Alex-Kushleyev Hi Alex, When I hold the drone in hand and move around the room, I see that VIO is at 100%. It only fails when I shake the drone or start flying it. I'm not sure what's going wrong.