Opkg install voxl-streamer

Hi Yu,

I've got answers to some of your questions, and will ping others to help with the rest.

voxl-cross and voxl-emulator (and voxl-hexagon if you're trying to make apps that use the SDSP) are build environments. voxl-emulator has a lot of the rootfs from voxl, allowing you to use some of the proprietary libs like MVSDK on voxl (which is why it's behind a EULA). voxl-cross is intended to be a lighter-weight image with the proper toolchains to allow you to build code for use on voxl in a more user-friendly environment. For your use case, you'll want to clone voxl-vision-px4, modify (or add your own) offboard mode to do the behavior you want, rebuild it in voxl-cross, and then deploy it to your drone.
As in 1, voxl-emulator is a build environment, we do not provide a simulator in there. If you want to test code against simulated data, you can use the voxl-logger command on voxl to log camera and imu data (look through the docs to make sure you include all the data you want), then disable the camera and imu services and use voxl-replay to replay the logs.
Will ping others for this one
All of the code that we provide is intended to run on the platform, not on your host pc. You should just run voxl-portal on the device (or enable the service to have it run in the background), and then you can view data in the browser.
The source code for the figure eight offboard mode is here. There are parameters at the top that you can play with, and looking through this file would be the right starting point for how to write your own offboard mode. Additionally, if you want a more dynamic approach, we already support "offboard trajectory" which opens up a pipe in the filesystem for you to write a polynomial trajectory to have the drone fly, but note that it WILL fly the path, so if you're trying to play with that feature, start very carefully by writing a path from it's current location to fly forward a meter or something of the likes.
The output of the vio algorithm is written via MPA to /run/mpa/qvio and can be accessed via an mpa pipe. An example of how to do this is the voxl-inspect-qvio tool, the source for which is here. I wouldn't exactly recommend trying to use an AI model for localization, as it'll run much more slowly than the VIO algorithm, which has been optimized for the hardware and outputs data much more frequently than it receives camera frames (as it uses the imu as well). For research purposes however, thish could be achieved with voxl-logger, which can log imu, camera, and vio data, which could then be used to train a model. You could then then use tflite server to run the model, and write your own offboard mode to take whatever data your model provides and behave as you wish.
All of the raw camera data is published via MPA pipes to /run/mpa/{camera_name}. you can use voxl-logger to compress the data and write it to disk, or write a simple tool (using voxl-inspect-cam as a template for reading the raw data) and write it to a file.
Will ping others for this one

Eric Katzfey

@yu-zhang For #3, How did you configure the video input for QGC?

Regarding speed, we've flown at 8-10 m/s during missions safely. You can experiment and run a simple mission to ensure the behavior is what you expect.

Also keep in mind wind conditions. To keep things safe, we usually fly with maximum wind velocity of 15 mph (or ~7 m/s) with gusts of 5 mph (or ~2 m/s). Always good to have enough speed to overcome current wind conditions when returning to launch.

Hope this helps!

yu zhang

Hi @Alex-Gardner , Thanks so much for your detailed reply!

It's more clear for my understanding now. And I already tried the voxl-portal, and it works well for me.

All right, I will follow your suggestions to start my first development on voxl, and hope I could achieve it very soon

yu zhang

@Eric-Katzfey , The video input for QGC looks like this:

Screen Shot 2022-02-15 at 9.55.50 PM.png

Screen Shot 2022-02-15 at 9.55.55 PM.png

It seems the server on voxl already got a connection to QGC, but there is still no video in the fly view (showing for waiting for video).
Screen Shot 2022-02-15 at 9.59.11 PM.png

So, is there anything else that I should config in QGC?

yu zhang

@RichieRich Thanks for the information!

I would test the maximum speed m500 could achieve in a normal condition. So, I will run the mission for this purpose.

By the way, for the motor on m500, I noticed it's the Motor 2216-880KV for 10in props. I also saw there was a 2216-920KV version on Holybro website for s500 frame.
https://shop.holybro.com/spare-parts-s500-v2-kit_p1251.html

I double check that different the KV rating of a motor will be able to achieve different speeds. In general, the higher KV refers to the faster speed, but the lower in a heavy-lift.

For the defualt 880KV, m500 can achieve 1kg+ payload with up to 10m/s+ speed. So, I am thinking what if replacing 920KV on m500, will it achieve a faster speed? I am wondering do you hava any experiments to test using a higher KV motor before?

yu zhang

@Eric-Katzfey , I am wondering if the aspect ratio is incorrect to get the source video? The 1.77777 is the default one, and I am not sure what is the default aspect ratio on m500?

yu zhang

@Eric-Katzfey , I have a look at the /etc/modalai/voxl-streamer.conf.

It seems this config includes a set of different input and output settings. I am wondering if my issue may refer to this config? Do you have a document to introduce the mapping of this config?

E.g., the second line is "configuration": "tracking". Does it mean the view showing in QGC will be related to the view of the tracking camera?

As default, QGC will show which camera's view? If I would prefer to show the Hi-res view, how should I config it?

I also simply try to type in 1.3333 for aspect ratio, but it seems it doesn't work.

Eric Katzfey

@yu-zhang voxl-streamer documentation is located here: https://docs.modalai.com/voxl-streamer/

yu zhang

@Eric-Katzfey Thanks, I double-check the config file, and it seems everything should be good. But there is still no video showing in QGC.

I am wondering if QGC blocks the stream data. I tested both QGC 4.1.4 and the latest 4.2.0 on both my Macbooks (m1 and intel). They all cannot show the video correctly.

I found QGC 4.1.1 got this issue on m1 Macbook, but it should already be fixed in the later versions.

I am wondering what's your QGC version? Do you have a Macbook at hand? If so, could you please make a quick test to check if it's an issue with QGC?

yu zhang

Hi @Eric-Katzfey ,

I tested using VLC on amd64 Macbook, and the streamer works well. It's all the same config as in QGC. So, I assume the latest QGC (4.1.4/4.2.0) still has the issue to fail receive streaming video on host PC.

I will continue to work on my target and hope QGC will cover this issue soon later.

Thanks again!

yu zhang

This post is deleted!

yu zhang

This post is deleted!

Eric Katzfey

@yu-zhang Can you please do us a big favor and ask your questions in separate posts instead just adding them all on to a single post? The forum is a resource for all of our customers and having each question in it's own forum posting makes it much easier for other people to benefit from the information in it. And regarding QGC 4.1.4, I use that version for testing but I am on a Linux Ubuntu 20.04 machine so maybe that's the difference.

yu zhang

@Eric-Katzfey Good to know that QGC 4.1.4 can work on Ubuntu.

All right, I will repost my lastest 2 questions in separate posts. Thanks for letting me know that.

Steve Turner

Late to the party on this -- but I have been able to build ipk's off of an M1 recently.

I followed: https://docs.modalai.com/install-voxl-docker/

And then also downloaded the image from the developer portal as laid out: https://docs.modalai.com/voxl-cross/

To load:
docker load --input voxl-cross_V1.8.tar

To run:
voxl-docker -i voxl-cross:V1.8

Then the docs in all of the Modal repo's work well.

I do have buildkit etc enabled as laid out here: https://medium.com/nerd-for-tech/developing-on-apple-m1-silicon-with-virtual-environments-4f5f0765fd2f but you shouldn't need any of that. Let me know if that helps you. Worked just fine for me -- no special rosetta2 terminal or Docker version.

I'll see if I can contribute something to the docs when I have a moment.

yu zhang

Hi @stevet Late to reply, and thanks so much for your instruction! It's great news for M1 developers like me

I will have a try if it works well, and let you know then.