https://gitlab.com/voxl-public/voxl-sdk/services/voxl-portal/-/blob/master/src/video_manager.cpp#L77
Is an example of allocating a buffer and copying the frame into it. There are some other good examples in that file as well for processing frames and converting them to various formats within the callback. Hope that helps!
Late to the party on this -- but I have been able to build ipk's off of an M1 recently.
I followed: https://docs.modalai.com/install-voxl-docker/
And then also downloaded the image from the developer portal as laid out: https://docs.modalai.com/voxl-cross/
To load:
docker load --input voxl-cross_V1.8.tar
To run:
voxl-docker -i voxl-cross:V1.8
Then the docs in all of the Modal repo's work well.
I do have buildkit etc enabled as laid out here: https://medium.com/nerd-for-tech/developing-on-apple-m1-silicon-with-virtual-environments-4f5f0765fd2f but you shouldn't need any of that. Let me know if that helps you. Worked just fine for me -- no special rosetta2 terminal or Docker version.
I'll see if I can contribute something to the docs when I have a moment.
While testing with VOXL and a Flight Core -- I am remote to the tester -- we performed a system image update so I could install some of the new variants of libraries the ModalAI team has been working on.
Once we did that and walkthrough the MPA configuration wizard, we began getting out of vixl-vision-px4:
WARNING PX4 DISCONNECTED FROM UART
I've checked the port configurations and it all looks good. The flight core has solid green and blinking blue LED so it looks like its alive. Any thoughts on what we are missing as far as config? I'm out of ideas! Thanks for any help.
I'll get the unit back in a week or so and can use the PX4 debug cable to dig further.
Modal AI ESC&2203.5 -- I don't see any reason why this wouldn't work.
Battery voltage selection depends on how fast you need to spin the given prop of choice to generate sufficient lift/performance you need in your platform. If you spin the motor at one speed with 3S and that generates the performance you need then go with that. If you move to 4S and in doing so get the performance you want in terms of speed, endurance etc then go with 4S. Really depends on what your goals are. You can pick up a variety of battery pack configurations from any drone/FPV store and try those out -- just make sure you match your connector or get an adapter.
Here are some posts on USB Cameras. I have tried this with some -- it works. https://docs.modalai.com/uvc-streaming/
https://docs.modalai.com/voxl-streamer/
For something like the Zed2 you probably need some custom drivers on the VOXL to work properly.
Re: other color cameras -- defer to the ModalAI team on those.
Hope this helps,
Steve
Alex,
This new work looks great on the dev branch. We might give it a try to see how it performs.
Highjacking the thread a bit -- have you looked at other VIO algorithms/implementations? Or is QC MVVISLAM about the best you guys have found on the platform, especially given what I imagine is hardware acceleration provided by the library?
Thanks!
Dan,
Check out this thread: https://forum.modalai.com/topic/589/flying-in-poorly-lit-conditions?_=1654205018092
TLDR is you need enough features for QVIO to pick points out for VIO. So you either need a diffuse light source that can illuminate features or some other low-light sensor that you would need to integrate with QVIO.
@yu-zhang If you are OK with losing any changes you made in the submodules, you can make sure you have checked out the v1.11.3-0.2.3 branch created by Modal and then in the root of the PX4 repo run: git submodule foreach --recursive 'git reset --hard' that will reset each subrepo back to a known state.
You can also go into a particular submodule and update/pull to whatever revision you want. .gitmodules contains the commited submodules. Versions of submodules are tracked in the commit of the outer repo.
https://www.vogella.com/tutorials/GitSubmodules/article.html is a good quick tutorial on some common commands for submodules. https://git-fork.com is a good GUI client for Windows and MacOS as well. GitKraken is good for Linux or if you want one client SmartGit will work on all platforms. None of these are free options but can help if the CLI is not your preferred interface.
It looks like v1.11.3-0.2.3 uses https://github.com/PX4/PX4-GPSDrivers/tree/bcd57ae2396fe0fd88b725d81a686e072ef0ecc9 which was commited April 29 2020. Update that submodule to the versions, try to recompile and see what happens.
I did a quick check and it looks like there may be some compile issues to resolve to make it work.
[579/1381] Building CXX object src/drivers/gps/CMakeFiles/drivers__gps.dir/devij
FAILED: src/drivers/gps/CMakeFiles/drivers__gps.dir/devices/src/mtk.cpp.obj
/usr/local/bin/ccache /opt/homebrew/bin/arm-none-eabi-g++ -DCONFIG_ARCH_BOARD_Pp
In file included from /Users/sturner/Documents/code/darkhive/modal/px4-firmware,
from /Users/sturner/Documents/code/darkhive/modal/px4-firmware:
/Users/sturner/Documents/code/darkhive/modal/px4-firmware/src/drivers/gps/devicd
283 | void gotRelativePositionMessage(sensor_gnss_relative_s &gnss_relative)
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
I've not tried latest PX4 v1.12.3 on the Flight Core myself but I would imagine you could cherry pick some of the GPS logging changes into v1.11.3-0.2.3
@Alex-Gardner Definitely interested in open VIO options. Would be awesome being able to use other techniques for picking out key points and even accelerate them on the hardware. Maybe QVIO already does that...
There is nothing that would prevent the VOXL Flight Deck or any of MODAL AI's kit from being used in a fixed wing configuration. The underlying autopilot, PX4 can be configured to fly a fixed wing aircraft. The software on the VOXL "brain" is tuned for quadcopters and will fly out of the box but with adjustments I'm sure it can be done.
Someone from ModalAI can chime in, but it would not surprise me if other customers are already flying Modal kit on a variety of other platforms from ground, to sea, and other air configurations.
We are attempting to use the VOXL ESC on our platform and having difficulties driving some smaller brushless motors, not unlike those on the Starling with the EMAX 1306B's.
Standalone with other ESC's we are able to drive our motors without a voltage drop and generate the thrust/RPM we need off of a 2S power supply at nominal voltage of 7.4V.
However, I've noticed when running voxl-esc-calibrate.py that the voltage drops as reported by the VOXL ESC, despite being connected to a constant power source that confirms it is supply a continuous 7.4 Volts.
As a result our curve to calibrate the ESC comes out like this:
Quadratic fit: motor_voltage = a2*rpm_desired^2 + a1*rpm_desired + a0
a0 = 69.74106389340773
a1 = 0.24428260831710763
a2 = 4.98882913108724e-06
ESC Params (after scaling):
pwm_vs_rpm_curve_a0 = 69.74106389340773
pwm_vs_rpm_curve_a1 = 0.24428260831710763
pwm_vs_rpm_curve_a2 = 4.98882913108724e-06
Additionally the ESC reports the following as the last line of calibration:
POW: 89, RPM: 17791.00, Voltage: 6.79V, Current: 2.90A
On our other ESC's with the same airframe we are able to take off no problem and drive the motors at a much lower current for constant voltage.
We will keep digging in, likely an issue on our part but it seems like the VOXL-ESC is not able to maintain voltage and perform in line with the other ESC's we've prototyped against. Appreciate any thoughts or insight!