https://gitlab.com/voxl-public/voxl-sdk/services/voxl-portal/-/blob/master/src/video_manager.cpp#L77
Is an example of allocating a buffer and copying the frame into it. There are some other good examples in that file as well for processing frames and converting them to various formats within the callback. Hope that helps!
Late to the party on this -- but I have been able to build ipk's off of an M1 recently.
I followed: https://docs.modalai.com/install-voxl-docker/
And then also downloaded the image from the developer portal as laid out: https://docs.modalai.com/voxl-cross/
To load:
docker load --input voxl-cross_V1.8.tar
To run:
voxl-docker -i voxl-cross:V1.8
Then the docs in all of the Modal repo's work well.
I do have buildkit etc enabled as laid out here: https://medium.com/nerd-for-tech/developing-on-apple-m1-silicon-with-virtual-environments-4f5f0765fd2f but you shouldn't need any of that. Let me know if that helps you. Worked just fine for me -- no special rosetta2 terminal or Docker version.
I'll see if I can contribute something to the docs when I have a moment.
You need to set your Ubuntu development environment with ROS to point to the ROS Master process on the VOXL.
Check out: https://answers.ros.org/question/35603/ros_master_uri-change-it-and-save-it/
So for example if the IP address of the VOXL is 192.168.1.118 then you need to execute export ROS_MASTER_URI=http://192.168.1.118:11311 on the Ubuntu system and then run rviz from the same terminal.
You then also need to run the MAVROS or MPA ROS Bridge on the VOXL along with the ROS master node.
Start ros master: roscore
mavros: https://docs.modalai.com/mavros/
On the voxl you need to edit the ROS_IP to match your voxl's IP address so that other hosts can connect to it.
If this doesn't work, consider examining your firewall on your ubuntu system to make sure it isn't block ROS traffic. If it's a vanilla ROS/Ubuntu install you should be fine.
Thanks Vinny -- we'll keep an eye out for the email!
Came over here from the other thread you tagged me in. I'm not a member of the ModalAI team, but I'll try my best.
It looks like you are using WSL for your Ubuntu VM. I have tried this previously with USB passthrough and at the time it actually was not supported by WSL/WSL2. VirtualBox or another VM solution would likely have much better support for USB passthrough. Even with that, as @tom said you are likely to encounter troubles in a VM. https://stackoverflow.com/a/61089833 Has an example that might help you.
I would confirm first that the device even shows up in Windows with ADB. https://developer.android.com/studio/releases/platform-tools Download platform tools and try running adb list
If you want to keep trying, https://www.xda-developers.com/wsl-connect-usb-devices-windows-11/ might be a good article to read through on getting USB devices to show up in your WSL VM.
We are very happy with the ModalAI APM function.
Is there an open schematic available for its design? We would like to incorporate it directly into a carrier board with other components since we know it works well for our applications. Totally understand if not!
Thanks,
Steve
@DarkKnightCH to followup on this -- a reflash of the VOXL Flight Core solved the problem for me. It was in an unresponsive state and I was unable to change parameters or talk to the Flight Core until I reflashed the current release of firmware from ModalAI.
Not out of the box -- you would need software running on the WiFi module to turn the serial/UART into packets to send over wifi in much the same way the VOXL does through voxl-vision-px4 and the mavlink server.
Something like https://www.seeedstudio.com/grove-uart-wifi-v2-esp8285.html?gclid=CjwKCAjwrNmWBhA4EiwAHbjEQAItVDD1g2yzKCTEn6CQgcpUFMOcq_KO3P3pyJJhrzkPx51yIZ2hCRoCrg4QAvD_BwE might work, but I haven't tried. Or a similar ESP-based board with a UART and Wifi included. Ublox makes the Nina line of chips as well that you could build something to do this for you.
I'm not aware of anything specifically for the VOXL 1 that would run right on the device other than compiling your own in one of the VOXL docker images.
You could try https://github.com/ASMfreaK/zeromq-openwrt since ZMQ is available for OpenWRT (which also uses the same package manager since it is based on Yocto Linux). ZMQ is fairly portable as well and should run under aarch64.
The other option is to run a bridge from any MPA/ROS topics to ZMQ in a container.
If VOXL2, I would imagine it will work out of the box since that runs Ubuntu natively.
After troubleshoothing this more. I think perhaps something is wrong with this particular VOXL Flight. I did get the other VOXL Flight working with the HIRES camera with the latest released 3.8.0-0.7 system image. I did a full reimage and wipe and it started working.
On the one unit that isn't working, it must be a hardware issue? Not sure what happened there. We haven't been unplugging the camera connections. I've tried reseating the connection several times without any luck. I've tried other cameras as well with no joy. Is there a process to return a VOXL Flight for evaluation/RMA?
Steve
@Alex-Gardner Thanks for your support a few months ago.
Having a strange issue on a VOXL 1 with the camera configuration and latest released VOXL system image. This is a Starling with ToF, Tracking, and HiRES camera all connected. I am running VOXL Platform 3.8.0-0.7 and voxl-camera-server 0.9.1.
When I run voxl-configure-cameras 5 and start the server all is well. I get good outputs from the ToF and the tracking camera. However, when I configure with voxl-configure-cameras 6 to enable the HIRES camera, I get a segfault as shown below. I've tried reseating the camera a few times to see if that made a difference. I've also tried now on two devices with the same result since upgrading to 3.8.0 system image. I'm sure I'm missing something silly or maybe a dependency is off?
Here's the output from the camera server before it crashes.
root@apq8096:~# voxl-camera-server -d 0
------ voxl-camera-server INFO: Port J2
------ voxl-camera-server: Done configuring hires camera
Port : port_J2
Name : hires
Enabled : 1
Type : hires
Api : hal3
P-W : 640
P-H : 480
P-Fmt : nv21
FPS : 30
OverrideId : -1
AEAlgo : isp
------ voxl-camera-server INFO: Port J3
------ voxl-camera-server: Done configuring tof camera
Port : port_J3
Name : tof
Enabled : 1
Type : tof
Api : hal3
P-W : 224
P-H : 1557
P-Fmt : blob
FPS : 15
Tof Mode : 9
OverrideId : -1
AEAlgo : isp
------ voxl-camera-server INFO: Port J4
------ voxl-camera-server: Done configuring tracking camera
Port : port_J4
Name : tracking
Enabled : 1
Type : tracking
Api : hal3
P-W : 640
P-H : 480
P-Fmt : raw8
FPS : 30
OverrideId : -1
AEAlgo : modalai
=================MODALAI Auto Exposure Settings==================
gain_min: 100
gain_max: 1000
exposure_min_us: 20
exposure_max_us: 30000
desired_msv: 68.000000
k_p_ns: 6000.000000
k_i_ns: 20.000000
max_i: 250.000000
p_good_thresh: 3
exposure_period: 2
gain_period: 4
display_debug: no
exposure_offset_for_gain_calc: 3000
=================================================================
Writing new configuration to /etc/modalai/voxl-camera-server.conf
Created pipe: hires channel: 0
Created pipe: tof channel: 6
Created pipe: tracking channel: 7
Starting Camera: tracking
SUCCESS: Camera module opened
----------- Number of cameras: 2
----------- ToF camera id: 0
----------- Mono camera id: 1
------ voxl-camera-server INFO: Client: qvio-server0 connected to channel: 7
Available resolutions for camera: tracking:
640 x 480
320 x 240
Resolution: 640 x 480 Found!
------ voxl-camera-server: Preview buffer allocations
Internal Exposure for: tracking 7
/builds/voxl-public/voxl-sdk/services/apq8096-camera-server/src/expgain_interface/expgain_interface_factory.cpp 45 Creating exposure interface
Creating ModalAI exposure interface
Initialize
Initialize
Port : port_J4
Name : tracking
Enabled : 1
Type : tracking
Api : hal3
P-W : 640
P-H : 480
P-Fmt : raw8
FPS : 30
OverrideId : -1
AEAlgo : modalai
=================MODALAI Auto Exposure Settings==================
gain_min: 100
gain_max: 1000
exposure_min_us: 20
exposure_max_us: 30000
desired_msv: 68.000000
k_p_ns: 6000.000000
k_i_ns: 20.000000
max_i: 250.000000
p_good_thresh: 3
exposure_period: 2
gain_period: 4
display_debug: no
exposure_offset_for_gain_calc: 3000
=================================================================
Starting Camera: tof
------ voxl-camera-server: Preview buffer allocations
SUCCESS: TOF interface created!
Received raw10 frame, checking to see if is actually raw8
Frame was actually 8 bit, sending as is
Internal AE new val:exposure 20000 gain 100
Internal AE new val:exposure 406225 gain 100
Internal AE new val:exposure 804255 gain 100
=========== modalai Royale3.31, Spectre4.7 CameraDevice::activateUseCase() : return SUCCESS!!
Setting use case: mode=9, fps = 15
=========== modalai Royale3.31, Spectre4.7 CameraDevice::activateUseCase() : return SUCCESS!!SUCCESS: TOF interface initialized!
Libcamera sending RAW16 TOF data. App calling the PMD libs to postprocess the RAW16 data
Loading lens parameters from /data/misc/camera/irs10x0c_lens.cal. Port : port_J3
Name : tof
Enabled : 1
Type : tof
Api : hal3
P-W : 224
P-H : 1557
P-Fmt : blob
FPS : 15
Tof Mode : 9
OverrideId : -1
AEAlgo : isp
Starting Camera: hires
------ voxl-camera-server INFO: Camera server is now stopping
There is a chance that it may segfault here, this is a mmqcamera bug, ignore it
------ voxl-camera-server INFO: Stopping tracking camera
------ Result thread on camera: tracking recieved stop command, exiting
------ voxl-camera-server INFO: tracking camera stopped successfully
------ voxl-camera-server INFO: Stopping hires camera
Here is my config file:
{
"version": 0.2,
"port_J2": {
"name": "hires",
"enabled": true,
"type": "hires",
"api": "hal3",
"frame_rate": 30,
"override_id": -1,
"auto_exposure_mode": "isp",
"preview": {
"enabled": true,
"width": 640,
"height": 480,
"format": "nv21"
},
"video": {
"enabled": false,
"width": 1024,
"height": 768,
"format": "h265"
},
"snapshot": {
"enabled": false,
"width": 1024,
"height": 768,
"format": "jpg"
}
},
"port_J3": {
"name": "tof",
"enabled": true,
"type": "tof",
"api": "hal3",
"tof_mode": 9,
"frame_rate": 15,
"override_id": -1,
"auto_exposure_mode": "isp",
"preview": {
"enabled": true,
"width": 224,
"height": 1557,
"format": "blob"
}
},
"port_J4": {
"name": "tracking",
"enabled": true,
"type": "tracking",
"api": "hal3",
"frame_rate": 30,
"override_id": -1,
"auto_exposure_mode": "modalai",
"manual_gain": 1000,
"modal_ae_gain_min": 100,
"modal_ae_gain_max": 1000,
"modal_ae_gain_period": 4,
"modal_ae_exposure_min_us": 20,
"modal_ae_exposure_max_us": 30000,
"modal_ae_exposure_period": 2,
"modal_ae_exposure_offset": 3000,
"modal_ae_desired_msv": 68,
"modal_ae_k_p_ns": 6000,
"modal_ae_k_i_ns": 20,
"modal_ae_max_i": 250,
"modal_ae_good_thresh": 3,
"modal_ae_display_debug": false,
"preview": {
"enabled": true,
"width": 640,
"height": 480,
"format": "raw8"
}
}
}
@DarkKnightCH I have not yet, but will be getting that system back to see what the issue was. Will let you know what I find.