getting omni-vision capability
-
Hello,
I have the starling max 2 drone.
I would like to get full 360 vision coverage from the perspective of the drone (I can see in any direction).
I am lost and confused on the camera documentation:
- how I can add cameras to the drone/voxl2 (electronic connection configuration is the confusing part. I am not concerned with the mechanical engineering here at all)
- What cameras are supported? can I use raspi cameras or generic ones as well?
- Would it be possible to add enough cameras to the voxl2 to get omni-directional vision? What connection ports and cameras can I use?
- Why are there support for different camera types on different ports?
- For a stereo camera set up, could I point the two cameras in non-overlapping FOVs to get more coverage?
- If I add lots of cameras this way, would I be able to process these feeds freely and run my own algorithms to process them? What frame rates could I expect?
The big question I am having here is how could I get omni-directional vision coverage with the voxl2 from the perspective of electrical engineering?
-
I would also like to know how I can best connect a raspberry pi to the drone. What communication protocol I could use, and also if I should power the raspberry pi off of the drone somehow or if I should have another battery connected to the raspberry pi.
-
@Mark-Ogata Regarding the Raspberry Pi, you could connect with a UART or you could connect via Ethernet (With proper intermediary hardware).
-
@Eric-Katzfey what do you suggest for supporting 2 video streams of around 30fps 720p (Im assuming high bandwidth)? Could I just use the USB port available on the voxl2?
-
@Mark-Ogata Just in terms of bandwidth you will definitely see more with the Ethernet versus UART. Don't use the USB-C adapter directly on the VOXL 2. That is really only for adb. You would want an add on board that exposes one or more USB2 ports. Then you could attach a USB Ethernet dongle. It's more hardware but more bandwidth. Are you sure you even need the Raspberry Pi? Couldn't you just do everything on VOXL 2?
-
Starling 2 Max already has 4 or 5 cameras (depending on configuration - 5th could be a TOF sensor if selected). There are two high resolution IMX412 cameras (up to 4056x3040 resolution rolling shutter RGB cameras) and there are two AR0144 monochrome global shutter cameras that are used for feature tracking / VIO (1280x800 resolution).
If you really wanted to, you could add two more high resolution cameras to VOXL camera port J8 (camera ports J6 and J7 are fully used up). Adding the first IMX412 camera would just require you to buy a camera with the correct adapter, such as M0161 camera (with ucoax cable) and M0155 adapter which would plug in to VOXL2's J8. In order to get two IMX412 cameras added to starling 2 Max, it will be a bit more complicated, including a small Kernel change.
It looks like you are potentially looking to re-configure cameras on Starling 2 Max, so my suggestion is first to understand the capabilities of the cameras you have (resolution, field of view, etc) . The lenses on the IMX412 cameras have about 120 degress of horizontal FOV (and you can check it) : https://www.modalai.com/products/msu-m0161 . So for full 360 degree coverage, you would technically only need three of these cameras. However, you have a total of 4 cameras already and the lower resolution AR0144 cameras also have a wide FOV (almost 140 degrees horizontal : https://docs.modalai.com/M0166/) and can be used to supplement the two high resolutions cameras.
It seems to me that you could take the cameras that you already have and build a hardware mount for them. Also, as Eric said you might want to consider using VOXL2 for whatever you are trying to do on the Raspberry pi, since VOXL2 runs Ubuntu linux (but no display).
In terms of USB adapters, your startling 2 max already has a wifi module that uses a USB2 adapter, so you can plug in an ethernet adapter instead of that wifi dongle. BTW M0151 adapter board (which is part of Starling 2 Max, this is were the wifi dongle is plugged in) supports USB 3.0 : https://www.modalai.com/products/m0151
Alex
