@mark , it looks like your version of voxl sdk has an early version of the imx664 driver (not supporting 2704x1540 resolution.
Are you able to upgrade sdk to 1.5.x or 1.6.x?
Alternatively, i could share the latest camera drivers to use with 1.4.5.
I would also recommend updating voxl-camera-server to the latest, but that would also require updating other packages.. would you consider upgrading the sdk or are you locked to 1.4.5?
Alex
@Leo-Allesch , if you are talking about VSLAM, which stands for Visual Simultaneous Localization and Mapping, then this is a sub-class of SLAM which uses only cameras (and often IMUs, then it is VISLAM).
VSLAM and VISLAM usually generate and maintain a sparse map of 3D features (points), which is used for self localization.
If you want to be able to detect a target, and you have your 3D pose from VIO (VIO = VISLAM), then you can either triangulate the target location or get it's location from a single frame if you know the size of the target (such as April tag, etc). Then you can transform the coordinate of the tag from the camera frame into your "world" (VIO) coordinates.
The depth sensor would allow you to build a more detailed 3D map / point cloud using the 3D pose output from the VIO algorithm (or just using a depth sensor, which may be more complicated). You can also build a dense 3D map using camera only (and IMU), but that typically requires a lot more computation than using a depth sensor.
You don't necessarily need a depth sensor for identifying the position of a specific target that is easily detected by a camera.
There is a lot of research material you can find on these topics, please look into it deeper and let us know if you have specific questions about using VOXL2 hardware and software.
Alex
@mark ,
Please try using input resolution of 2704x1540, not the binned one which you have currently selected. Then you can set MISP width and height whatever you need and the output should be properly downscaled and cropped. Choosing higher input resolution will result in a better downscaled image due to debayering artifacts at lower resolution.
The reason why the image decoding does not work has to do with the image line stride of the raw10 buffer not matching the requirement of the opencl debayering functions. In newer versions of camera server, we automatically re-align the image to make it work, but that requires copying the whole bayer image into a different buffer (more cpu overhead).
Using input resolution of 2704x1540 will result in best results in terms of image quality and cpu usage. This resolution works up to 120FPS. Binned mode does not actually provide any advantages (specifically for the IMX664 camera), not even faster read-out or higher max fps or anything like that.
Please try that and let me know if that works for you.
Alex
@Catalystmachine , for questions regarding orders, please reach out here : https://www.modalai.com/pages/contact-us
Hi @Hunter-Scott ,
We are looking into this. We may not have an exact CAD model for every single variant of the products. Is there something specific in your product that significantly deviates from the provided CAD files? Maybe we can provide something that is close (but not exactly correct).
Alex
Hi @jimbow77 , @oasisartisan ,
I updated the voxl-camera-server on a branch to save the tof v2 calibration to a file. Here is the commit : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commit/f7aadc13246df35757498ee8c2633897864d47ff .
Are you able to build the camera server from this branch?
https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/tree/save-tof-v2-lens-cal
We will review the changes more thoroughly a bit later and we can merge them to dev..
On my drone, the calibration file (which will include the serial number of the TOF sensor) is created when running camera server:
cat /data/misc/camera/irs2975c_8E7F-3715-16E7-2CA0_lens.cal
VERSION:1.0:VERSION
#px,py,cx,cy,tan-coeff[0..1], rad-coeff[0..n-1]
120.985 87.9267 101.435 101.435 0.000431714 -0.000511291 -0.0315991 -0.0224586 0.00496448
Alex
@Andrew-Jong , we are able to reproduce the error that you are seeing. I believe that you may need to set up a virtual display in order to get this to work. You could try doing so using Weston compositor, which would normally run if the display was available. Please try it and let us know if you are able to get it to work (or not).
You would need to create a config file /etc/xdg/weston/weston.ini
and try running it manually like this :
weston --idle-time=0 --config=/etc/xdg/weston/weston.ini
a sample config file might look something like this:
# configuration file for Weston
[core]
idle-time=0
repaint-window=15
[output]
name=virtual-1
mode=on
We have not tested this yet, but hopefully it's a good start.
Alex
https://www.modalai.com/products/m0155
https://www.modalai.com/products/m0188 (fixed link)
https://www.modalai.com/products/m0194
If you need M0195, you can contact us directly for a custom order (it's not in the shop yet).
Alex
Please see some responses below:
i2c_geni a84000.i2c / 884000.i2c: i2c error :-107
This is likely due to a service that is enabled for a sensor that's not present, so the service is restarting and trying to detect the sensor.
voxl-rangefinder-server is not running and also voxl-lepton-server. you can disable the service using systemd : systemctl disable voxl-rangefinder-server.CAM_ERR: CAM-SENSOR: cam_cci_i2c_read: 35 rc = -22
this one is usually present when you have a sensormodule driver in /usr/lib/camera for a camera that is not actually plugged in, so the probe fails. Double check the cameras that you have plugged in vs sensormodules peresent in the directory above. However, this is usually harmless, if all of your cameras are working properly.
Pin conflict.. We will double check that, but GPIO 130 is on a Board to Board connector that you are most likely not using, so that should not be an issue.
VIO performance..
Please let us know if you have any other questions.
Alex
@oasisartisan , I just checked the API for the TOF sensor and it seems we can get the lens calibration. I would need to make a change in the voxl camera server to try it. Please give me a few days..
Alex
@Leo-Allesch , actually we do have a very limited quantity of the TOF V1, so hopefully that would avoid doing the re-design of the camera mounts and flex routing.
Please reach out to https://modalai.com/contact for a custom order. you probably should get one or two of M0135 as well.
Alex
So it seems like the VOXL2 ports (J6L, J7L, J7U) seem to be functioning (when tested with the hires or tracking camera and a working M0135)?
Can you please provide more information on your requirements: VSLAM and target detection. Both can be done without TOF sensor. TOF sensor is often used to help with obstacle detection / avoidance in indoor environments.
The easiest way to add the new TOF sensor to your drone would be to use the following:
The connection for the new TOF module to VOXL2 would look like this :
VOXL2 <-> M0172 <-> M0177 <-> M0178 <-> IRS2975 TOF sensor
You are right that the cable routing may be an issue, you would likely need to update the mounting bracket for the camera as well.
I am going to double check if we have any TOF V1 left at all.. will get back to you soon.
Replacement M0135 (if you need):
Alex
@Dan-Jennings , M0181 can be used to enable a Boson and another hires or tracking camera (or Hadron combo), see diagram below. You would need to use the kernal variant 1.0.1 (same one used for M0173, which I am assuming you are already using).
So basically, you can use any standard config C26-C29 + two more cameras connected to VOXL2 J8. Please note that VOXL2 J8 shares CCI (i2c) buses with J6 and J7, so you have to be a little careful about what cameras you connect to J8 to avoid i2c address conflicts. I am going to put together some more information on that soon. But if you are adding a Hadron or Boson + IMX412 to J8, there will be no conflict.
Please let us know if you have any other questions.
Alex
It seems the TOF sensor and camera port 2 are not functional. I am assuming that you tried using both M0135 adapters to test port 2? Considering that it was working before, this is likely not a software issue, but a hardware issue, but it is not clear why exactly.
All of your cameras are interchangeable in the camera ports, as long as the camera ports and sensors are working (and the sensormodule drivers are set up correctly).
By default, the port 1 (camera slot 1), is reserved for use in a stereo configuration when used with M0076, like this one : https://docs.modalai.com/voxl2-camera-configs/#c10---front-stereo-only . So it will not work as a generic camera port when used with M0135. The functionality can be re-configured in software (requires a change in the kernel), but we don't have a ready-to-go kernel with just this change (we can revisit this later, if needed).
By the way, you can also test VOXL2 J8. J8 is set up in a similar way as J6, that is the J8L can be used for any camera, but J8U is reserved (by default) for another stereo pair. However, this will allow you to test yet another port just to double check things. The camera slot IDs for J8 are 4 and 5. Please note that the orientation of J8 is rotated compared to J6 and J7. You can see how a TOF sensor is attached to VOXL2 J8 via M0076 adapter : https://docs.modalai.com/voxl2-camera-configs/#cx---two-time-of-flights-tof . M0076 is a single port version of M0135 interposer (only providing the Lower camera port).
So, if you test VOXL2 J8, use the lower camera port 4 (J8L).
Yes, M0040 is EOL, so the replacements are not available. The upgraded version of TOF sensor is here : https://docs.modalai.com/M0169/ , however it has different dimensions and connector requirements. We can discuss this further if needed.
What is your goal? Do you need the original configuration working or are you potentially looking for any updates?
Alex
OK, i understand. So i think the next step is to figure out what exactly is not working: cameras, M0135 interposers or VOXL2.
Since we know that the camera slot 3 hardware and software path seems to be working, you can try plugging in other cameras (hires, TOF) in to that slot:
com.qti.sensormodule.imx214_3.bin from /usr/share/modalai/chi-cdk/imx214 to /usr/lib/camera/. You can leave the existing sensormodules in /usr/lib/camera/, no need to delete.voxl-camera-server -l to see if the camera is detectedcom.qti.sensormodule.irs1645_3.bin camera driver)Alternatively, if you have another VOXL2, you could do some testing with that, but I am assuming that another VOXL2 is not available.
Alex
@Leo-Allesch , thanks for checking that. I am assuming that the cameras used to work at some point - can you confirm? What happened between the working and non-working state?
Alex
@Leo-Allesch , sorry for the delay.
Based on your previous post, only one camera is detected:
Cam idx: 0, Cam slot: 3, Slave Address: 0x00E2, Sensor Id: 0x7750
(if more cameras were detected, you would see them show up in that list, which is output of voxl-camera-server -l).
This is the tracking camera in camera slot 3 (VOXL2 J7 Upper slot). The slot numbers are also labeled in the diagram https://docs.modalai.com/voxl2-d0005/, specifically you can see 0, 2, 3 labeled on the M0135 interposers that are plugged into VOXL2:
I do see the correct sensormodules in /usr/lib/camera/. (slot 0 for TOF and slot 2 for IMX214).
I suspect that you have the hires camera and TOF module plugged in backwards into the M0135 adapters. The tracking camera (unfortunately?) has a different connector orientation compared to other cameras, but pin1 is correctly marked with a dot (or a number 1) on all connectors.
Your cameras should be connected like in the picture below:
Another look (in the assembly of the drone):
Please double check the connections.
Alex
@jimbow77 and @oasisartisan ,
TOF V1 (which is now EOL, so you don't have it on Starling 2 Max) was set up the following way:
/data/misc/camera/, then re-used for next timepmd.spc, tof_cal_eeprom.bin and irs10x0c_lens.cal - the latter likely containing the intrinsics calibraiton data (although i have not checked explicitly)TOF V2 (what is shipping in Starling 2 (and Max) ), is set up so that the TOF Library downloads (via i2c) the calibration data each time the voxl-camera-server starts. This is done internally to the TOF libraries. I don't know exactly why there is a difference between how the calib data is handled for two sensors by the TOF software, but this is how our sensor vendor helped us set it up. The TOF processing software downloads the calibration from the sensor each time voxl-camera-server starts and applies it to generate the point cloud. It may be possible to get the TOF intrinsics using the TOF library API, but we have not checked.
Do you have a specific reason why the TOF intrinsics are needed?
If you wanted to calibrate the TOF sensor yourself, then you would either need a checkerboard some material that is IR reflective ("white") and non-reflective ("black"). Alternatively, you could use an array of IR leds (or IR reflective dots + IR light) and use dot pattern detector as opposed for checkerboard for calibration.
Regarding Hi-res cameras, the intrinsics are not calibrated and not included. It can be calibrated using our camera calibration app (voxl-camera-calibration), but you should do it at half resolution or smaller (not full size like 4056x3040), otherwise the app runs very slowly. If you need more details how to calibrate at high resolution, let me know (involves downscaling exactly by a factor of 2 and then upscaling the resulting intrinsics)
Alex
Hi @Haxerus ,
I will double check regarding enabling J10 in spi mode on VOXL2 Mini.
Can you please confirm that you did all 3 changes:
devcfg.mbn file from SDK 1.0.0voxl-gpio -w 67 1Can you please clarify which second i2c channel was used in your test for the 2nd lepton?
Thanks!
Alex
@Dronodev , this is a bypass capacitor 10uF 10V (0402) and it is used for one of the main power rails (digital power supply VDD). You should try to replace it, since without the capacitor, the camera module would be out of spec. You can use a similar capacitor, even rated down to 6.3V or lower (actual voltage is 1.2V on VDD).
Alex