Camera orientation parameter

sspiller

Our drone camera configuration changes the ribbon cable positions so they do not all exit in the same orientation as the VOXL FLIGHT DECK camera mounting configuration. Our main camera ribbon cable exits on the bottom (vs. the top), the left depth camera (from the front) ribbon cable exits to the right and the right depth camera exits to the left. Where are the parameters set so we can rotate the images for our orientation on VOXL?

Alex Kushleyev

Hello,

90 degree rotations are not supported in cameras (as opposed to horizontal and vertical flipping). However, it seems that the cameras that have 90 degree rotation in your case are the stereo cameras (what you refer to as depth camera, I believe), so it is possible that you don't need to rotate the images and just specify the appropriate extrinsic parameters (translation and rotation of the stereo cameras with respect to each other) and the Depth From Stereo algorithm should take care of it? We have not tried this. The only thing I can think of as down side of doing this is that the memory access could be a bit slower for matching pixels which are close to the same line (instead of reading consecutive pixels in a row, the algorithm will need to hop rows, but I don't know enough specifics about DFS to say whether in fact it will be slower in this configuration).

Alternatively, to fix the stereo camera rotation, you could rotate the images pixel by pixel in software and form two 480x640 images (instead of 640x480). This will incur some cpu overhead, obviously.

As for the 4K camera, can you please confirm which camera is being used? If it is IMX214, we can do 180 degree rotation right in the camera without any cpu overhead.

sspiller

We are using the standard IMX214 based hires main camera and it is flipped 180 degrees with ribbon exiting from bottom of camera. What is required for you to do the 180 degree rotation in the camera, do I need to return the cameras to you to do this? Regarding the stereo cameras, can you confirm which extrinsic parameters need to be adjusted and where in the code is this done at? Thank you.

Alex Kushleyev

In order to flip orientation of IMX214 camera (actually it's a 180 degree rotation), you can do this by editing the camera configuration file:

edit file /system/etc/camera/camera_config.xml (make sure to create a backup of the config file first)
replace <SensorName>imx214_rot180</SensorName> with <SensorName>imx214</SensorName> - there should be several occurrences.
this will load a different camera driver, which instructs the camera to flip the image before sending to VOXL
actually, the default configuration in camera_config.xml is flipped/rotated for imx214, as you can see, because the way the camera is built, the image would show up flipped if using <SensorName>imx214</SensorName> on standard VOXL Flight Deck

For stereo cameras, I would suggest go ahead and try to follow this procedure for calibrating: https://docs.modalai.com/calibrate-cameras/#calibrate-stereo-cameras . This will calibrate the ov7251 camera intrinsic parameters and relative translation + rotation of the two stereo cameras in the pair. Hopefully this succeeds (like I said we have not tried rotating both stereo cameras 90 degrees, but in principle it should work). After that, you will also need to adjust the rotation of the stereo pair with respect to the drone itself, if you want to put the depth points in the correct frame. However, even before doing that last step, you can still check if stereo is working by looking at the resulting point cloud.

Alex Kushleyev

Also, make sure to reboot VOXL after you change camera_config.xml

afdrus

@Alex-Kushleyev I don't have any camera_config.xml file in the path that you specified, and it does not get created when using voxl-configure-cameras neither. Therefore, how can I change the rotation of the hires/stereos? In voxl-portal they keep being flipped.

Alex Kushleyev

@afdrus , the original thread was related to VOXL1. Is your question about VOXL1 or VOXL2?

Alex

afdrus

@Alex-Kushleyev, sorry, I missed that, my question is about VOXL2

afdrus

@Alex-Kushleyev then, how can I change the rotation of the hires/stereos? In voxl-portal they keep being flipped. Is it only possible to rotate the mounting of the cameras?

Alex Kushleyev

@afdrus please check the bottom of this thread how to rotate the hires (IMX214) camera:

https://forum.modalai.com/topic/2422/voxl-streamer-rotation/20

@modaltb do we also have rotation option for the stereo cameras (are we building rotated ov7251 drivers?)

Alex

modaltb

@afdrus,

We're trying to improve customizability... our approach now is we're shipping various flavors of sensors/variants/HW locations....

TLDR: here's pre-release binaries to try, these go into /usr/lib/camera:

https://storage.googleapis.com/modalai_public/forum/2023-08-18-imx214-flip/com.qti.sensor.imx214_flip.so
https://storage.googleapis.com/modalai_public/forum/2023-08-18-imx214-flip/com.qti.sensormodule.imx214_flip_0.bin
https://storage.googleapis.com/modalai_public/forum/2023-08-18-imx214-flip/com.qti.sensormodule.imx214_flip_2.bin
https://storage.googleapis.com/modalai_public/forum/2023-08-18-imx214-flip/com.qti.sensormodule.imx214_flip_3.bin
https://storage.googleapis.com/modalai_public/forum/2023-08-18-imx214-flip/com.qti.sensormodule.imx214_flip_4.bin

Longer version: we've started docs here: https://docs.modalai.com/voxl2-image-sensors/

Starting in SDK1.0, we began to put the sensor MODULE bin options in /usr/share/modalai/chi-cdk.

These are the binaries that tie the HW location, and also is in charge of pulling in the appropriate sensor SO in /usr/lib/camera

voxl2:/usr/share/modalai/chi-cdk$ ls
imx214  imx214-flip  imx335  imx377  imx412  imx678  irs1645  ov7251  ov7251-combo  ov7251-combo-flip  ov7251-fsin  ov7251-fsout  ov9782  ov9782-combo  ov9782-combo-flip

For example, this is our preview release, there's imx214-flip.

The idea is, you want to find the appropriate bin to use, for example my options:

voxl2:/usr/share/modalai/chi-cdk$ ls imx214-flip/
com.qti.sensormodule.imx214_flip_0.bin  com.qti.sensormodule.imx214_flip_2.bin  com.qti.sensormodule.imx214_flip_3.bin  com.qti.sensormodule.imx214_flip_4.bin

The so that will be used by these bins is /usr/lib/camera/com.qti.sensor.imx214_flip.so.

modaltb

@afdrus , @Alex-Kushleyev ,

Just realizing you asked for STEREO sensors haha, those should be on target already and found under /usr/share/modalai/chi-cdk/ov7251-combo-flip....

afdrus

@modaltb Thanks, despite the delay, for the response! I only have one last question in this regard: is it possible to rotate the stereo ov7251 by 90 degrees or currently it is only possible to flip them by 180?
This info would help me a lot. Thanks!