Voxl-streamer rotation

hmlow

Hi devs

The rotation does not work if the video source is already h264/h265 encoded?

Eric Katzfey

@hmlow That is correct. When encoded data is coming in to voxl-streamer it pretty much just acts as an RTSP server and doesn't change the video data in any way.

hmlow

@Eric-Katzfey

Can the rotation be done in voxl-camera-server?

I noticed there is a "flip" parameter here:
https://docs.modalai.com/voxl-camera-server/

hmlow

Are there any plans to make it rotatable in voxl-streamer or voxl-camera-server?
I upgraded from SDK 0.9.5 to 1.0.0 and now im stuck with an inverted camera unless i make structural/hardware changes

hmlow

I tried building a custom version of the voxl-camera-manager by playing around with the ROTATION_MODE definition (i have no idea what are the possible values, but i tried 0 and 1) within hal3_camera_mgr.cpp but it doesnt seem to do anything.

#elif QRB5165
    **#define ROTATION_MODE  2**
    #define OPERATION_MODE CAMERA3_STREAM_CONFIGURATION_NORMAL_MODE
    #define ENCODER_USAGE  GRALLOC_USAGE_HW_VIDEO_ENCODER
    #define SNAPSHOT_DS    HAL_DATASPACE_V0_JFIF
    #define NUM_STREAM_BUFFERS   16 // shouldn't need more than 10, if the buffer pool is empty then OMX should be dropping more frames
    #define NUM_RECORD_BUFFERS   16 // shouldn't need more than 10, if the buffer pool is empty then OMX should be dropping more frames

;
;

 if(en_small_video) {
        small_vid_stream.stream_type = CAMERA3_STREAM_OUTPUT;
        small_vid_stream.width       = small_video_width;
        small_vid_stream.height      = small_video_height;
        small_vid_stream.format      = vid_halfmt;
        small_vid_stream.data_space  = HAL_DATASPACE_UNKNOWN;
        small_vid_stream.usage       = ENCODER_USAGE;
        **small_vid_stream.rotation    = ROTATION_MODE;**
        small_vid_stream.max_buffers = NUM_STREAM_BUFFERS;
        small_vid_stream.priv        = 0;

        streams.push_back(&small_vid_stream);
        streamConfig.num_streams ++;
        M_VERBOSE("Adding small video stream for camera: %d\n", cameraId);

Is it still possible to rotate in software if the video is OMX hardware encoded now?

hmlow

@Eric-Katzfey

Was searching around for a solution and i stumbled upon this.

https://forum.modalai.com/topic/2043/hires-camera-rotation

I downloaded libmmcamera_imx214_rot180.so and renamed it into com.qti.sensor.imx214_rot180.so but voxl-camera-server wasnt't happy. (i guess it is due to the new implementation on the SDK-1.0 )

Would you have a functional com.qti.sensor.imx214_rot180.so?
That would solve all my problems.

Thanks!

Zachary Lowell 0

@hmlow said in Voxl-streamer rotation:

Would you have a functional com.qti.sensor.imx214_rot180.so?

@hmlow all you need to do is change the pipe you are subscribing to in voxl-streamer. You cannot rotate an encoded stream, but you can to the raw stream. Change the pipe in the voxl-streamer.conf file to point towards hires_small_color and make sure the rotation parameter is filled and the next time you run voxl-streamer, the frame will have been rotated properly.

hmlow

@Zachary-Lowell-0

I am using the encoded stream so this approach does not work for me.

Can the rotation be achieved through the use of the rotated IMX214 driver similar to how it was done here? (@Alex-Kushleyev )
https://forum.modalai.com/topic/2043/hires-camera-rotation

Alex Kushleyev

@hmlow , the post you referenced was for VOXL1, unfortunately the camera drivers are not compatible with VOXL2. VOXL2 camera driver does not (yet) have an option to rotate the image on the camera, which is what was being done in the different IMX214 .so drivers.

Maybe I missed something, but I don't understand why @Zachary-Lowell-0 's suggestion would not work. Do you want to use both : rotated and not rotated images or something like that? If you just want the rotated encoded image, then it seems what Zachary said should work for you.

Alex

hmlow

@Alex-Kushleyev

Is a VOXL2 camera driver with 180deg rotation in the works?
The alternative for us would be physically modify the structure on our fleet of aircrafts - which is not something i am very keen on.

With SDK1.0, voxl-camera-server sends OMX pre-encoded H265 streams to voxl-streamer. The encoding done by voxl-camera-server is way more efficient (keeps the VOXL2 nice and cool) than the encoding done by voxl-streamer in SDK0.9.5. @Zachary-Lowell-0 explained that pre-encoded streams cannot be rotated by voxl-streamer, which i have verified as well.

hmlow

@Alex-Kushleyev appreciate your help with the drivers pls

Alex Kushleyev

@hmlow we are not yet working on a rotated version of the driver, although that could be a good option. I am checking if we can enable rotating in the ISP, which would also be very efficient..

hmlow

@Alex-Kushleyev thank you very much for the help.
looking forward

hmlow

@Alex-Kushleyev any luck with this?

Alex Kushleyev

@hmlow , one of our developers is out until Monday, so I will discuss then and get back to you. Sorry about the delay! @modaltb

hmlow

@Alex-Kushleyev
any good news?

Alex Kushleyev

@hmlow , we built and tested the rotated camera driver, just need to see how quickly we can get it to you.. hang on!

@hmlow Please use the following procedure to enable driver-level image rotation for the imx214:

Download the file com.qti.sensor.imx214_flip.so and place it in /usr/lib/camera on VOXL2
Download the file com.qti.sensormodule.imx214_flip_3.bin and place it in /usr/lib/camera on VOXL2
Remove the file /usr/lib/camera/com.qti.sensormodule.imx214_3.bin on VOXL2. If this file does not exist on VOXL2, please let me know -- it will require a different procedure.

hmlow

@Ariel-Young it works! thank you very much!

@hmlow Glad it worked! For future reference, the imx214_flip driver should be available internally from SDK 1.1 (system image 1.7) onwards. Rather than downloading from the above links, after SDK 1.1 is released the procedure will be as follows:

rm /usr/lib/camera/com.qti.sensormodule.imx214_3.bin
cp /usr/share/modalai/chi-cdk/imx214-flip/com.qti.sensormodule.imx214_flip_3.bin /usr/lib/camera