ModalAI Forum

How to use the USB Type-C interface on the board's ADB shell as a regular USB interface?

@Alex-Kushleyev

I have checked inference_helper and modify to handle RGB format successfully.

@Alex-Kushleyev

Of course, I have been use cv::VideoCapture in my laptop for testing.

Because I am on a business trip, so I asked a colleague to make modifications in voxl-tflite-server and conduct some tests. However, the results did not match our expectations.

As you mention, this function https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server/-/blob/master/src/inference_helper.cpp?ref_type=heads#L282 should be able to handle RGB format directly. So we just add another condition in switch to prevent the function return false.

We send RGB format to MPA and the video is fine in Portal, but tflite cant handle it properly. The journal message show that the tflite service restart repearly.

The new voxl-tflite-server we built can handle NV12 correctly as the old one, regarding this situation, what methods should be used to provide more detailed information?

We are happy to contribute our successfully tested code. However, we have a limited number of VOXL2 boards available for development, as some are currently in the R&A process. We have already planned to purchase more VOXL2 boards to meet our development needs.

Thanks.

@Alex-Kushleyev

Okay, I would try that.

Also I'm thinking about implement rtsp to mpa in cpp.

Is there some guidance?

Thanks!

@Alex-Kushleyev

The code in voxl-tflite-server inference-helper, it only handle NV12 YUV422 NV21 and RAW8.

But when rtsp got in to opencv frame, original it is BGR format, than I use video/x-raw,format=NV12 videoconvert. So the tflite-server can handle the mpa.

However in tflite-server, the NV12 will be convert to RGB.

Is there something that can make this process more efficient?

Thanks!

@Alex-Kushleyev

Hi, I try to use HW based decoder with HW resize.
I also try SW decoding, cv2.VideoCapture(stream,cv2.CAP_GSTREAMER) doesn't workout.

But in protal, it cannot show anything.

The following is using gi to launch gstreamer and upscale 720 to 1080.

@Alex-Kushleyev

For the original topic, integrate RTSP stream for tflite server.

I found that the problem is the sample code u provide, in metadata, format is RGB.

However, in tflite, https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server/-/blob/master/src/inference_helper.cpp?ref_type=heads#L305 it only handle NV12 NV21 YUV422 RAW8.

Thus, i need to make the RTSP stream NV12, than tflie can take the rest of the job.

! ... ! ... ! videoconvert ! video/x-raw,format=NV12 ! appsink

Are there proper way to handle such format problem? Because NV12 got into tflite, it still need to resize to RGB format.

Thanks!

@Alex-Kushleyev
yes, this is same as i install that package.

@Alex-Kushleyev said in Seeking Reference Code for MPA Integration with RTSP Video Streams for TFLite Server:

@Ethan-Wu , I uploaded the opencv deb here : https://storage.googleapis.com/modalai_public/temp/voxl2-misc-packages/voxl-opencv_4.5.5-3_arm64.deb

it was built using this branch of voxl-opencv project : https://gitlab.com/voxl-public/voxl-sdk/third-party/voxl-opencv/-/tree/add-python3-bindings/

You can just install that deb to overwrite your existing opencv installation and try the python scripts for rtsp natively on voxl2

By the way, how do you get gstreamer to output that debug info? does it automatically print it if there is an issue?

@Alex-Kushleyev I see the add-python3-bindings has new commit.

could u rebuilt the deb? I cant build the package with gstreamer.

Thanks!

@rogerli said in Unable to communicate with add-on board:

@Vinny I have another VOXL2 where only DS1 and DS2 lights up constantly, but D1 doesn't light up. Is this normal? (There was a slight deformation near the power interface due to a previous fall.)

@tom The SDK version of the VOXL2 that I'm using is 1.1.3.

I initially bought two VOXL2 boards, both version 1.1.2. One was deformed due to a crash, but the other still works well. Later, I purchased two more VOXL2 boards, which arrived with version 1.1.3. One of them was completely non-functional, and the other also had communication issues. Due to time constraints, we urgently ordered two more VOXL2 boards. Should we downgrade the SDK to 1.1.2 for safety? Currently, as we are in the pre-mass production stage and sending products for inspection, we cannot afford sudden failures of critical components.

@tom QRB5165M

@tom
The 2nd screenshot is another VOXL2 board.