ModalAI Forum

I tried cross-compiling latest tflite-server on voxl-corss:V4.4 and got the same error despite adding /usr/include/ in the src/Cmakelist.txt

Hey ModalAI,

Your documentation on custom inference using voxl-tflite-server is quite outdated. Yolo Ultralytics has stopped exporting frozen graph model format to tflite fp16 quantization. Ultralytics says there's no quantize=16 option — an FP32 .tflite model automatically runs in FP16 at runtime when using a GPU delegate (WebGPU/OpenCL/Metal), so a separate FP16 export file isn't needed. That's why half=True/quantize=16 throws the assertion.

Can voxl-tflite-server automatically execute FP32 models internally in FP16?
Hardware: VOXL2
Source code: Latest voxl-tflite-server code on modalai gitlab page.

My main goal is to test the voxl-tflite-server running on my custom trained object detection model. I want to replicate the exact real world scenario where the tflite server will be taking inputs from the voxl-camera-server's MPA pipe. Once way I found is via voxl-replay where I just replay a previously recorded log file and it publishes the PNG images to /run/mpa/cam0/ and my voxl-tflite-server will ingest it.

Now, I want to publish a set of images to the camera's mpa pipe similar to how the voxl-replay does. But these set of images were not captured using the voxl-logger so I am basically copying these files into a log directory to make it appear as if it was captured using the voxl-logger. However, the voxl-replay keeps giving error "failed to read png at /dummy_log/run/mpa/cam0/00001.png". It gives such error for all the PNGs except the 00000.png. I am sure that the files are present and I also did $ chown $USER:$USER -R . to ensure there is no permissions issue.

I was thinking about looking through the voxl-replay's source code to see what exactly I need to modify in order to make this dummy_log work seamlessly. Please share the voxl-replay's source code or give some alternate way to make this work.

I'm having the same issue trying to compile the latest tflite-server from gitlab. I haven't even changed any code and it still did not build for qrb5165. It has multiple missing files such as modal_json.h, modal_pipe.h, c_api_internal.h, delegate.h, etc.

I copied and pasted the json and pipe headers from the libmodal repositories from voxl SDK/Code Libs on Gitlab and that fixed the first two. But still got the file not found error.

@tom said in Where to see fprintf messages coming from voxl-tflite-server?:

systemctl stop voxl-tflite-server

Still I can't see the outputs of fprintf(stderr,...)

I am accessing voxl2 via WSL2 Ubuntu on Windows 11 through adb shell. I want to debug the voxl-tflite-server line by line similar to the way one can debug software on visual studio by putting breakpoints and checking the variable values.

Is there a way to do so while executing the package on voxl? I want to debug voxl-tflite-server while taking input camera frames from the high res camera.

I have trained a custom object detection model for VOXL2 and already deployed it using voxl-cross. To test it, I do voxl-configure-tflite and select my custom model to work with high-res camera.

My custom model is based on mobilenetv2, trained using tflite mediapipe library exactly the way shown here: https://colab.research.google.com/github/googlesamples/mediapipe/blob/main/examples/customization/object_detector.ipynb#scrollTo=Mz3-eHe07FEX

The problem is that my model loads up and shows on the VOXL portal under tflite on multi-view page, but does not draw any bounding boxes around the objects. I'm not sure what's wrong and trying to debug it going through entire tflite source code. I wanted to see the outputs of fprintf(stderr,...) in the source code but not sure how to access that. Any suggestions?

Thanks

I have hires cam connected and configured to group 1 on VOXL2. I can see the output on portal. However, when I configure tflite server, I do not see any output on the portal. The source for "tflite" says "unknown" instead of "voxl-tflite-server". Everything works fine if I configure the tflite server to run detection on M0014 tracking camera. Only the M0025-2 is not working with voxl tflite server.