<![CDATA[Use Hardware Acceleration with TFLite outside the voxl-tflite-server]]>For my given application, it is not practical for my case to be running the voxl-tflite-server all the time. I have a python application that needs to perform a lot of inferences only some of the time. My object detectors are in tflite format and currently run on the CPU. I want to use the NPU on the voxl, but don't know where or how to load its delegates. I thought maybe, just maybe the delegates might be in /usr/lib but I cannot get them to load in python (I get undefined symbol errors).

Can someone point me to where I can find the delegate to load in python to use the NPU (on the VOXL2 does using the GPU really get to the NPU) or tell me what I need to do to compile a delegate to use the NPU?

]]>https://forum.modalai.com/topic/2950/use-hardware-acceleration-with-tflite-outside-the-voxl-tflite-serverRSS for NodeTue, 11 Aug 2026 01:42:44 GMTThu, 14 Dec 2023 21:37:16 GMT60<![CDATA[Reply to Use Hardware Acceleration with TFLite outside the voxl-tflite-server on Tue, 16 Jan 2024 20:43:00 GMT]]>@aharvey

I believe TFLite Delegate logic is available only in C++ per TFLite Sources. Likely you would want to implement your inference code in C++ anyways as on VOXL your inference times will likely be much faster than in Python.

We have some example of assigning delegates inside our voxl-tflite-server source code, check out these lines: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server/-/blob/master/src/inference_helper.cpp#L216

Hope this helps,

Thomas
thomas.patton@modalai.com

]]>https://forum.modalai.com/post/13974https://forum.modalai.com/post/13974Tue, 16 Jan 2024 20:43:00 GMT