Thanks @thomas, I had to edit a few of the files in the voxl-tflite-server repo, but now have it working nicely. This was necessary to ensure that once packaged and deployed the model-specific services each had different names, so didn't overwrite one another on installation. I now have a set of voxl-tflite-server-[model] services that work in parallel (at the cost of GPU heat!).
Nick
Hi ModalAI Team,
Can you provide guidance on running multiple TFLite models onboard the QRB5165 board (voxl-suite 1.1.3-1, tflite 2.8.0-2)? I've read the SDK documentation but the following is unclear to me:
1. Do you need to configure and run multiple instances of voxl-tflite-server?
I've tried this by adding additional service definitions into /etc/systemd/system (e.g., one called voxl-tflite-server-pose.service) and running via systemctl. Does each instance need to be pointed at a discrete config file (e.g., /etc/modalai/voxl-tflite-server-pose.conf)?
On testing this approach I was able to run several parallel instances of voxl-tflite-server, but they all seem to reference the default config file (/etc/modalai/voxl-tflite-server.conf), where I can only define a single model. I'm unable to specify a config file in the systemd.service file as the -c option seems to be reserved for config (i.e., it doesn't actually run the model).
2. Do you use a single config file and voxl-tflite-server service to run multiple models?
Given that the voxl-tflite-server service seems locked to the voxl-tflite-server.conf file, do you instead specify multiple models in the config file (setting allow_multiple and output_pipe_prefix accordingly) and let the service manage system resources etc? If this is the case, have you got an example of the syntax? (I tried adding a second model definition to the conf but it didn't work).
3. Am I missing something like the need to manually publish another pipe from the additional servers?
Thanks,
Nick
