Pytorch/tflite models

  • Hi, I've been able to run custom mobilenet tflite models via voxl-tflite-server, but when trying to run a Pytorch Yolov5s model ported to tflite, I get the error:

    Resolved reporter
    ERROR: Didn't find op for builtin opcode 'RESIZE_NEAREST_NEIGHBOR' version '3'

    ERROR: Registration failed.

    Failed to construct interpreter

    It seems like this op is only supported for newer version of TF. Will there be support for this in voxl-tflite-server soon? I noticed the docs say it needs TF<=2.2.3 for now, but I was wondering if this was going to be updated.

    I think the conversion produces an ONNX model as an intermediate step- is there a way to run an ONNX model on Voxl1? Alternately, is there a way to run Pytorch models with hardware acceleration? I've tried running CPU models on Pytorch via docker, but keep encountering issues or the board crashing.

  • Dev Team

    Hey @sarahl,

    On voxl, we are limited to TF v2.2.3 due to the need for a newer glibc/gcc toolchain. This is not likely to change, but you could use the docker strategy with a current TF version and newer gcc to get around this.

    We have not done any testing with Pytorch, but for ONNX support you can look into the Qualcomm Neural Processing SDK, as this supports Tensorflow, Caffe, and ONNX models.

  • Hi @Matt-Turi thanks for the information! On M0054/rb5, are you still also limited to tf v2.2.3? Also, can the same GPU/hardware acceleration used in the voxl-tflite-server be done at the docker level? Same question also regarding using the qualcomm neural processing sdk?

  • Dev Team

    On the qrb5165/M0054 platforms, we are using tensorflow v2.8.0.

    For hardware acceleration within a docker, this would require exposing the gpu/other accelerator drivers to the running docker, which I have not done before. However, the qualcomm neural processing sdk will have direct access to hardware accelerators if setup correctly.

Log in to reply