cannot get model conversion script to work

Samuel Lehman

Hello, I have been trying to get my model converted using the script on the deep learning page of the docs but have been unsuccessful in doing so. Here is the code I have been using:

# IF ON VOXL 1, MAKE SURE TF VERSION IS <= 2.2.3
# i.e., pip install tensorflow==2.2.3
import tensorflow as tf

# if you have a saved model and not a frozen graph, see: 
# converter = tf.compat.v1.lite.TFLiteConverter.from_saved_model()

tf.compat.v1.enable_control_flow_v2()

# INPUT_ARRAYS, INPUT_SHAPES, and OUTPUT_ARRAYS may vary per model
# please check these by opening up your frozen graph/saved model in a tool like netron
converter =  tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(
  graph_def_file = '/home/sam/Desktop/model_to_convert/frozen_inference_graph.pb', 
  input_arrays = ['image_tensor'],
  input_shapes={'image_tensor': [1,300,300,3]},
  output_arrays = ['detection_boxes','detection_scores','num_detections','detection_classes'] 
)

# IMPORTANT: FLAGS MUST BE SET BELOW #
converter.use_experimental_new_converter = True
converter.allow_custom_ops = True
converter.target_spec.supported_types = [tf.float16]

tflite_model = converter.convert()
with tf.io.gfile.GFile('model_converted.tflite', 'wb') as f:
  f.write(tflite_model)

The errors I receive are:

2025-03-20 09:57:55.030795: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-03-20 09:57:55.038034: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1742489875.046979   73285 cuda_dnn.cc:8579] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1742489875.049574   73285 cuda_blas.cc:1407] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
W0000 00:00:1742489875.056635   73285 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1742489875.056657   73285 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1742489875.056658   73285 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1742489875.056660   73285 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
2025-03-20 09:57:55.059151: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-03-20 09:57:56.025478: E external/local_xla/xla/stream_executor/cuda/cuda_platform.cc:51] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: UNKNOWN ERROR (303)
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
W0000 00:00:1742489876.400603   73285 tf_tfl_flatbuffer_helpers.cc:365] Ignored output_format.
W0000 00:00:1742489876.400628   73285 tf_tfl_flatbuffer_helpers.cc:368] Ignored drop_control_dependency.
Traceback (most recent call last):
  File "/home/sam/Desktop/model_to_convert/convert.py", line 24, in <module>
    tflite_model = converter.convert()
                   ^^^^^^^^^^^^^^^^^^^
  File "/home/sam/miniconda3/lib/python3.12/site-packages/tensorflow/lite/python/lite.py", line 3385, in convert
    return super(TFLiteConverter, self).convert()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sam/miniconda3/lib/python3.12/site-packages/tensorflow/lite/python/lite.py", line 1250, in wrapper
    return self._convert_and_export_metrics(convert_func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sam/miniconda3/lib/python3.12/site-packages/tensorflow/lite/python/lite.py", line 1202, in _convert_and_export_metrics
    result = convert_func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sam/miniconda3/lib/python3.12/site-packages/tensorflow/lite/python/lite.py", line 3009, in convert
    return super(TFLiteFrozenGraphConverter, self).convert()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sam/miniconda3/lib/python3.12/site-packages/tensorflow/lite/python/lite.py", line 2609, in convert
    result = _convert_graphdef(
             ^^^^^^^^^^^^^^^^^^
  File "/home/sam/miniconda3/lib/python3.12/site-packages/tensorflow/lite/python/convert_phase.py", line 212, in wrapper
    raise converter_error from None  # Re-throws the exception.
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sam/miniconda3/lib/python3.12/site-packages/tensorflow/lite/python/convert_phase.py", line 205, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/sam/miniconda3/lib/python3.12/site-packages/tensorflow/lite/python/convert.py", line 885, in convert_graphdef
    data = convert(
           ^^^^^^^^
  File "/home/sam/miniconda3/lib/python3.12/site-packages/tensorflow/lite/python/convert.py", line 350, in convert
    raise converter_error
tensorflow.lite.python.convert_phase.ConverterError: Merge of two inputs that differ on more than one predicate {s(Postprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/Greater:0,else), s(Postprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/pred_id/_61__cf__64:0,then), s(Postprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/Greater/_60__cf__63:0,then)} and {s(Postprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/Greater:0,else), s(Postprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/pred_id/_61__cf__64:0,else), s(Postprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/Greater/_60__cf__63:0,else)}
        for node {{node Postprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/Merge}}
        Failed to functionalize Control Flow V1 ops. Consider using Control Flow V2 ops instead. See https://www.tensorflow.org/api_docs/python/tf/compat/v1/enable_control_flow_v2.

I am using tensorflow 2.19.0 and python version 3.12.4. My model is mobilenetv1 ssd that i trained on a custom dataset a couple of years ago.

If for some reason it isnt possible, or if it would just be easier, if anyone knows of a good jupyter notebook for training a new mobilenetv1 ssd model on a custom data set, i would appreciate a link to it.

aditya24

@Samuel-Lehman This is an issue with TensorFlow and your model. You could try enabling the v2 control flow ops as mentioned here (https://www.tensorflow.org/api_docs/python/tf/compat/v1/enable_control_flow_v2). Your model likely contains some older ops that are not compatible with Tflite (you mentioned having trained it a few years back). You could try the official solution given by tensorflow as linked or you could perhaps retrain your model with newer libraries.

Note that you do not necessarily need to train with TensorFlow; you can train using any other framework as well. Moreover, you are converting a frozen graph, and the official docs recommend converting from a saved model (https://ai.google.dev/edge/litert/models/convert_tf). Hope this resolves your issues!

Samuel Lehman

@aditya24 thanks for the reply, I retrained my model and converted it from a saved model to a tfite however using it in the tflite server gives me the following error

INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
ERROR: Encountered unresolved custom op: TensorArrayV3.
See instructions: https://www.tensorflow.org/lite/guide/ops_custom
ERROR: Node number 3 (TensorArrayV3) failed to prepare.
ERROR: Encountered unresolved custom op: TensorArrayV3.
See instructions: https://www.tensorflow.org/lite/guide/ops_custom
ERROR: Node number 3 (TensorArrayV3) failed to prepare.
Failed to allocate tensors!

This is probably due to how I trained it but I am no machine learning expert so I have very little clue as to how to fix it and most other guides on how to train a new model specifically for mobilenet v1 seems to be broken. If you have any resources on how to train mobilenet v1 ssd that work I would love to seem them.

aditya24

This post is deleted!

aditya24

@Samuel-Lehman I think the issue is with the tflite version running on voxl, which does not support v3 ops. An update to tflite on the voxl is being pushed but it will take some time to be merged into the main SDK. However, you can try tensorflow 2.17.1 from this branch.

Note that since it is not completely integrable, running it is slightly more complex. Make sure you are using the newest voxl-cross image to build voxl-tflite-server.

On Voxl, it cannot run directly, and you would have to run it via a Docker container. You can find a sample Dockerfile provided in docker_samples/ in the 'update_tf' branch. For further context on integrating opencl please refer to this.

Hopefully, the updated tflite will contain the necessary v3 op and it works smoothly.