voxl-tflite-server with USB Camera

Philemon Benner

Hey,
i'm currently trying to setup the voxl-tflite-server with a usb camera connected to the Voxl. Is it possible to use the tflite-server with a usb camera? I didn't find any support on the voxl-camera-server that forwards the camera stream to voxl-tflite-server, for usb.

Hello @Philemon-Benner ,

It is possible to use any MPA image pipe as input for voxl-tflite-server.

For a usb camera, you must be using the latest voxl-uvc-server to get the camera frames from your usb device into our MPA pipes.

Then, you must modify the config file (under /etc/modalai/voxl-tflite-server.conf) such that the input pipe matches the new usb pipe:

voxl:/$ vi /etc/modalai/voxl-tflite-server.conf  
/**
 * This file contains configuration that's specific to voxl-tflite-server.
 *
 * skip_n_frames - how many frames to skip between processed frames. For 30hz
 *                   input frame rate, we recommend skipping 5 frame resulting
 *                   in 5hz model output.
 * model         - which model to use. Currently support mobilenet for
 *                   object detection or midas for monocular depth
 * input_pipe    - which camera to use (tracking or hires).
 */
{
        "skip_n_frames":        0,
        "model":        "/usr/bin/dnn/ssdlite_mobilenet_v2_coco.tflite",
        "input_pipe":   "/run/mpa/USB_PIPE_NAME/",  // CHANGE HERE
        "delegate":     "gpu"
}

The only limitation with the tflite server is supported image formats. In this code block, there is a switch for the supported formats, so if your input data is not in NV12, NV21, or RAW8 format you will have to add handling for your specific type.