Training custom yolov8 model
-
I am following the tutorial below to train a custom yolov8 model with 295 photos of a fuel canister. I'm training the model on a GCP instance with a NVIDIA Tesla P4. Instead of using the full COCO database, I'm only using 0=background and 1=fuel_canister. I am getting flooded with 'should not reach here' error logs when trying to run voxl-tflite-server. I have tried both configs below. Any ideas?
https://gitlab.com/voxl-public/support/voxl-train-yolov8.git
voxl2:~$ journalctl -u voxl-tflite-server -n 20 --no-pager -- Logs begin at Thu 2023-03-02 12:58:02 UTC, end at Tue 2025-05-13 00:18:20 UTC. -- May 13 00:18:15 m0054 bash[10642]: Error in TensorData<float>: should not reach here May 13 00:18:16 m0054 bash[10642]: Error in TensorData<float>: should not reach here May 13 00:18:16 m0054 bash[10642]: Error in TensorData<float>: should not reach here May 13 00:18:16 m0054 bash[10642]: Current pipeline throughput: 4.64115 frames per second1) docker build -t yolov8-train:latest . 2) docker run --gpus all -it --shm-size=8g -v "$(pwd)":/app/ yolov8-train:latest /bin/bash 3) /app python3 train.py 4) /app yolo task=detect mode=val model=/app/runs/detect/fuel_canister_exp12/weights/best.pt data=/app/dataset.yaml split=test name=fuel_canister_test_eval 5) /app yolo export model=./runs/detect/fuel_canister_exp12/weights/best.pt format=tflite imgsz=640 nms=False half=Truecat fuel_canister_labels.txt 0 background 1 fuel_canistercat /etc/modalai/voxl-tflite-server /** * voxl-tflite-server Configuration File * * skip_n_frames - how many frames to skip between processed frames. For 30 Hz * input, skip 5 ^g^r 5 Hz inference. Set 0 for full rate. * model - which model to use. Bundled choices include mobilenet, * fastdepth, posenet, deeplab, yolov5, yolov8. * input_pipe - which camera pipe to read (tracking, hires, stereo, etc.). * delegate - hardware acceleration: gpu, cpu, or nnapi. "gpu" is best * on VOXL 2 for float16 models. * allow_multiple - if true, removes single-instance lock so multiple servers * can run (one per config file). * output_pipe_prefix - prefix added to the default output pipes when * allow_multiple is true. */ { "skip_n_frames": 0, "model": "/etc/modalai/tflite_models/fuel_canister_yolov8_custom.tflite", "input_pipe": "/run/mpa/hires_front_small_color", "delegate": "cpu", "requires_labels": true, "labels": "/etc/modalai/tflite_models/fuel_canister_labels.txt", "allow_multiple": false, "output_pipe_prefix": "fuel_canister", "output_meta_type": "YOLO_V8", "debug_en": false, "confidence_threshold": 0.1 }/** * voxl-tflite-server Configuration File * * skip_n_frames - how many frames to skip between processed frames. For 30 Hz * input, skip 5 ⇒ 5 Hz inference. Set 0 for full rate. * model - which model to use. Bundled choices include mobilenet, * fastdepth, posenet, deeplab, yolov5, yolov8. * input_pipe - which camera pipe to read (tracking, hires, stereo, etc.). * delegate - hardware acceleration: gpu, cpu, or nnapi. "gpu" is best * on VOXL 2 for float16 models. * allow_multiple - if true, removes single-instance lock so multiple servers * can run (one per config file). * output_pipe_prefix - prefix added to the default output pipes when * allow_multiple is true. */ { "skip_n_frames": 0, "model": "/etc/modalai/tflite_models/fuel_canister_yolov8_custom.tflite", "input_pipe": "/run/mpa/hires_front_small_color", "delegate": "cpu", "requires_labels": true, "labels": "/etc/modalai/tflite_models/fuel_canister_labels.txt", "allow_multiple": false, "output_pipe_prefix": "fuel_canister", "confidence_threshold": 0.5 }voxl2:~$ voxl-version ──────────────────────────────────────────────────────────────────────────────── system-image: 1.8.02-M0054-14.1a-perf kernel: #1 SMP PREEMPT Mon Nov 11 22:47:44 UTC 2024 4.19.125 ──────────────────────────────────────────────────────────────────────────────── hw platform: M0054 mach.var: 1.0.1 ──────────────────────────────────────────────────────────────────────────────── voxl-suite: 1.4.3 ──────────────────────────────────────────────────────────────────────────────── Packages: Repo: http://voxl-packages.modalai.com/ ./dists/qrb5165/sdk-1.4/binary-arm64/ Last Updated: 2025-04-17 20:21:35 List: kernel-module-voxl-fsync-mod-4.19.125 1.0-r0 kernel-module-voxl-gpio-mod-4.19.125 1.0-r0 kernel-module-voxl-platform-mod-4.19.125 1.0-r0 libfc-sensor 1.0.7 libmodal-cv 0.5.16 libmodal-exposure 0.1.3 libmodal-journal 0.2.3 libmodal-json 0.4.3 libmodal-pipe 2.10.6 libqrb5165-io 0.4.9 libvoxl-cci-direct 0.2.5 libvoxl-cutils 0.1.1 modalai-slpi 1.1.19 mv-voxl 0.1-r0 qrb5165-bind 0.1-r0 qrb5165-dfs-server 0.2.0 qrb5165-imu-server 1.1.3 qrb5165-rangefinder-server 0.1.5 qrb5165-slpi-test-sig 01-r0 qrb5165-system-tweaks 0.3.5 qrb5165-tflite 2.8.0-2 voxl-bind-spektrum 0.1.1 voxl-camera-calibration 0.5.9 voxl-camera-server 2.1.2 voxl-ceres-solver 2:1.14.0-10 voxl-configurator 1.0.0 voxl-cpu-monitor 0.5.3 voxl-cross-template 0.0.1 voxl-docker-support 1.3.1 voxl-elrs 0.4.2 voxl-esc 1.5.1 voxl-feature-tracker 0.5.2 voxl-flow-server 0.3.6 voxl-fsync-mod 1.0-r0 voxl-gphoto2-server 0.0.10 voxl-gpio-mod 1.0-r0 voxl-io-server 0.0.5 voxl-jpeg-turbo 2.1.3-5 voxl-lepton-server 1.3.3 voxl-lepton-tracker 0.0.4 voxl-libgphoto2 0.0.4 voxl-libuvc 1.0.7 voxl-logger 0.5.0 voxl-mavcam-manager 0.5.8 voxl-mavlink 0.1.4 voxl-mavlink-server 1.4.5 voxl-modem 1.1.5 voxl-mongoose 7.7.0-1 voxl-mpa-to-ros 0.3.9 voxl-mpa-tools 1.3.7 voxl-open-vins 0.4.17 voxl-open-vins-server 0.3.0 voxl-opencv 4.5.5-2 voxl-osd 0.1.3 voxl-platform-mod 1.0-r0 voxl-portal 0.7.9 voxl-px4 1.14.0-2.0.98 voxl-px4-imu-server 0.1.2 voxl-px4-params 0.6.7 voxl-qvio-server 1.1.1 voxl-remote-id 0.0.9 voxl-reset-slpi 0.0.1 voxl-state-estimator 0.0.4 voxl-streamer 0.7.5 voxl-suite 1.4.3 voxl-tag-detector 0.0.4 voxl-tflite-server 0.3.9 voxl-utils 1.4.6 voxl-uvc-server 0.1.7 voxl-vision-hub 1.8.20 voxl-vtx 1.2.2 voxl2-io 0.0.3 voxl2-system-image 1.8.02-r0 voxl2-wlan 1.0-r0
