Docker not working correctly on VOXL

[memeliner787]

Hi,

I have a VOXL Flight Deck on my hands but am unfortunately not able to get Docker containers running on it.
First of all, docker-autorun.service does not start:

# systemctl status docker-autorun.service -l
● docker-autorun.service - docker autorun service for VOXL
   Loaded: loaded (/etc/systemd/system/docker-autorun.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 1970-01-01 00:12:04 UTC; 53 years 11 months ago
  Process: 2168 ExecStartPre=/usr/bin/docker-prepare.sh (code=exited, status=1/FAILURE)

Jan 01 00:11:42 apq8096 systemd[1]: Starting docker autorun service for VOXL...
Jan 01 00:11:45 apq8096 docker-prepare.sh[2168]: preparing docker with docker-prepare.sh
Jan 01 00:11:45 apq8096 docker-prepare.sh[2168]: this may take a few seconds
Jan 01 00:12:04 apq8096 docker-prepare.sh[2168]: docker-prepare: failed to see cpuset appear after 15 seconds
Jan 01 00:12:04 apq8096 systemd[1]: docker-autorun.service: Control process exited, code=exited status=1
Jan 01 00:12:04 apq8096 systemd[1]: Failed to start docker autorun service for VOXL.
Jan 01 00:12:04 apq8096 systemd[1]: docker-autorun.service: Unit entered failed state.
Jan 01 00:12:04 apq8096 systemd[1]: docker-autorun.service: Failed with result 'exit-code'.

Attempting to start it manually doesn't work either:

/ # sudo systemctl start docker-autorun.service
Job for docker-autorun.service failed because the control process exited with error code. See "systemctl status docker-autorun.service" and "journalctl -xe" for details.

I can start the docker daemon manually as follows:

sudo systemctl start docker-daemon
sudo systemctl status docker-daemon -l

● docker-daemon.service - docker service for VOXL
sudo systemctl status docker-daemon -l
   Loaded: loaded (/usr/bin/docker; enabled; vendor preset: enabled)
   Active: active (start-post) since Sun 2023-12-31 12:46:20 UTC; 12s ago
 Main PID: 4385 (docker);         : 4386 (docker-prepare.)
   CGroup: /system.slice/docker-daemon.service
           ├─4385 /usr/bin/docker daemon -g /data
           └─control
             ├─4386 /bin/bash /usr/bin/docker-prepare.sh
             └─4456 sleep 1

but this fails shortly with:

Dec 31 12:47:03 apq8096 docker-prepare.sh[4386]: docker-prepare: failed to see cpuset appear after 15 seconds
Dec 31 12:47:03 apq8096 systemd[1]: docker-daemon.service: Control process exited, code=exited status=1
Dec 31 12:47:03 apq8096 docker[4385]: time="2023-12-31T12:47:03.642284000Z" level=info msg="Processing signal 'terminated'"
Dec 31 12:47:03 apq8096 systemd[1]: docker-daemon.service: Unit entered failed state.
Dec 31 12:47:03 apq8096 systemd[1]: docker-daemon.service: Failed with result 'exit-code'.

I've followed this thread: https://forum.modalai.com/topic/86/docker-daemon-fails-to-start-on-voxl/40 and tried reconfiguring Docker via voxl-configure-docker-support, and also removing /data/network/files. When rerunning voxl-configure-docker-support it always fails when attempting to enable docker-autorun:

detected hello-world docker image already loaded
you can test by running:
voxl-docker -n -i hello-image -e ""
--------------------------------------------------
--------------------------------------------------
enabling docker-autorun
--------------------------------------------------
Job for docker-autorun.service failed because the control process exited with error code. See "systemctl status docker-autorun.service" and "journalctl -xe" for details.

Additionally, when running the voxl-configure-docker-support script I appear to be missing the line detected APQ8096 as shown by the tutorial here: https://docs.modalai.com/docker-on-voxl/

The output for voxl-version is as follows:

 voxl-version
--------------------------------------------------------------------------------
system-image: 4.0.0
kernel:       #1 SMP PREEMPT Thu Oct 13 17:57:05 UTC 2022 3.18.71-perf
--------------------------------------------------------------------------------
hw version:   VOXL
--------------------------------------------------------------------------------
voxl-suite:   1.1.0
--------------------------------------------------------------------------------
Packages:
Repo: http://voxl-packages.modalai.com/dists/apq8096/stable/binary-arm64/
Last Updated: Never
List:
	apq8096-dfs-server          0.3.1
	apq8096-imu-server          1.0.3
	apq8096-libpng              1.6.38-1
	apq8096-rangefinder-server  0.1.3
	apq8096-system-tweaks       0.2.3
	apq8096-tflite              2.8.3-1
	libapq8096-io               0.6.0
	libmodal-cv                 0.4.0
	libmodal-exposure           0.1.0
	libmodal-journal            0.2.2
	libmodal-json               0.4.3
	libmodal-pipe               2.9.2
	libvoxl-cci-direct          0.2.1
	libvoxl-cutils              0.1.1
	voxl-camera-calibration     0.5.1
	voxl-camera-server          1.8.7
	voxl-configurator           0.4.4
	voxl-cpu-monitor            0.4.7
	voxl-docker-support         1.3.0
	voxl-gphoto2-server         0.0.10
	voxl-jpeg-turbo             2.1.3-5
	voxl-lepton-server          1.2.0
	voxl-libgphoto2             0.0.4
	voxl-libuvc                 1.0.7
	voxl-logger                 0.3.5
	voxl-mavcam-manager         0.5.3
	voxl-mavlink                0.1.1
	voxl-mavlink-server         1.3.1
	voxl-modem                  1.0.8
	voxl-mongoose               7.7.0-1
	voxl-mpa-to-ros             0.3.7
	voxl-mpa-tools              1.1.2
	voxl-opencv                 4.5.5-2
	voxl-portal                 0.6.2
	voxl-qvio-server            0.9.4
	voxl-remote-id              0.0.9
	voxl-streamer               0.7.4
	voxl-suite                  1.1.0
	voxl-system-image           4.0-r0
	voxl-tag-detector           0.0.4
	voxl-tflite-server          0.3.1
	voxl-utils                  1.3.3
	voxl-uvc-server             0.1.6
	voxl-vision-hub             1.7.1
	voxl-vpn                    0.0.6
--------------------------------------------------------------------------------

Any further troubleshooting tips will be gladly appreciated, thanks.

[memeliner787]

Hi, any updates on this? Unfortunately the lack of Docker/ROS functionality blocks the rest of our project unless there's a viable alternative.

[tom]

@memeliner787 I was able to get past the issue by modifying /usr/bin/docker-prepare.sh and adding mkdir -p /sys/fs/cgroup/cpuset/system.slice/ before the for loop.

I also then connected to wifi to get internet access for one boot (unsure if this had an effect or not)

Next I pushed roskinetic-xenial_v1_0.tgz to target and installed it with docker load -i roskinetic-xenial_v1_0.tgz

Then I was able to do the following:

voxl:/$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED                  VIRTUAL SIZE
roskinetic-xenial   v1.0                0e12749bdc00        Less than a second ago   1.633 GB
hello-image         latest              efc161607398        Less than a second ago   2.088 kB
voxl:/$ voxl-docker -i roskinetic-xenial:v1.0     
roskinetic:~$ ls
ros_catkin_ws  yoctohome
roskinetic:~$

Unsure exactly what caused this regression but if this fixes your issue then we can mainline the fix

[memeliner787]

@tom Thanks - that worked.