Docker daemon fails to start on voxl
-
RE: https://forum.modalai.com/topic/37/docker-daemon-fails-to-start-voxl
I'm also facing the same issue.yocto:/# voxl-docker -i roskinetic-xenial:opencv
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
yocto:/#
yocto:/#
yocto:/# docker images
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
yocto:/#
yocto:/#
yocto:/# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 2.8G 2.2G 581M 79% /
devtmpfs 1.8G 0 1.8G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 17M 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
tmpfs 1.9G 4.0K 1.9G 1% /var/volatile
/dev/mmcblk0p1 7.4G 204M 7.2G 3% /mnt/sdcard
/dev/sda2 2.2M 80K 2.0M 4% /persist
/dev/sde12 12M 4.1M 7.4M 36% /dsp
/dev/sde11 95M 33M 63M 35% /firmware
/dev/sda3 58M 40K 57M 1% /cache
/dev/sda9 15G 13G 2.4G 85% /data
yocto:/#As you can see, /data partition has all the data and docker images.
What can be a solution for this?
-
What do you see after running
voxl-configure-docker.sh?Should be something like:
yocto:/# voxl-configure-docker-support.sh Stopping original docker service Enabling our own services docker-start & docker-prepare starting docker-start.service loading hello-world docker image successfully loaded hello-world starting docker-prepare service done configuring voxl-docker-support -
yocto:/# voxl-configure-docker-support.sh
Stopping original docker service
Enabling our own services docker-start & docker-prepare
starting docker-start.service
loading hello-world docker image
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
yocto:/# -
yocto:/# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2021-01-11 06:31:53 UTC; 1min 53s ago
Docs: http://docs.docker.com
Main PID: 4675 (code=exited, status=2)Jan 11 06:31:53 apq8096 systemd[1]: Started Docker Application Container Engine.
Jan 11 06:31:53 apq8096 docker[4675]: flag provided but not defined: --raw-logs
Jan 11 06:31:53 apq8096 docker[4675]: See '/usr/bin/docker daemon --help'.
Jan 11 06:31:53 apq8096 systemd[1]: docker.service: Main process exited, co...NT
Jan 11 06:31:53 apq8096 systemd[1]: docker.service: Unit entered failed state.
Jan 11 06:31:53 apq8096 systemd[1]: docker.service: Failed with result 'exi...'.
Jan 11 06:31:57 apq8096 systemd[1]: Stopped Docker Application Container Engine.
Jan 11 06:33:06 apq8096 systemd[1]: Stopped Docker Application Container Engine.
Hint: Some lines were ellipsized, use -l to show in full. -
This post is deleted!
-
As I'm working on this issue, I noticed the following.
yocto:/# ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:32 errors:0 dropped:0 overruns:0 frame:0
TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2224 (2.1 KiB) TX bytes:2224 (2.1 KiB)wlan0 Link encap:Ethernet HWaddr :::::
inet addr:192.168..** Bcast:192.168..*** Mask:...
inet6 addr: :::::**/** Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:467 errors:0 dropped:0 overruns:0 frame:0
TX packets:88 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:3000
RX bytes:40293 (39.3 KiB) TX bytes:10317 (10.0 KiB)
error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/containers/json: read unix @->/var/run/docker.sock: read: connection reset by peer
I am start docker using this command: /usr/bin/dockerd --bip=172.30.224.1/21 --ip-masq=false --mtu=1450 --containerd=/run/containerd/containerd.sock when I using docker ps command to list running ...
Stack Overflow (stackoverflow.com)
It's not showing any IP address for docker0.
How to configure the IP address that will listen to docker daemon on voxl-core board? -
Are you following the docs here?
The voxl-docker command is for developing in a Docker on a PC, it should not be run on VOXL.
The tutorial steps to run on target are:
yocto:/# voxl-configure-docker-support.sh yocto:/# docker images yocto:/# docker run hello-worldIf you have run the PC scripts on VOXL, not really sure what would happen
-
Are you following the docs here?
The voxl-docker command is for developing in a Docker on a PC, it should not be run on VOXL.
The tutorial steps to run on target are:
yocto:/# voxl-configure-docker-support.sh yocto:/# docker images yocto:/# docker run hello-worldIf you have run the PC scripts on VOXL, not really sure what would happen
@Chad-Sweet Thanks for the fast response!
yocto:~# voxl-configure-docker-support.sh Stopping original docker service Enabling our own services docker-start & docker-prepare starting docker-start.service loading hello-world docker image Cannot connect to the Docker daemon. Is the docker daemon running on this host? yocto:~# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: http://docs.docker.com May 13 15:46:33 apq8096 systemd[1]: Stopped Docker Application Container Engine. yocto:~# yocto:~# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: http://docs.docker.com May 13 15:46:33 apq8096 systemd[1]: Stopped Docker Application Container Engine. yocto:~# systemctl start docker yocto:~# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled) Active: failed (Result: exit-code) since Thu 2021-05-13 15:48:03 UTC; 2s ago Docs: http://docs.docker.com Process: 3678 ExecStart=/usr/bin/docker daemon -H fd:// --raw-logs (code=exited, status=2) Main PID: 3678 (code=exited, status=2) May 13 15:48:03 apq8096 systemd[1]: Started Docker Application Container Engine. May 13 15:48:03 apq8096 docker[3678]: flag provided but not defined: --raw-logs May 13 15:48:03 apq8096 docker[3678]: See '/usr/bin/docker daemon --help'. May 13 15:48:03 apq8096 systemd[1]: docker.service: Main process exited, code=exited, status=2/INVALIDARGUMENT May 13 15:48:03 apq8096 systemd[1]: docker.service: Unit entered failed state. May 13 15:48:03 apq8096 systemd[1]: docker.service: Failed with result 'exit-code'. yocto:~# docker images An error occurred trying to connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.21/images/json: read unix @->/var/run/docker.sock: read: connection reset by peer -
Not really sure what state your system is in. This github issue shows it could be related to disk space or needs a reboot. Did you install other Dockers on the image? They are very large and maybe all of the space is consumed
https://github.com/docker/for-linux/issues/435 -
Just looking at your script, I realized that modal has it's own docker services.. so perhaps this provides a little more info: I have created a few images from the kinetic image
yocto:~# systemctl disable docker.service yocto:~# systemctl enable docker-start.service yocto:~# systemctl enable docker-prepare.service yocto:~# systemctl status docker-prepare.service ● docker-prepare.service Loaded: loaded (/etc/systemd/system/docker-prepare.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Thu 1970-01-01 00:00:24 UTC; 51 years 4 months ago Main PID: 2352 (code=exited, status=1/FAILURE) Jan 01 00:00:20 apq8096 docker-prepare.sh[2352]: /usr/bin/docker-prepare.sh: line 17: /sys/fs/cgroup/cpuset/syst...ctory Jan 01 00:00:21 apq8096 docker-prepare.sh[2352]: /usr/bin/docker-prepare.sh: line 17: /sys/fs/cgroup/cpuset/syst...ctory Jan 01 00:00:22 apq8096 docker-prepare.sh[2352]: /usr/bin/docker-prepare.sh: line 17: /sys/fs/cgroup/cpuset/syst...ctory Jan 01 00:00:23 apq8096 docker-prepare.sh[2352]: /usr/bin/docker-prepare.sh: line 17: /sys/fs/cgroup/cpuset/syst...ctory Jan 01 00:00:24 apq8096 docker-prepare.sh[2352]: /usr/bin/docker-prepare.sh: line 17: /sys/fs/cgroup/cpuset/syst...ctory Jan 01 00:00:24 apq8096 docker-prepare.sh[2352]: failed to see cpuset appear after 15 seconds Jan 01 00:00:24 apq8096 systemd[1]: docker-prepare.service: Main process exited, code=exited, status=1/FAILURE Jan 01 00:00:24 apq8096 systemd[1]: Failed to start docker-prepare.service. Jan 01 00:00:24 apq8096 systemd[1]: docker-prepare.service: Unit entered failed state. Jan 01 00:00:24 apq8096 systemd[1]: docker-prepare.service: Failed with result 'exit-code'. Hint: Some lines were ellipsized, use -l to show in full. yocto:~# yocto:~# yocto:~# systemctl status docker-start.service ● docker-start.service Loaded: loaded (/etc/systemd/system/docker-start.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Thu 2021-05-13 15:57:11 UTC; 3min 11s ago Main PID: 3873 (code=exited, status=1/FAILURE) May 13 15:57:11 apq8096 systemd[1]: Started docker-start.service. May 13 15:57:11 apq8096 docker[3873]: time="2021-05-13T15:57:11.453820000Z" level=info msg="API listen on /var/....sock" May 13 15:57:11 apq8096 docker[3873]: time="2021-05-13T15:57:11.477567000Z" level=info msg="[graphdriver] using...lay\"" May 13 15:57:11 apq8096 docker[3873]: time="2021-05-13T15:57:11.527540000Z" level=info msg="Firewalld running: false" May 13 15:57:11 apq8096 docker[3873]: time="2021-05-13T15:57:11.595538000Z" level=fatal msg="Error starting dae...oints" May 13 15:57:11 apq8096 systemd[1]: docker-start.service: Main process exited, code=exited, status=1/FAILURE May 13 15:57:11 apq8096 systemd[1]: docker-start.service: Unit entered failed state. May 13 15:57:11 apq8096 systemd[1]: docker-start.service: Failed with result 'exit-code'. Hint: Some lines were ellipsized, use -l to show in full. yocto:~# yocto:~# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/root 2837860 1890496 930980 68% / devtmpfs 1861956 0 1861956 0% /dev tmpfs 1930660 0 1930660 0% /dev/shm tmpfs 1930660 17136 1913524 1% /run tmpfs 1930660 0 1930660 0% /sys/fs/cgroup tmpfs 1930660 4 1930656 1% /var/volatile /dev/sda2 2168 80 1964 4% /persist /dev/sde12 12016 4184 7508 36% /dsp /dev/sda3 58888 40 57568 1% /cache /dev/sde11 97232 33616 63616 35% /firmware /dev/sda9 15248568 7060916 8171268 47% /data tmpfs 386132 0 386132 0% /run/user/0 -
I had a battery go pretty low and the system crashed while running mavros in docker last night, after reboot, docker deamon is not responding and I cannot see any images.
-
@kiprock I would reflash the system, setup your IP connection, configure Docker and just verify that it works as expected.
-
just looking at the logs, the docker deamon fails here:
level=fatal msg="Error starting daemon: Error initializing network controller: could not delete the default bridge network: network bridge has active endpoints"Most forums suggest removing the docker network folder: rm -r /var/lib/docker/network
I did this but still having issues. Just curious if the modalAI setup stores docker files somewhere else?
-
@kiprock Everything is stored on /data. But again, I would reflash and start over just to verify Docker operation.
-
@Chad-Sweet Thanks for the fast response!
yocto:~# voxl-configure-docker-support.sh Stopping original docker service Enabling our own services docker-start & docker-prepare starting docker-start.service loading hello-world docker image Cannot connect to the Docker daemon. Is the docker daemon running on this host? yocto:~# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: http://docs.docker.com May 13 15:46:33 apq8096 systemd[1]: Stopped Docker Application Container Engine. yocto:~# yocto:~# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: http://docs.docker.com May 13 15:46:33 apq8096 systemd[1]: Stopped Docker Application Container Engine. yocto:~# systemctl start docker yocto:~# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled) Active: failed (Result: exit-code) since Thu 2021-05-13 15:48:03 UTC; 2s ago Docs: http://docs.docker.com Process: 3678 ExecStart=/usr/bin/docker daemon -H fd:// --raw-logs (code=exited, status=2) Main PID: 3678 (code=exited, status=2) May 13 15:48:03 apq8096 systemd[1]: Started Docker Application Container Engine. May 13 15:48:03 apq8096 docker[3678]: flag provided but not defined: --raw-logs May 13 15:48:03 apq8096 docker[3678]: See '/usr/bin/docker daemon --help'. May 13 15:48:03 apq8096 systemd[1]: docker.service: Main process exited, code=exited, status=2/INVALIDARGUMENT May 13 15:48:03 apq8096 systemd[1]: docker.service: Unit entered failed state. May 13 15:48:03 apq8096 systemd[1]: docker.service: Failed with result 'exit-code'. yocto:~# docker images An error occurred trying to connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.21/images/json: read unix @->/var/run/docker.sock: read: connection reset by peer -
@Eric-Katzfey if you cannot start docker images, you can try the command above to clear them. Good luck.
-
@Eric-Katzfey if you cannot start docker images, you can try the command above to clear them. Good luck.
Thanks, but I think you meant to ping @shlee853
-
@Eric-Katzfey if you cannot start docker images, you can try the command above to clear them. Good luck.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login