My first time testing out LXD, and I decided to take the “simplest” of tasks and get a Kubernetes node running inside it.
I’m having trouble installing Docker properly. The strangest thing is that I managed to install it well a few days back, but now with a separate setup I get all the way up to apt-get install
which runs through as such:
$ lxc exec mylxd bash
root@mylxd:~$ apt-get install docker-ce
Reading package lists... Done
... snip ...
The following NEW packages will be installed:
aufs-tools cgroupfs-mount containerd.io docker-ce docker-ce-cli libltdl7 pigz
0 upgraded, 7 newly installed, 0 to remove and 4 not upgraded.
Need to get 85.8 MB of archives.
After this operation, 385 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://download.docker.com/linux/ubuntu bionic/stable amd64 containerd.io amd64 1.2.13-1 [20.1 MB]
Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 pigz amd64 2.4-1 [57.4 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 aufs-tools amd64
... snip ...
Setting up docker-ce-cli (5:19.03.8~3-0~ubuntu-bionic) ...
Setting up pigz (2.4-1) ...
Setting up docker-ce (5:19.03.8~3-0~ubuntu-bionic) ...
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /lib/systemd/system/docker.socket.
And that’s it. After that last symlink it just freezes, my CPU keeps jumping up and down, and my hard drive is getting filled with something at 250-500MB/sec, until I run out of space. The data is being written to the container storage pool, but I can’t fathom where there it could be or why. My PC gets quite sluggish when this problem appears, so I can’t do any proper debugging or monitoring otherwise.
The only thing that works after this is lxc stop --force <container>
. lxc delete <container>
frees up all the disk space the error condition stole from me.
I’m using the 4.*
snap version of LXD. Here’s how I launch my container:
$ lxc launch ubuntu:18.04 mylxd -s mypool
$ lxc config set mylxd limits.cpu 4
$ lxc config set mylxd limits.memory 3GB
$ lxc config set mylxd security.privileged true
$ lxc config set mylxd security.nesting true
$ lxc config set mylxd linux.kernel_modules 'xt_conntrack,ip_tables,ip6_tables,netlink_diag,nf_nat,overlay'
$ lxc config set mylxd raw.lxc 'lxc.apparmor.profile=unconfined\nlxc.cap.drop= \nlxc.cgroup.devices.allow=a\nlxc.mount.auto=proc:rw sys:rw'
The mypool
is a basic dir
storage pool. Once the container is running, I exec into it and attempt to install docker.
Need help with the following:
- If someone has a hunch on what is going wrong would be neat to know!
- Where can I start looking for information on what is filling my disk when this occurs? Some log getting flooded perhaps?
- Where can I find proper logs that might show what is happening, instead of just seeing a frozen terminal? The logs at
/var/snap/lxd/common/...
are quite empty. - I need pointers on how to help you help me, not sure what I could provide to make this easier to debug and solve?
Some googling showed that sometimes systemd-resolved goes into a loop in certain situations, could that relate to this as Docker presumably does something to the network configs?