Hi,
After extensive testing, it is a docker related issue - mainly to do with docker’s storage drivers.
When installed for the first time, it automatically checks which storage driver to use together with the underlying file system. If your pool is btrfs, it will either use it’s btrfs driver together with subvolumes or the overlay2 driver - both work fine if the host file system is btrfs. If it is zfs, it will use the “vfs” storage driver.
But, if you have already run docker once in the container, then the storage driver will not change after a move. Also rsync cannot copy/move these directories used with either docker’s btrfs driver or the overlay2 driver. If you want to copy/move a lxd container that has docker running with a storage driver other than “vfs”, then you need to do the following first before the lxc copy/move operation:
- lxc shell into the container
- stop the docker system (snap: snap stop docker)
- delete the contents of /var/lib/docker/* (snap: /var/snap/docker/common/var-lib-docker/*)
- stop the lxc container
Then the lxc copy/move operation will succeed.
In order to force docker to always use the “vfs” driver, do this:
- stop the docker service
- make sure /var/lib/docker/* is empty
- add the following to the daemon.json config file:
{
“storage-driver”: “vfs”
]
Now docker will always use vfs as the driver no matter which file system is used underneath and you can now copy/move the container without doing any changes to docker beforehand.
BIG NOTE: If you use snapshots, then all the snapshots will also need to be changed unless you use “–instance-only” option during the copy/move operation.
Quinn