Migrating legacy overlayfs containers to ZFS-based storage


I was wondering now how I can transfer roots from a legacy container to lxd without losing file system information on users/groups, and possibly, when the container is offline.


As described in a previous issue, I need to transfer containers from an existing legacy lxc host (v1.0.7) to a lxd host (v2.21) that uses ZFS for storage. The lxd host is one that has lots of storage attached, I am unfortunately not able to upgrade it to 3.x :man_shrugging::

I copied all data in /var/lib/lxc from the legacy lxc host to a backup directory on the lxd host. Using overlayfs, I was able to mount the complete rootfs for each container.

The python-based lxc-to-lxd fails:

$ sudo lxc-to-lxd --lxcpath /external-disk/lxc-migration/lxc --dry-run --debug my-imported-container
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/lxd/containers/my-imported-container/rootfs'

It seems like the migration script assumes a basic directory-based storage backend.

I have then tried the following approach:

  1. Launching a very basic alpine container via lxd
  2. Finding the directory on the file system where the rootfs is stored (/var/lib/lxd/containers/my-imported-container/rootfs)
  3. Copying the rootfs from legacy container into the new container rootfs:
rsync -rva --progress /external-disk/lxc-migration/my-imported-container/rootfs/ .

There are two issues:

  1. When the newly created container is NOT running, there is no rootfs mounted on the host; it just exists as an unmounted ZFS. I don’t know where to copy the data to.
  2. When copying the rootfs while the container runs, it is possible to restart the container and it will re-boot. Unfortunately, all files are owned by nobody:nogroup and most processes will not work because of that.