How to transfer an LXD container from one host to another when the networking is different

LXD
#1

I’ve been running an LXD container on host1 using an lxdbr0 default bridge and a proxy. I would now like to transfer this container to a new machine (host2) which has a NIC-bound br0 bridge so that the container will now appear as a host directly attached to the local network along with some other containers already in place. I transferred the container as follows:

host1$ lxc stop atom
host1$ lxc export atom atom_2023-04-23.tar.gz
- copy  atom_2023-04-23.tar.gz to host2
host2$ lxc import atom_2023-04-23.tar.gz atom

On host1 the container just used the default profile. The default profile on host2 is configured with the br0 bridge. I assumed the import would succeed and that I’d have to fiddle with the profile to get the networking and IP address sorted out, but unfortunately the import failed completely:

archives@www2:~$ lxc import ./atom_2023-05-24_export.tar.gz  atom
Error: Failed importing backup: Failed creating instance record:
Failed initialising instance: Invalid devices: Device validation failed for "eth0":
Failed loading device "eth0": Failed to load network "lxdbr0" for project "default":
Network not found

The documentation on importing/exporting containers is very sparse, so I’m not sure of the best way to proceed. host1 does not have a NIC-bound br0 bridge, so it’s not possible for me to re-engineer the networking on host1 before transferring the container to host2. Presumably I could create a default profile on host2 matching the default profile on host1, import the container and then update the profile after import, but that seems awfully kludgy – surely there’s a better way to do this.

#2

The lxc import command has a -s flag for overriding the storage pool.
But unlike lxc init and lxc launch (which have a -n flag for overriding the network) the lxc import command doesn’t have a -n flag to override the network. Perhaps this is because an instance maybe connected to multiple networks or not use eth0 as the singular NIC name.

Anyway, the way to solve this is to open up the tarball and find the backup.yaml file, in there you should be able to see reference to a NIC device using lxdbr0, you can change this file so that it either references a different network or remove that NIC device entirely.

That should allow you to import it.

1 Like
#3

Some additional context is in order. host1 (the original base for the image) is an Arch Linux system running bare metal LXD. The destination host2 is an Ubuntu 22.04 system running the LXD snap. This means the storage is also changing from /var/lib/lxd/storage-pools/default to /var/snap/lxd/common/lxd/storage-pools/default. I guess this would be addressed with the -s flag.

Some comments about the exported tar.gz file. This untars into a folder simply called backup (that could definitely cause issues for some people) and I was unable to untar it as an lxd-privileged user, as it creates device nodes. I had to do this as the root user.

There are 2 yaml files in the resulting backup folder which include reference to both the previous bridge lxdbr0 and the previous storage pool: ~/backup/index.yaml and ~/backup/container/backup.yaml – what’s the purpose of index.yaml?

Exporting the whole container like this includes the snapshots, and it seems like the snapshots will all have the incorrect bridge and attached storage pool still associated with them, which could confusion down the road. So I ended up abandoning this transfer method in favor of what I’ll detail in the next comment.

#4

This method worked for transferring the container to a new host with a different LXD instantiation. Make a snapshot of the container; publish this snapshot as an image; export this image; and then import the image on the new container host:

$ lxc publish atom/snapshot1 --alias my_atom_image
- The following creates a tar file (xxxxxxx.tar.gz) of the image in the current directory
$ lxc image export my_atom_image

- Copy the xxxxxxx.tar.gz to the new server, then on the new server:
 $ lxc image import xxxxxxx.tar --alias atom-image
 $ lxc init atom-image atom

Since the container on the originating server used the default profile, the new container uses the default profile on the new server, which is configured with the correct bridge and storage pool. The only potentially necessary additional step is removing /etc/netplan/50-cloud-init.yaml and replacing it with a netplan yaml file reflecting the networking you want for the container on the new host. Since I’m assigning static IP addresses to the containers on this host, this was necessary for me, but if you’re using DHCP then the only potentially necessary step is updating the relevant dhcpd.conf file (i.e. in the case of IP reservations).

Added bonus: you don’t need to be root to export/import the container this way.

1 Like