Zfs receive + lxd import

LXD
,
1

lxc export is not working for me and because it’s production I don’t want to make any changes or restart until I have a backup. I decided to do a zfs send and receive to another machine, but can’t do a import there because the destination storage pool has a different name.

I tried to edit the backup.yaml but it’s read-only. If I used yq it says the file doesn’t exist (see below). How can I make this file readable? I never used nsenter before, and I noticed yq is not a recognised command when I am inside nsenter. How do I make yq visible inside the namespace?

Last attempt I did something like this:

nsenter -t $(cat /var/snap/lxd/common/lxd.pid) -m
mkdir /var/snap/lxd/common/lxd/storage-pools/local/containers/my-container
mount -t zfs my-pool/my-dataset/my-container /var/snap/lxd/common/lxd/storage-pools/local/containers/my-container

I tried without nsenter, but import simply says it doesn’t exist. When I use nsenter to mount the dataset zfs received at least import says the storage name is different (local <> default) so I kept using nsenter.

I also tried to mount inside and outside that nsenter because the folder is empty when I leave. Or mount in a different path when I was outside nsenter. Nothing allowed me to change backup.yaml.

I don’t understand ns well.

This works:

cat /var/snap/lxd/common/lxd/storage-pools/local/containers/my-container/backup.yaml | yq e '.pool.config.source' -

This say file (that has just been read) does not exist:

yq e '.pool.config.source = "my-pool/my-dataset"' /var/snap/lxd/common/lxd/storage-pools/local/containers/my-container/backup.yaml

Can someone help please? Thanks!

Both servers have the latest version (4.20) snap keeps it updated).

2

It turned out export was indeed working. It just took too long without saying anything, even with verbose or debug.

But now I have a problem: the container is larger than the free space, so I think I will have to go back to send send/receive approach.

Does anyone know any way to do a zfs send to another server, then zfs receive, and before running lxc import edit the database.yml because the server I have available has a different name for the pool/fs. I appreciate the commands, please.

3

Did you try lxc export --optimized-storage --compression none?

This would result in LXD doing zfs send into a file which is then wrapped into a backup tarball. You’ll need quite a bit of disk space as that gets generated, but it would let you use the normal lxc import on the target server.

Otherwise with your manual send/receive, you’ll need lxd recover to import it back into LXD on the target. It should handle a lot of those corner cases already but if it doesn’t, editing backup.yaml to have the correct pool name should do the trick. The rest of the pool bits in backup.yaml are only relevant if the pool doesn’t exist.

4

I was using lxc export --debug -v --compression gzip --instance-only <container> /tmp/bkp.tar.gz. Can --optimized-storage be used without having a previous export? Both source and destination are using ZFS but with different settings.

The recover didn’t work for me because of the pool having a different name, I think. But I wasn’t able to edit the backup.yaml on the target. I can see it (with cat, for example), but I wasn’t able to edit it (with yq, as sed would be too complicated for YAML). It says file does not exist. I think it is because of namespaces, not sure what is the proper way to mount for edition and/or mount for use.

5

I am trying this now, watching the disk space. From when I wrote the first message I also set lxc config set storage.backups_volume to a new dataset in the pool. It helped to prepare the tarball, I think, because the error message changed. It failed when the .tar.gz file was being written because it ran out of space (locally).

I see the dataset being growing in size when export starts, but in the end it’s the local disk that grows.

6

I was stupid, I was using /tmp, which is local. I now mounted the pool I created for storage.backups_volume and instead of using /tmp I used the mount, /backups.

lxc export --debug -v --compression gzip --instance-only \
    <container> /backups/<container>.tar.gz

I think this will allow me to at least export the tar to a bucket (backup) and other server (I am migrating from a single server to a LXD cluster).

I know my problem is solved, but if I had to use send/receive (I want to sync production into staging) how I could edit the database.yml in the destination to use the new zpool name? I am using Ubuntu 20.04. Just installed LXC/LXD, ZFS and Apache in it, but clearly I don’t know how to mount or when to mount (ex: after/before nsenter) because I haven’t been able to change database.yml yet, only see it…