`lxd recover` as a non-interactive command for specific container recoveries

Hi @tomp ,

before the release of the new disaster recovery tool I used the import command
lxd import foo --force
to import containers inside my restore-scripts that sync’ed the data beforehand by rsync into the appropriate directory.
In this restore-process the container name foo might have been changed from the original name with intension. For example to setup a test environment from a rsync backup of a production environment with a different container name.

It would be great to extend lxd recover in the following way:

  • a flag for non-interactive execution
  • with a parameter that refers to the recovery of a specific container
  • and a ‘–force’ parameter that ignores a difference between the container name in metadata.yml and the given container name parameter in the command

Thanks a lot for your work!
Florian

1 Like

Backward compatibility would be nice.

Please can you open an issue here with your ideas, https://github.com/lxc/lxd/issues, thanks.

Note that we’re not very likely to do anything to make lxd recover be used for anything other than its goal (disaster recovery).

Yes, it was possible to script lxd import but we’re yet to see any case where this was done to perform disaster recovery. Those commands rely on pretty dangerous internal APIs and are really really not meant to be run automatically, they should be run when dealing with a disaster situation and then have an administrator re-check all the imported stuff to make sure that nothing broke.

Hi, imagine that you have been using something for many years without any problems and suddenly there is practically no replacement that would allow a quick fix without having to use an older version. It’s so sad for me.

I don’t think it’s wise, especially because it breaks backward compatibility.

I have a couple of custom-scripts that I use to migrate containers if necessary, which I migrate using zfs send / receive, mount through nsenter + lxd import container-name --force.

After upgrading to the latest version (after a long force-version in the snap) it’s just broken, lxd import said it can’t --force, if I try it without it, it will refer me to lxd recover, and this command cannot be used non-interactively or only for one defined container.

Please, can you consider, at least, to return the lxd import command in its original form and functionality, in order to maintain backward compatibility?

1 Like

You mentioned you were using lxd import to migrate, can I ask why you are not using lxc copy or lxc move to do this (which will use zfs send/receive under the hood) as that would be the recommended and supported way to migrate an instance.

An example scenario is:

  • rsync / dar - backup / restore
  • low level / step by step - copy / migration control

Sometimes we just need an import raw container, not start recovery process.

1 Like

So if I understand you correctly, you are using it for backup and restore purposes?

Also, yes. Backup can be done using various tools and restored at different locations. IMHO the lxd recover (alias lxd import) tool could support importing more directly, considering the flexibility of real-world environments. Thank you!

1 Like

Because my scripts which uses my rules have shortest downtime than copy/migrate via your official tools.

Its practically non-sense to explain reasons. The reality is that you have broken backward compatibility, and we are asking for this situation to be rectified, nothing more, nothing less.

Hi @tomp,

yes, lxd import with the properties of my first post is required for our restore process.
To clarify the need I show the script lxc-restore.sh to you:

#!/bin/bash

# restore one or multiple containers from remote backup snapshot;
# local restore target names may differ from remote backup container names
# 
# Florian Sager, sager@agitos.de, 2020-11-19

set -e
set -x

LXDBINPATH=/snap/bin
LXCBIN=$LXDBINPATH/lxc
LXDBIN=$LXDBINPATH/lxd

ZFSBIN=/sbin/zfs
ZFSROOT=tank/containers/

FSROOT=/var/snap/lxd/common/lxd/storage-pools/

UMOUNTBIN=/bin/umount

NSENTERCMD="/usr/bin/nsenter --mount=/run/snapd/ns/lxd.mnt"

RESTOREPREFIXCMD="/usr/bin/rsync -azHAX -e'ssh -p 2222 -o StrictHostKeyChecking=no' --numeric-ids "

if [[ -z "$1" || "$1" == "-h" || "$1" == "--help" ]]; then
    echo 'usage: lxc-restore.sh userXXX@userXXX.trustedspace.de:snapshot/20191006-040921-170/ foo-backup:foo bar-backup:bar ...'
	exit
fi 

RESTOREPREFIXEXTCMD="$RESTOREPREFIXCMD $1"

i=0
for containersrcdst in "$@"
do
	if [ $i == 0 ]; then
		# skip first argument
	    let "i+=1"
	    continue
	fi

	let "i+=1"
	
	IFS=":" read -a containersrcdstarr <<< $containersrcdst

	CONTAINERBACKUPNAME=${containersrcdstarr[0]}
	CONTAINERNAME=${containersrcdstarr[1]}
		
	if [[ -z "$CONTAINERBACKUPNAME" || -z "$CONTAINERNAME" ]]; then
		echo '$containersrcdst is not of format containerbackupname:targetcontainername'
		exit 1
	fi

	# /sbin/zfs create tank/containers/foo
	$ZFSBIN create $ZFSROOT$CONTAINERNAME
	# /sbin/zfs set mountpoint=/var/snap/lxd/common/lxd/storage-pools/tank/containers/foo tank/containers/foo
	$ZFSBIN set mountpoint=$FSROOT$ZFSROOT$CONTAINERNAME $ZFSROOT$CONTAINERNAME

	# restore container from backup
	# alternative: cp -Rp /tmp/footest/foo-backup/* /var/snap/lxd/common/lxd/storage-pools/tank/containers/foo/	
	# /usr/bin/rsync -azHAX -e 'ssh -oStrictHostKeyChecking=no' --numeric-ids userXXX@userXXX.trustedspace.de:snapshot/20191006-040921-170/foo-backup/ /var/snap/lxd/common/lxd/storage-pools/tank/containers/foo/
	eval $RESTOREPREFIXEXTCMD$CONTAINERBACKUPNAME/ $FSROOT$ZFSROOT$CONTAINERNAME
	if [ $? != 0 ]; then
		echo "rsync failed with $?"
		exit 2
	fi

	# /bin/umount /var/snap/lxd/common/lxd/storage-pools/tank/containers/foo
	$UMOUNTBIN $FSROOT$ZFSROOT$CONTAINERNAME

	# /sbin/zfs set canmount=noauto tank/containers/foo
	$ZFSBIN set canmount=noauto $ZFSROOT$CONTAINERNAME
	
	# /usr/bin/nsenter --mount=/run/snapd/ns/lxd.mnt mount -t zfs tank/containers/foo /var/snap/lxd/common/lxd/storage-pools/tank/containers/foo
	$NSENTERCMD mount -t zfs $ZFSROOT$CONTAINERNAME $FSROOT$ZFSROOT$CONTAINERNAME

	# lxd import foo --force
	$LXDBIN import $CONTAINERNAME --force

	# lxc start foo
	$LXCBIN start $CONTAINERNAME
	
	# lxc ls agiprxtest
	$LXCBIN ls $CONTAINERNAME

done
1 Like