Dear Devs (@stgraber and @brauner , and everyone I missed) )) –
– to follow up on 3073 (Future incremental copy - (lxc refresh) - ZFS backend), let me check the present state of availability of incremental backups for lxc/lxd
:
-
3073 – the most popular post that I can find on
linuxcontainers.org
that we can consider a feature request )- 3225 – one another, probably there are more
Now lxc copy --refresh
apparently uses rsync
only :
- https://discuss.linuxcontainers.org/t/lxc-copy-refresh/9302
- https://github.com/lxc/lxd/issues/7902#issuecomment-696090517
This could be noticeably inefficient compared to a zfs send | recv
– and as far as I know, there is at least one successful attempt to implement this feature in the userland:
To add, I have also devised a similar solution based on Florian’s work (@usrflo ) and a look at lxd sql global .dump
. This procedure makes lxd
aware of any new zfs
snapshots that were added on a particular container:
In essence, it compares two lists – (1) current zfs
snapshots of a container and (2) the list of snapshots as known by lxd
and stored in instances_snapshots
, and then tries to recreate missing records using some lxd sql
magic.
It “works for me” ™, but I am concerned that with lxd
now following a rolling-release scheme via snap
(another issue that I shall mention separately), this lxd sql
trickery can break any moment you guys decide to make any improvements to the sql .schema
)
So I wonder if there is a way to have incremental zfs
sync being less fragile )
I’m not sure – may be we already have something like this :
- Effective backup/restore using zfs snapshots/send/receive - #2 by stgraber – do I read it right that I can re-import an existing container and the import magic will update the snapshot list for me?
Otherwise, I wonder how hard it could be to add an additional feature to extend --optimized-storage
to do incremental backups, so that they could be then incrementally restored on a container that was imported earlier?
For the reference:
-
issue 3730 that implements
--optimized-storage
feature - LXD: Backup by exporting and importing containers - asciinema – a very helpful tutorial by @brauner, explaining the feature
If you think that this is feasible, I can draft a feature request on Github – I am thinking of something like --incremental-since $snapname
or --incremental-since $start-date
for lxc export
and then making lxc import
to be smart enough to detect an incremental update (may be requiring an --incremental
flag from the user so there’s no misunderstanding).
Upd. Adding @my5t3ry to the list of tagged users; reason: a related discussion at github issue 7902.