“…subsequent refreshes will then compare the list of snapshots, delete any snapshot which was removed from the source or which appears to have been changed and then sync the missing snapshots and container state using rsync.”
Does this mean, that there is no incremental aspect? Just snapshots are checked and the container itself is not compared?
I use the DIR backend and of course it would be cool to only load over the changes, especially for that backend and the purpose to have a backup. And saving network bandwidth when copying to a remote.
So it needs that long to compare the changes when I run the command lxc copy container backup --refresh
on a container that has not changed at all and it takes a minunte?
It may not need to write anything but it will still need to look at every single file in the source and target, look at their last modification date and file size to try to figure out if they’ve changed or not.
This changes the way rsync checks if the files have been changed and are in need of a transfer. Without this option, rsync uses a “quick check” that (by default) checks if each file’s size and time of last modification match between the sender and receiver. This option changes this to compare a 128-bit checksum for each file that has a matching size. Generating the checksums means that both sides will expend a lot of disk I/O reading all the data in the files in the transfer (and this is prior to any reading that will be done to transfer changed files), so this can slow things down significantly.
Whereas our remote copy function does not use this option from what I can tell:
I think our remote copy logic predates the ability to refresh existing data so never really needed it. Adding it would make sense but as I suspect we need to pass it on both sides, we’ll need it to be a negotiated option similar to compression, delete, … (still very easy to add)
Hi Chris, thanks for your question.
Unfortunately I do not understand what you mean.
If you installed LXD somewhere you can simply type “lxc” and get the general help section right in the CLI. There you can find out lxc copy and after going a few levels deep lxc storage volume copy:
lxc copy
Description:
Copy instances within or in between LXD servers
Transfer modes (--mode):
- pull: Target server pulls the data from the source server (source must listen on network)
- push: Source server pushes the data to the target server (target must listen on network)
- relay: The CLI connects to both source and server and proxies the data (both source and target must listen on network)
The pull transfer mode is the default as it is compatible with all LXD versions.
Usage:
lxc copy [<remote>:]<source>[/<snapshot>] [[<remote>:]<destination>] [flags]
Aliases:
copy, cp
Flags:
--allow-inconsistent Ignore copy errors for volatile files
-c, --config Config key/value to apply to the new instance
-d, --device New key/value to apply to a specific device
-e, --ephemeral Ephemeral instance
--instance-only Copy the instance without its snapshots
--mode Transfer mode. One of pull, push or relay (default "pull")
--no-profiles Create the instance with no profiles applied
-p, --profile Profile to apply to the new instance
--refresh Perform an incremental copy
--stateless Copy a stateful instance stateless
-s, --storage Storage pool name
--target Cluster member name
--target-project Copy to a project different from the source
Global Flags:
--debug Show all debug messages
--force-local Force using the local unix socket
-h, --help Print help
--project Override the source project
-q, --quiet Don't show progress information
--sub-commands Use with help or --help to view sub-commands
-v, --verbose Show all information messages
--version Print version number
lxc storage volume copy
Description:
Copy storage volumes
Usage:
lxc storage volume copy [<remote>:]<pool>/<volume>[/<snapshot>] [<remote>:]<pool>/<volume> [flags]
Aliases:
copy, cp
Flags:
--destination-target Destination cluster member name
--mode Transfer mode. One of pull (default), push or relay. (default "pull")
--refresh Refresh and update the existing storage volume copies
--target Cluster member name
--target-project Copy to a project different from the source
--volume-only Copy the volume without its snapshots
Global Flags:
--debug Show all debug messages
--force-local Force using the local unix socket
-h, --help Print help
--project Override the source project
-q, --quiet Don't show progress information
--sub-commands Use with help or --help to view sub-commands
-v, --verbose Show all information messages
--version Print version number