How to disable rsync compression for "lxc copy"? 100% cpu and slow for local network transfer

Over the 1Gb private local connection on an amd 3900x using 2x nvme raid0 LVM thin ext4 container lxc copy is only able to transfer at about 40MB/s or 300Mbps with rsync cpu at 100% on one core.

Here is the rysnc process spawned by lxc copy:

sh -c /snap/lxd/current/bin/lxd netcat @lxd/6a92977a-7acd-4f61-bd3f-df53e1a3a64b xxx localhost rsync --server -logD
tpXrSze.iLsfxC --compress-level=2 --delete --partial --numeric-ids . /tmp/foo

The 2 local servers is able to sustain full 1Gbp/s transfer and we believe the compression-level=2 option is causing rsync to use 100% of cpu.

However, we have not found a way to optionally disable or pass options to rsync that is started by lxc copy.

Any help is appreciated. Thanks.

Name    Version   Rev    Tracking       Publisher   Notes
core18  20200724  1885   latest/stable  canonicalâś“  base
lxd     4.6       17320  latest/stable  canonicalâś“  -
snapd   2.46.1    9279   latest/stable  canonicalâś“  snapd

Should be possible to introduce a new rsync.compress option on the storage pools to disable transfer compression when copying to/from a particular pool.

That would be great for future release.

In the meantime, can we compile lxd with the compression args disabled in the rsync code and place the custom LXD binary in /snap/lxd/current/bin? Is this the best way to run a custom LXD binary?

Yeah, a Github issue for it would be good. We’ll tag it as easy+hacktoberfest so hopefully someone picks it up as a way to start contributing to LXD :slight_smile:

To run an alternate LXD binary, we actually have a cleaner option which is to put it at /var/snap/lxd/common/lxd.debug. This will also cause a message to be logged on startup, reminding you that you’re not running the original binary.

Created github issue. Turns out my two server had 10g nics. and disabling compression on rsync improved lxc copy from 300Mbps to 3500Mbps. Cpu went from 100% to 80%. :exploding_head:

2 Likes