Rsync files into container from host?

Hi,

to push some app folders into container, following works:

sudo lxc file push -r * test-container/source/app1

but this seems copying all files every time, is there a way to do a rsync like copying? Thanks,

sq

You have to be slightly creative but it can be made to work pretty well.

stgraber@castiana:~$ cat fake-ssh 
#!/bin/sh
ctn="${1}"
shift
exec lxc exec "${ctn}" -- "$@"

stgraber@castiana:~$ rsync -avPz recover/ -e ./fake-ssh xenial:/tmp/a
sending incremental file list
created directory /tmp/a
./
IMG_3705.JPG
      8,664,462 100%   28.68MB/s    0:00:00 (xfr#1, to-chk=14/16)
IMG_3726.JPG
      7,648,038 100%   13.61MB/s    0:00:00 (xfr#2, to-chk=13/16)
IMG_3757.JPG
      5,792,282 100%    7.72MB/s    0:00:00 (xfr#3, to-chk=12/16)
IMG_3782.JPG
      9,272,999 100%    8.89MB/s    0:00:00 (xfr#4, to-chk=11/16)
IMG_3788.JPG
      7,071,538 100%    5.57MB/s    0:00:01 (xfr#5, to-chk=10/16)
IMG_3809.JPG
      6,089,773 100%   14.52MB/s    0:00:00 (xfr#6, to-chk=9/16)
IMG_3835.JPG
      8,419,272 100%   12.35MB/s    0:00:00 (xfr#7, to-chk=8/16)
IMG_3855.JPG
      8,346,781 100%    8.88MB/s    0:00:00 (xfr#8, to-chk=7/16)
IMG_3867.JPG
      5,521,290 100%    4.86MB/s    0:00:01 (xfr#9, to-chk=6/16)
IMG_3892.JPG
      3,527,415 100%   16.49MB/s    0:00:00 (xfr#10, to-chk=5/16)
IMG_3894.JPG
      5,490,255 100%   13.74MB/s    0:00:00 (xfr#11, to-chk=4/16)
IMG_3899.JPG
      5,402,756 100%    9.45MB/s    0:00:00 (xfr#12, to-chk=3/16)
IMG_3908.JPG
      7,180,567 100%    9.01MB/s    0:00:00 (xfr#13, to-chk=2/16)
IMG_3915.JPG
      5,370,654 100%    5.53MB/s    0:00:00 (xfr#14, to-chk=1/16)
IMG_3939.JPG
      5,320,861 100%    4.65MB/s    0:00:01 (xfr#15, to-chk=0/16)

sent 98,861,183 bytes  received 333 bytes  28,246,147.43 bytes/sec
total size is 99,118,943  speedup is 1.00

stgraber@castiana:~$ lxc exec xenial -- ls -lh /tmp/a/
total 95M
-rw-r--r-- 1 root root 8.3M Aug 19 20:28 IMG_3705.JPG
-rw-r--r-- 1 root root 7.3M Aug 19 21:06 IMG_3726.JPG
-rw-r--r-- 1 root root 5.6M Aug 20 18:09 IMG_3757.JPG
-rw-r--r-- 1 root root 8.9M Aug 20 19:05 IMG_3782.JPG
-rw-r--r-- 1 root root 6.8M Aug 20 19:09 IMG_3788.JPG
-rw-r--r-- 1 root root 5.9M Sep 22 18:10 IMG_3809.JPG
-rw-r--r-- 1 root root 8.1M Sep 23 00:24 IMG_3835.JPG
-rw-r--r-- 1 root root 8.0M Sep 23 00:38 IMG_3855.JPG
-rw-r--r-- 1 root root 5.3M Sep 23 00:50 IMG_3867.JPG
-rw-r--r-- 1 root root 3.4M Sep 23 01:48 IMG_3892.JPG
-rw-r--r-- 1 root root 5.3M Sep 23 01:51 IMG_3894.JPG
-rw-r--r-- 1 root root 5.2M Nov 10 06:43 IMG_3899.JPG
-rw-r--r-- 1 root root 6.9M Nov 10 06:47 IMG_3908.JPG
-rw-r--r-- 1 root root 5.2M Nov 10 06:50 IMG_3915.JPG
-rw-r--r-- 1 root root 5.1M Nov 10 20:53 IMG_3939.JPG
stgraber@castiana:~$ 
3 Likes

still trying to understand this line:
exec lxc exec “${ctn}” – “$@”

The script above will:

  1. Store the first argument provided by rsync (target name) in the ctn variable
  2. Shift all arguments by 1 (effectively dropping the argument we just stored)
  3. Exec “lxc exec” for the container name we stored earlier and passing in all the remaining arguments as the command to execute in the container
1 Like

that’s really creative!

1 Like

How about a full rsync of the container “rootfs” ?
In case of full backup/restore what would be the best way to sync a remote backup of rootfs to the the actual container ?

If the container is running, it’s a problem . I tried, my host got 90 of load and not happy ^^
If the container is not running, how coud i interact with the FS (ZFS ) ?

Thanks a lot.

I’m not sure I fully understand what needs to change for a file pull. Also I notice this post is 2+ years old, is there any update regarding this functionality?

Thank you so much for your slightly creative script and the complete rsync command, which are the very things I’m looking for, @stgraber. By the way, let me improve a little bit what you’ve created here (Take the .vim folder as an example):

  1. Install rsync for your LXC container instance, or there’ll appear a “remote command not found” error;
  2. chmod u+x ./fake-ssh (Make file fake-ssh executable);
  3. Run the rsync command whether or not your destination folder exists:
    rsync -avPz ./.vim/ -e ./fake-ssh myconainter:/tmp/.vim/
  4. Another option - the default style of Grsync:
    rsync -r -t --progress -s ./.vim/ -e ./fake-ssh myconainter:/tmp/.vim/

Please keep an eye on the trailing file separator “/” in both the source and the destination folders as rsync is sensitive to it. Without it, a redundant folder might be created in the destination folder.

Can we run the other way around - sync a folder inside the container with its counterpart on the host? I believe it’s viable as this is really the basic functionality of rsync. The practical requirement does exist, say we want to export/sync something from inside the container and lxc file pull -r ... is cumbersome or overrides the files unnecessarily.

For other people who come here, this also works without needing the script file:

rsync -avP -e "sh -c 'lxc exec \"\$0\" -- \"\$@\"'" containername:/containerpath /outsidepath

etc.