Lxc copy from an AWS instance to a remote Digital Ocean Instance fails (I think do to AWS Public/Private IP addressing use)

LXD
1

All servers and containers are Ubuntu 18.04

All are using SNAP LXD version 3.14

I have 2 servers on AWS (S1 and S2) and did the lxd init on each so they could access each other over the “internet” and gave them both the same password.

Note: on AWS each server has a PUBLIC IP and a PRIVATE IP. The Private IP is not accessible from the Internet but only from other AWS instances!

On S1 I executed:

$ lxc remote add S2 <Public IP of S2 AWS instance>

On S1 server I have no problem successfully copying a container to S2.

$ lxc copy cn1 S2:cn1

After a couple minutes ON S2 “lxc list” shows a cn1 container there and I can start it and access it.

Now the problem comes when I try to lxc copy from the AWS instance to a Digital Ocean server instance setup the same way to allow LXD over the internet using port 8443.

On AWS S1 I execute:

$ lxc remote add DO-S1

On AWS S1 I can now see the DO-S1 server in the remote list

$ lxc remote list

But when I try to execute the copy command from AWS S1 to Digital Ocean DO-S1 it now fails and
not because of Digital Ocean but because the AWS S1 LXD says it cannot access the AWS S1 “Private IP” address 10.0.0.223:8443 even though I used the AWS S1 Public IP address when I setup LXD with lxd init.

$ lxc copy cn1 DO-S1:cn1
Error: Failed container creation: Error transferring container data: Unable to connect to: 10.0.0.223:8443

Note the failure message displays the private not public IP address of the source server S1

I’ve used both AWS and Digital Ocean clouds for long time. I understand how AWS uses their Instance’s Public and Private IP address and so I understand that when doing the

lxc copy cn1 S2:cn1

and both S1 and S2 are on AWS that despite configuring the S2 remote on S1 using the PUBLIC IP of S2

$ lxc remote add S2 <Public IP of AWS S2 server instance

that AWS actually routes using the Private IP of S2 (but that won’t work going outside of AWS to Digital Ocean)

So has anyone else tried to copy a container from AWS to another Cloud like Digital Ocean and figured out how to make it work.

thanks for any ideas.

Brian

2

Try using --mode=push to have the source push to the target.
Worst case scenario, use --mode=relay which will have your client relay the data between the two.

3

Thanks Stephane!

Yes adding adding the --mode=push worked !

So on AWS if you want to copy an LXD container to some Server outside of AWS (re Digital Ocean or Hetzner etc) you would have to add the “–mode=push” option to the command line !

lxc copy cn1 S2:cn1 --mode=push

But to copy an LXD container from one AWS server to another AWS server you do not need that option.