Problems setting up proxy device in LXD 4.17

LXD
1

Hi all,

I am trying to set up a container with a fixed IP in order to use a proxy device in NAT mode.
I have done this in the past, but now with LXD 4.17 I am running into several errors.
First I am wondering why one cannot specify a fixed IP address right at the beginning when launching a new container.

david@nnwh:~$ lxc launch ubuntu-minimal:focal c1 -c ipv4.address=10.65.76.163
Creating c1
Error: Failed instance creation: Failed creating instance record: Unknown configuration key: ipv4.address

And one cannot set it in one step either:

david@nnwh:~$ lxc config device set c1 eth0 ipv4.address 10.65.76.163
Error: Device from profile(s) cannot be modified for individual instance. Override device or modify profile instead

So it seems one needs to do this in several steps manually.

david@nnwh:~$ lxc stop c1
david@nnwh:~$ lxc network attach lxdbr1 c1 eth0 eth0
david@nnwh:~$ lxc config device set c1 eth0 ipv4.address 10.65.76.163

Now we have a container with a fixed IP address.
And now we want to define a proxy device in NAT mode.

david@nnwh:~$ lxc config device add c1 h22c1 proxy listen=tcp:xxx.xxx.xxx.xxx:16322 connect=tcp:127.0.0.1:22 nat=true
Device h22c1 added to c1

That seems to have worked. However, we cannot start the container anymore:

david@nnwh:~$ lxc start c1
Error: Proxy connect IP cannot be used with any of the instance NICs static IPs
Try lxc info --show-log c1 for more info
david@nnwh:~$ lxc info --show-log c1
Name: c1
Status: STOPPED
Type: container
Architecture: x86_64
Created: 2021/09/02 08:26 UTC
Last Used: 2021/09/02 08:38 UTC

Log:

lxc c1 20210902083815.566 WARN conf - conf.c:lxc_map_ids:3389 - newuidmap binary is missing
lxc c1 20210902083815.566 WARN conf - conf.c:lxc_map_ids:3395 - newgidmap binary is missing
lxc c1 20210902083815.567 WARN conf - conf.c:lxc_map_ids:3389 - newuidmap binary is missing
lxc c1 20210902083815.567 WARN conf - conf.c:lxc_map_ids:3395 - newgidmap binary is missing
lxc c1 20210902083815.568 WARN cgfsng - cgroups/cgfsng.c:fchowmodat:1293 - No such file or directory - Failed to fchownat(43, memory.oom.group, 1000000000, 0, AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW )
lxc c1 20210902083816.774 WARN conf - conf.c:lxc_map_ids:3389 - newuidmap binary is missing
lxc c1 20210902083816.843 WARN conf - conf.c:lxc_map_ids:3395 - newgidmap binary is missing

2

When modifying a NIC that comes from a profile you can use the lxc config device override command to make a copy of the NIC into the instance and modify it at the same time, e.g.

lxc init ubuntu-minimal:focal c1
lxc config device override c1 eth0 ipv4.address=10.65.76.163

Then when adding a proxy device in NAT mode, because this uses firewall rules on the host to forward packets over the network connection between the host and the container, you must use the container’s actual IP address and not the local loopback address (127.0.0.1), as otherwise that would refer to the local loopback address on the host.

So either specify the connect IP explicitly, or you can specify the shortcut 0.0.0.0 or :: (for IPv6) that will then select the statically assigned IP for the instance.

E.g.

lxc config device add c1 p1 proxy listen=tcp:n.n.n.n:22 connect=tcp:10.65.76.163:22 nat=true

or

lxc config device add c1 p1 proxy listen=tcp:n.n.n.n:22 connect=tcp:0.0.0.0:22 nat=true

Either way when using nat=true the instance’s NIC requires a static IP to ensure it won’t change later on and end up forwarding packets to a different instance.

The “Error: Proxy connect IP cannot be used with any of the instance NICs static IPs” message is saying that the connect IP you have specified (127.0.0.1) is not one of the instance’s NIC addresses.

You should also ensure that the service you are connecting to is listening on the connect IP (or wildcard address) inside the container.

You can only use 127.0.0.1 as the connect IP when using nat=false mode because this then uses a separate proxy process to actually relay the packets from the host namespace to the container’s network namespace (and doesn’t go over the network connection itself). In fact you can use a proxy device in non-nat mode even if it doesn’t have a network connection at all.

2 Likes
3

Thanks a lot!

4

minimal correction:

lxc config device override c1 eth0 ipv4.address 10.65.76.163

should read

lxc config device override c1 eth0 ipv4.address=10.65.76.163

1 Like
5

Thanks fixed.

6

I’ve added some improved error messages for proxy errors to help clarify things as well:

https://github.com/lxc/lxd/pull/9179