Iptables rule to access site from within? (proxy NAT mode)

burbilog · July 6, 2020, 2:49pm

architecture: x86_64
config:
  image.architecture: amd64
  image.description: Alpine 3.9 amd64 (20190321_13:00)
  image.os: Alpine
  image.release: "3.9"
  image.serial: "20190321_13:00"
  volatile.base_image: 0cbd911b5a203c7e475241b8b22cc5332d10fd30ae27916bae1558bcb118c9ce
  volatile.eth0.host_name: veth196022e7
  volatile.eth0.hwaddr: 00:16:3e:6b:04:12
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    ipv4.address: 172.16.172.116
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  root:
    path: /
    pool: tank
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

tomp · July 6, 2020, 2:50pm

Yep, so your eth0 NIC has a static IP defined already. So when you added a proxy in NAT mode with a connect option of tcp:0.0.0.0:7777 will use the static IP to setup the NAT rule.

tomp · July 6, 2020, 2:53pm

In order to define a static IPv6 address, you also need to enable stateful DHCPv6 on the LXD managed network (otherwise the container may use SLAAC to generate a random IP). You will get an error to that effect when you try and define a static IPv6 address on a bridged NIC.

burbilog · July 6, 2020, 3:06pm

I did not enable stateful dhcpv6, but right now I was able to set static ipv6 address and create ipv6 proxy for that container.

tomp · July 6, 2020, 3:07pm

Maybe you have it enabled already or you’re running an earlier version without the check.

burbilog · July 6, 2020, 3:08pm

Where can I look up that option?

lxd     4.3        15913  latest/stable  canonical✓  -

tomp · July 6, 2020, 3:08pm

lxc network show lxdbr0
config:
  ipv4.address: 10.109.89.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:d37c:f0f2:a5f::1/64
  ipv6.dhcp.stateful: "true"

burbilog · July 6, 2020, 3:15pm

config:
  ipv4.address: 172.16.172.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:dad8:c4ad:e744::1/64
  ipv6.nat: "true"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/alp
- /1.0/instances/cms
- /1.0/instances/dd
- /1.0/instances/grr
- /1.0/instances/main
- /1.0/instances/static
- /1.0/profiles/default
managed: true
status: Created
locations:
- none

tomp · July 6, 2020, 3:50pm

Here’s an example of what I mean:

lxc network show lxdbr0
config:
  ipv4.address: 10.109.89.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:d37c:f0f2:a5f::1/64
  ipv6.nat: "true"
description: ""
name: lxdbr0
type: bridge

lxc init images:ubuntu/focal c1

lxc config device override c1 eth0 ipv6.address=fd42:d37c:f0f2:a5f:216:3eff:feb8:899b 
Error: Invalid devices: Device validation failed "eth0": Cannot specify "ipv6.address" when "ipv6.dhcp" or "ipv6.dhcp.stateful" are disabled on network "lxdbr0"

lxc network set lxdbr0 ipv6.dhcp.stateful=true
lxc config device override c1 eth0 ipv6.address=fd42:d37c:f0f2:a5f:216:3eff:feb8:899b 
Device eth0 overridden for c1

tomp · July 6, 2020, 4:00pm

burbilog · July 6, 2020, 5:38pm

lxc network set lxdbr0 ipv6.dhcp.stateful=false 
lxc config device set grr eth0 ipv6.address fd42:dad8:c4ad:e744:216:3eff:fecf:5770

No complaints so far. What’s the difference between set and override?.. override does not work for me:

lxc config device override grr eth0 ipv6.address=fd42:dad8:c4ad:e744:216:3eff:fecf:5770      
Error: The device already exists

tomp · July 6, 2020, 5:51pm

Set modifies an existing device in the container, but wont work if nic is from profile , override copies device config from profile into container and then modifies one or more keys on the copy.

tomp · July 6, 2020, 5:53pm

Id need to see network and container config before and after setting ipv6 address which dhcpv6 stateful false to advise.

burbilog · August 8, 2020, 1:52am

ns pts/0#lxc network show lxdbr0 
config:
  ipv4.address: 172.16.172.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:dad8:c4ad:e744::1/64
  ipv6.dhcp.stateful: "false"
  ipv6.nat: "true"
  volatile.bridge.hwaddr: 00:16:3e:22:17:f9
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/alp
- /1.0/instances/cms
- /1.0/instances/dd
- /1.0/instances/grr
- /1.0/instances/main
- /1.0/instances/static
- /1.0/profiles/default
managed: true
status: Created
locations:
- none

Before:

ns pts/0#lxc config show grr --expanded                                                                [/home/rm 4:50 Sat 8] 
architecture: x86_64
config:
  image.architecture: amd64
  image.description: Alpine 3.11 amd64 (20200705_13:00)
  image.os: Alpine
  image.release: "3.11"
  image.serial: "20200705_13:00"
  image.type: squashfs
  volatile.base_image: 6d6ea490506430013373d4049212fc2412a7b62e63a8807ddeaba8dbefa128f5
  volatile.eth0.host_name: veth934917ee
  volatile.eth0.hwaddr: 00:16:3e:cf:57:70
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    ipv4.address: 172.16.172.113
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  proxy777:
    connect: tcp:0.0.0.0:777
    listen: tcp:168.235.88.175:777
    nat: "true"
    type: proxy
  proxy777v6:
    connect: tcp:[::]:777
    listen: tcp:[2604:180:2:5f8::2]:777
    nat: "true"
    type: proxy
  root:
    path: /
    pool: tank
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

Setting ipv6 address:

ns pts/0#lxc config device set grr eth0 ipv6.address fd42:dad8:c4ad:e744:216:3eff:fecf:5770
ns pts/0#

After:

ns pts/0#lxc config show grr --expanded                                                                [/home/rm 4:51 Sat 8] 
architecture: x86_64
config:
  image.architecture: amd64
  image.description: Alpine 3.11 amd64 (20200705_13:00)
  image.os: Alpine
  image.release: "3.11"
  image.serial: "20200705_13:00"
  image.type: squashfs
  volatile.base_image: 6d6ea490506430013373d4049212fc2412a7b62e63a8807ddeaba8dbefa128f5
  volatile.eth0.host_name: veth934917ee
  volatile.eth0.hwaddr: 00:16:3e:cf:57:70
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    ipv4.address: 172.16.172.113
    ipv6.address: fd42:dad8:c4ad:e744:216:3eff:fecf:5770
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  proxy777:
    connect: tcp:0.0.0.0:777
    listen: tcp:168.235.88.175:777
    nat: "true"
    type: proxy
  proxy777v6:
    connect: tcp:[::]:777
    listen: tcp:[2604:180:2:5f8::2]:777
    nat: "true"
    type: proxy
  root:
    path: /
    pool: tank
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

tomp · August 10, 2020, 8:11am

I cant tell for sure because the output is from the command run with the --expanded argument which includes the profiles config. But I suspect the container has its own eth0 device, which is what allows set to run.

tomp · August 10, 2020, 8:19am

Ah, sorry, I’d forgotten what was the issue we were discussing, your other thread reminded me, it was the ability to set a static IPv6 with stateful off.

I presume you enabled stateful DHCP on the network, then set the NIC’s static IP, and then turned off stateful on the parent network (I don’t think there is a check when disabling stateful that checks all linked NICs for incompatible options).

Note: Even if you have been able to do that, as @stgraber said on the other thread, that static IP won’t take effect unless the parent network actually has stateful mode enabled.

tomp · August 10, 2020, 8:23am

Ah, no, sorry again, I checked the code and the check for parent network having stateful mode enabled is only implemented if using the NIC network=<parent> option not the parent=<parent> approach you’ve used here.

We haven’t enforced the IP allocation rules when using the parent=<parent> option because you can specify an unmanaged parent bridge with that option, and there are plans to allow users to enable the security.ip_filtering* features on a NIC when using an unmanaged parent which would be blocked if we replicated the stateful check in this configuration.

See Isolation of LXD containers? for a recent discussion about this.