EDIT: After experimenting with other devices on my network I was able to reproduce the issue so it was somehow related to my network. It turns out where I’d expanded my internal subnet size the firewall’s outbound NAT rules hadn’t changed, so it was only NATting the first /24.
Pretty new user to LXD here so I apologise if I’ve done something stupid. I’d like my containers to use static IPs I assign on eth0
from a range outside of my DHCP pool, however it’s not quite behaving as I expect. My host has eth0
bridged to br0
with the following config:
$ ip a s dev br0
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether bc:ae:c5:11:9f:41 brd ff:ff:ff:ff:ff:ff
inet 10.101.0.50/22 brd 10.101.3.255 scope global br0
valid_lft forever preferred_lft forever
I’m also attaching containers directly to this bridge:
devices:
eth0:
name: eth0
nictype: bridged
parent: br0
type: nic
$ brctl show
br0 8000.bcaec5119f41 no eth0
vethS9Q44S
This works fine when I allow containers to obtain an IP via DHCP and also when I assign a static IP in the 10.101.0.0/24 range (I can ping 1.1.1.1 for example), however once I try to use the full range of the /22 subnet things get a bit strange. Below is a sample cloud-init config:
version: 1
config:
- type: physical
name: eth0
subnets:
- type: static
ipv4: true
address: 10.101.1.211/22
gateway: 10.101.0.1
control: auto
With this I can ping my gateway just fine, but I cannot ping the outside world. I ran some traffic captures on my gateway’s WAN interface and saw very odd behaviour - packets going out onto the internet still have their RFC1918 address which obviously means Cloudflare’s DNS server is going to drop them as garbage traffic:
08:49:21.172012 IP 10.101.1.211 > 1.1.1.1: ICMP echo request, id 442, seq 1, length 64
08:49:22.171418 IP 10.101.1.211 > 1.1.1.1: ICMP echo request, id 442, seq 2, length 64
08:49:23.171289 IP 10.101.1.211 > 1.1.1.1: ICMP echo request, id 442, seq 3, length 64
Whereas if I ping from the host itself:
09:24:09.243287 IP 81.154.6.x > 1.1.1.1: ICMP echo request, id 57368, seq 1, length 64
09:24:09.247632 IP 1.1.1.1 > 81.154.6.x: ICMP echo reply, id 57368, seq 1, length 64
09:24:10.245032 IP 81.154.6.x > 1.1.1.1: ICMP echo request, id 57368, seq 2, length 64
09:24:10.249472 IP 1.1.1.1 > 81.154.6.x: ICMP echo reply, id 57368, seq 2, length 64
09:24:11.245538 IP 81.154.6.x > 1.1.1.1: ICMP echo request, id 57368, seq 3, length 64
09:24:11.249914 IP 1.1.1.1 > 81.154.6.x: ICMP echo reply, id 57368, seq 3, length 64
I’m rather scratching my head here - is this something wrong with my network setup or with my LXD setup? There are no iptables rules involved (the only rules are the auto-generated ones for lxcbr0). I could understand it if nothing worked at all, but a container with an IP from the first /24 can ping the outside world fine whether the IP is from DHCP or statically assigned.
Any suggestions welcome!