LXC DHCP Local Addresses

I see. I suppose in theory it can be done. You’d need to make sure that the DHCP server is giving out the external network’s IP for the gateway and not the VMware VM’s IP.

But you’ll also need to make sure that the Vmware VM’s ethernet interface is bridged onto the host’s external network, and check that there is no MAC filtering enabled in Vmware.

Once that has occurred, it will then expose your DHCP server onto the external network but this will then start giving IPs from the DHCP server to other devices on your network. If there is already a DHCP server on the external network then you will get conflicts.

Even if this didn’t happen, how would you prevent two independent DHCP servers from giving out conflicting addresses?

A simpler approach in my view would be to keep the LXD private bridge lxdbr0 and its DHCP server on a different subnet, and then setu a route in your external router/gateway that routes that additional subnet to the VMware VM’s IP.

Gotcha. If I just used the builtin lxdbr0 bridge and currently it’s assigning IPs in the 10.209.x range, would I need to do anything to expose that? Or just do a route for that subnet on the router/gateway to that IP? This assumes the host is on a totally different subnet.

Lets say the external gateway is on 10.0.0.1/24, the vmware host is on 10.0.0.2/24 and the LXD VM’s interface to the host is on 10.0.0.3/24 (I’m assuming you’ve bridged your VM onto the external network).

Then you would put a route on the gateway that routes 10.209.0/24 to 10.0.0.3.

Then make sure you allow that traffic through the vmware host’s external interface, and into the LXD VMs interface, and make sure you allow the traffic through any firewall running on the LXD VM host.

This my config for the lxdbr0. So do I need to modify that to be 10.0.0.3/24? Wouldn’t that then modify the DHCP scope it uses?

config:
  ipv4.address: 10.209.198.1/24
  ipv4.nat: "true"
  ipv6.address: none
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/ubuntu-test
- /1.0/profiles/default
managed: true
status: Created
locations:
- none

Please show output of ip a and ip r on LXD host, and indicate the network config you’re using to get connectivity from the vmware host into the LXD host.

ens192 is the physical interface on the VM. br0 is the bridge I created that acts as the interface.

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
    link/ether 00:50:56:bf:ad:b0 brd ff:ff:ff:ff:ff:ff
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:50:56:bf:ad:b0 brd ff:ff:ff:ff:ff:ff
    inet 10.1.22.73/24 brd 10.1.22.255 scope global noprefixroute br0
       valid_lft forever preferred_lft forever
    inet6 fe80::88c7:3586:4755:1da4/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
4: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:61:43:b4 brd ff:ff:ff:ff:ff:ff
    inet 10.209.198.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe61:43b4/64 scope link 
       valid_lft forever preferred_lft forever
 
default via 10.1.22.1 dev br0 proto static metric 425 
10.209.198.0/24 dev lxdbr0 proto kernel scope link src 10.209.198.1 
10.1.22.0/24 dev br0 proto kernel scope link src 10.1.22.73 metric 425

And is 10.1.22.0/24 the same subnet that is used on the VMware host to connect to the external gateway?

Sorry if I keep pressing this point, but I am still trying to establish your network configuration, thanks.

Yes, that is the defined network on the vswitch port that is connected to that VM

Hrm, not quite the answer I was looking for :slight_smile:

But is that vswitch network connected to the wider real network (i.e the same one that the external router is connected to)?

Yes. That is correct. The vswitch network is connected to the wider real network

OK so remove br0 entirely and put the 10.1.22.73/24 IP and gateway config back onto ens192 as it was before.

Then put a route on the gateway that routes the lxdbr0 subnet 10.209.198.0/24 to 10.1.22.73.

I’ll give that a try. Does it require any changes at the VMware switch level?

I’m not sure, as I don’t use Vmware.

However as you’re using routing, it shouldn’t require disabling the MAC filtering feature.

This may not work in our current setup. So this leaves to options:

  • Expose the LXC server as a DHCP server on the subnet so the containers can get DHCP addresses for the a valid subnet. Downside is that it will be a DHCP server for that entire subnet. Could firewall rules get around that?

  • External DHCP server on that subnet

I would recommend the latter option.

Would then I just configure the br0 interface like I did earlier to bridge to the external NIC and forgo the use of lxbr0?

Yes and you’ll need to ensure the external vmware nic is bridged to the external network and has no mac filtering enabled on it.

Got the external DHCP server setup. The container gets an IP from it. But, I still can’t access that IP on the broader network even though it’s a valid routable IP. The LXC host has a bridged adapter hooked up to a vNIC on the vm host that has mac filtering disabled. The iptables firewall is also disabled on the LXC host. What else could I be missing?

Generally with these sorts of issue you need to use tcpdump on each network transition interface and check where the packets are getting lost.