3.19 and Routed networking mode configuration example needed

LXD
2

Hi @bmullan

So routed network mode works as follows:

  • It uses liblxc under the hood to create a veth pair of interfaces, and moves one side of the veth pair into the container.
  • It then preconfigures the static IP addresses you have specified onto the interface.
  • Next it sets up default routes in the container ponting to 169.254.0.1 for IPv4 and fe80::1 for IPv6. These are link-local addresses and serve only to get packets to/from the container to the host.
  • On the host, static routes are added for the container’s IP pointing to the host-side of the veth pair.
  • Additionally the link-local gateway IPs are created on every host-side veth pair.

This means you should avoid having any start up services inside the container that remove IPs from the interfaces and attempt to do DHCP. You should also be aware that if you are using static IPs from an existing DHCP range then unlike ‘bridged’ mode, these do not create static reservations in the parent bridged’s local DHCP server and so you should ensure you’ve configured the DHCP range to not overlap with the static IPs assigned.

Note: The routed network mode does not require you to specify a parent option. This means that the IPs you specify for the container do not have to be part of any subnet on the host, and you can instead choose to propagate these routes using a routing daemon.

However you can optionally specify a parent option, and in this case proxy ARP and proxy NDP entries are added to that parent interface “advertising” those IPs at layer2 to the parent interface. If those IPs are in the same subnet as the parent interface’s network, this then acts as a kind of ‘bridge’ allowing those containers to appear on the parent’s interfaces network. However you do not need to use any bridge, and all the containers will appear to be using the host’s parent interface’s MAC address.

I’m not sure what you mean about the IP for the bridged, 10.0.2.1, as routed mode will not alter the bridge IP, especially one it is not a parent of. They key point here is that routed mode doesn’t need any bridges to operate.

So if I start with a container config like so:

devices:
  eth0:
    ipv4.address: 10.138.198.132
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic

With the lxdbr0 interface having an IP of 10.138.198.1.

Then I can change the container’s nictype to “routed”:

devices:
  eth0:
    ipv4.address: 10.138.198.132
    name: eth0
    nictype: routed
    parent: lxdbr0
    type: nic

Starting the container then shows on the host:

ip r
10.138.198.0/24 dev lxdbr0 proto kernel scope link src 10.138.198.1 linkdown 
10.138.198.132 dev veth0ff2a75d scope link

You can also see the container’s IP being ‘advertised’ via proxy ARP to the parent interface.

ip neigh show proxy
169.254.0.1 dev veth0ff2a75d  proxy
10.138.198.132 dev lxdbr0  proxy

And in the container you can see the default link-local routes created:

lxc exec c1 ip r
default via 169.254.0.1 dev eth0 
169.254.0.1 dev eth0 scope link

Note: In this example, if I removed the “parent: lxdbr0” part from my container, then it would prevent other containers connected to the bridged from communicating with my container. This is because without the parent option the container is not advertised at layer 2 onto the lxdbr0 interface.

However the container would still be able to communicate with the host via its default routes.

3 Likes