I have created a network hook for LXC called lxc-routednet.
The inspiration for this program came from OpenVZ venet networking, which by default uses the host as a router and the containers use the network interface inside the container as the default ‘device’ route.
This approach to container networking offers some benefits:
- Less invasive to the host config, compared to bridging, as no bridge interface needs to be defined.
- Doesn’t allow containers to allocate their own IP or steal another IP from the network.
- Containers don’t get broadcast traffic (might be a pro or a con depending on your needs).
- Uses the host machine’s MAC address on the wider network when publishing the container’s IPs (good for networks that only allow 1 MAC per port or some other MAC based restriction).
- Allows iptables on the host to filter container traffic.
- Allows host and containers to talk to each other (unlike MACVLAN and IPVLAN, this might be a pro or con depending on your needs).
- In environments that use routing rather than layer 2 networking, this allows you to run a routing daemon on the host and publish the static routes added by the hook to upstream routers (e.g. Vultr and BGP).
- In environments that route non-sequential or out of subnet IPs to the host, the static routes added by the host allow traffic to reach the container.
OpenVZ kernels have a module called venet which provides a layer 3 tunnel between the container and the host. However I have emulated this style of networking by using a veth pair, without connecting the host end to a bridge. Instead the hook script adds a static route to the host and where appropriate, adds a proxy ARP/NDP entry on the host so that it claims traffic for the container’s IP.
The hook also sets up proxy ARP/NDP entries on the container’s veth end for the gateway IPs to allow the container to reach the host using a fixed gateway IP independent of the actual gateway on the network.
Source code: https://github.com/tomponline/lxc-routednet
# Network configuration lxc.net.0.type = veth lxc.net.0.flags = up lxc.net.0.name = eth0 lxc.net.0.ipv4.address = 192.168.31.3/32 lxc.net.0.ipv4.gateway = 169.254.0.1 lxc.net.0.ipv6.address = 2a00:1098:0:xx:xxxx::3/128 lxc.net.0.ipv6.gateway = fe80::1 lxc.net.0.script.up = /usr/libexec/lxc/lxc-routednet lxc.net.0.script.down = /usr/libexec/lxc/lxc-routednet
I would appreciate any feedback or ideas for improvement.