Dhcp and routing issue inside LXC container

I have Ubuntu 16 with KVM installed running on the hardware. I have an ubuntu 18 VM running there. Inside the ubuntu 18 I have LXC version 3.0.3 freshly installed from ubuntu repos. I’m trying to make containers there that are able to reach the internet. I’m following a tutorial from the linuxfoundation.
When the container boots up it receives no IP from dhcp (dnsmasq IS running in the ubuntu 18 VM).
In the ubuntu 18 Vm I have:
ens3 - 192.168.122.201/24
lxcbr0 - 10.0.3.1/24
vethRQ1I0A - (no ip)

In the container I have Ubuntu 18 as well, with the following:
eth0 (no ip), and dhclient eth0 does nothing.

However the weird thing is, I can assign a static IP to the container (say 10.0.3.2/24) and it accepts the IP.
I can then ping from Ubuntu 18 VM to the container (10.0.3.1 can ping 10.0.3.2) but NOT from the container to the VM (10.0.3.2 CANNOT ping 10.0.3.1). Routing also doesn’t work, like if I add a default route in the container (ip route add default via 10.0.3.1 dev eth0 proto static), it still cannot reach its own gateway or higher up in the nat.

Ipv4 and v6 forwarding is enabled on the ubuntu 18 VM and the container.

Here is my /etc/default/lxc-net:

USE_LXC_BRIDGE=“true”
LXC_BRIDGE=“lxcbr0”
LXC_ADDR=“10.0.3.1”
LXC_NETMASK=“255.255.255.0”
LXC_NETWORK=“10.0.3.0/24”
LXC_DHCP_RANGE=“10.0.3.2,10.0.3.254”
LXC_DHCP_MAX=“253”
LXC_DHCP_CONFILE=""
LXC_DOMAIN=""

This is what I did to start the container:
lxc-create -n bucket -t download – -d ubuntu -r bionic -a amd64
lxc-start -n bucket
lxc-attach -n bucket

And that was it, in the linuxfoundation tutorial it was supposed to just work. I’ve been stuck on this now for several days. Not sure why I have this issue.

We would need to see your LXC container config file to be able to advise further.

Also please show output of ip a and ip r both in the VM and in the container.

Also, are you planning on NATting the outbound traffic from the container to the VM’s IP so it can access the external network?

Here is what I have:

/var/lib/lxc/bucket/config

lxc.include = /usr/share/lxc/config/common.conf

For Ubuntu 14.04

lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.mount.entry = mqueue dev/mqueue mqueue rw,relatime,create=dir,optional 0 0
lxc.arch = linux64

Container specific configuration

LXC.apparmor.profile = generated
LXC.apparmor.allow_nesting = 1
lxc.rootfs.path = dir:/var/lib/lxc/bucket/rootfs
lxc.uts.name = bucket

Network configuration

lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:02:da:78

From the vm:

VM Ip Configuration

chris@lfs201:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:3a:66:65 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.201/24 brd 192.168.122.255 scope global ens3
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe3a:6665/64 scope link
valid_lft forever preferred_lft forever
3: lxcbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:16:3e:00:00:00 brd ff:ff:ff:ff:ff:ff
inet 10.0.3.1/24 scope global lxcbr0
valid_lft forever preferred_lft forever
inet6 fe80::216:3eff:fe00:0/64 scope link
valid_lft forever preferred_lft forever
5: veth2QDRRL@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxcbr0 state UP group default qlen 1000
link/ether fe:ad:e6:fa:cb:79 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::fcad:e6ff:fefa:cb79/64 scope link
valid_lft forever preferred_lft forever
chris@lfs201:~$ ip r
default via 192.168.122.1 dev ens3 proto static
10.0.3.0/24 dev lxcbr0 proto kernel scope link src 10.0.3.1
192.168.122.0/24 dev ens3 proto kernel scope link src 192.168.122.201

In the container:

Ip configuration of the container

root@bucket:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
4: eth0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:16:3e:02:da:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::216:3eff:fe02:da78/64 scope link
valid_lft forever preferred_lft forever
root@bucket:/# ip r
root@bucket:/#

Also, are you planning on NATting the outbound traffic from the container to the VM’s IP so it can access the external network?

Yes, however I think this may already be being done because I see the following in iptables -t nat -L -n -v

Chain POSTROUTING (policy ACCEPT 105 packets, 10274 bytes)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all – * * 10.0.3.0/24 !10.0.3.0/24

Actually I reimaged the vm and it just worked out of the box. I’m now looking at the differences between the images, but idk what it could be.

1 Like