Hi 
, everyone,
I have been using LXD on an Ubuntu 18.04 host to learn for a while now, and it seems I stumbled upon the first issue that I cannot resolve at all.
This dedicated server from Hetzner has been running a single container for a few months now, without any issues. Today I had to shut the server down to install two more drives, and when I booted the server back up, network on the contain was nowhere to be seen.
I went back to the host server, issue a lxc list command and saw that there were no IP addresses assigned to the container.
root@tardis-01.madpony.host:/home/doctor# lxc list
+----------+---------+------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+----------+---------+------+------+------------+-----------+
| smith-01 | RUNNING | | | PERSISTENT | 0 |
+----------+---------+------+------+------------+-----------+
When I deployed this server I was running 4.15.0-72-generic, but the restart got us into 5.0.0-37-generic, so I though it could be a kernel issue, as the issue looked similar to this one reported yesterday — no luck though, still no IP address or any internet connection from inside the container.
I tried setting the old IP address to the container with lxc config device set, but no luck either.
root@tardis-01.madpony.host:~# lxc config device set smith-01 lxdbr0 ipv4.address 10.171.234.205
root@tardis-01.madpony.host:~# lxc list
+----------+---------+------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+----------+---------+------+------+------------+-----------+
| smith-01 | RUNNING | | | PERSISTENT | 0 |
+----------+---------+------+------+------------+-----------+
I can give the container a dedicated IP address how the way I used to do it, but there is still no internet connection, since the bridge doesn’t seem to work.
root@tardis-01.madpony.host:~# ip route add 5.9.194.144/28 via 10.171.234.205 dev lxdbr0
root@tardis-01.madpony.host:~# lxc exec smith-01 -- bash
root@smith-01.madpony.space:~# ip addr add 5.9.194.144/28 dev eth0
root@smith-01.madpony.space:~# ping google.com
ping: google.com: Temporary failure in name resolution
So now I have no idea where to go.
Output of lxc config show smith-01 --expanded:
architecture: x86_64
config:
image.architecture: amd64
image.description: ubuntu 18.04 LTS amd64 (release) (20190604)
image.label: release
image.os: ubuntu
image.release: bionic
image.serial: "20190604"
image.version: "18.04"
limits.memory: 16GB
security.idmap.isolated: "true"
volatile.base_image: c234ecee3baaee25db84af8e3565347e948bfceb3bf7c820bb1ce95adcffeaa8
volatile.eth0.host_name: vethe5bca34d
volatile.eth0.hwaddr: 00:16:3e:55:a3:5c
volatile.idmap.base: "1065536"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'
volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'
volatile.last_state.power: RUNNING
volatile.lxdbr0.host_name: vethb3a18707
volatile.lxdbr0.hwaddr: 00:16:3e:45:9c:5b
volatile.lxdbr0.name: eth1
devices:
eth0:
name: eth0
nictype: bridged
parent: lxdbr0
type: nic
lxdbr0:
nictype: bridged
parent: lxdbr0
type: nic
root:
path: /
pool: tardis
size: 110GB
type: disk
ephemeral: false
profiles:
- smith
stateful: false
description: ""
The volatile.lxdbr0.name: eth1 part wasn’t something I seen before, I believe.
Output of lxc profile show smith:
config:
limits.memory: 16GB
security.idmap.isolated: "true"
description: LXD profile for Smith servers
devices:
eth0:
name: eth0
nictype: bridged
parent: lxdbr0
type: nic
root:
path: /
pool: tardis
size: 110GB
type: disk
name: smith
used_by:
- /1.0/containers/smith-01
Output of ifconfig lxdbr0:
lxdbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.171.234.1 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::5c6e:b1ff:fe04:5231 prefixlen 64 scopeid 0x20<link>
ether 02:9c:b6:5e:be:e1 txqueuelen 1000 (Ethernet)
RX packets 53 bytes 9904 (9.9 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 180 bytes 7848 (7.8 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Output of lxc info smith-01:
Name: smith-01
Location: none
Remote: unix://
Architecture: x86_64
Created: 2019/06/10 18:24 UTC
Status: Running
Type: persistent
Profiles: smith
Pid: 3908
Ips:
eth0: inet 5.9.194.144 vethe5bca34d
eth0: inet6 fe80::216:3eff:fe55:a35c vethe5bca34d
eth1: inet6 fe80::216:3eff:fe45:9c5b
lo: inet 127.0.0.1
lo: inet6 ::1
Resources:
Processes: 1131
Disk usage:
root: 6.57GB
CPU usage:
CPU usage (in seconds): 206
Memory usage:
Memory (current): 1.71GB
Memory (peak): 1.72GB
Network usage:
lo:
Bytes received: 1.97MB
Bytes sent: 1.97MB
Packets received: 23396
Packets sent: 23396
eth0:
Bytes received: 23.28kB
Bytes sent: 12.73kB
Packets received: 491
Packets sent: 48
eth1:
Bytes received: 34.77kB
Bytes sent: 1.15kB
Packets received: 523
Packets sent: 15
Hm, eth1 here too.
Looking at my bash_history I can see that I disabled IPv6 for the container, but since everything worked before and I did reboot this server a few times before this happened, I believe this is not the cause.
Still, these are the commands I used at the time:
2019-06-10 18:00:41 lxc network set lxdbr0 ipv6.address none
2019-06-10 18:31:29 lxc network set lxdbr0 ipv6.nat false
I created the bridge by running lxd init and LXD is installed using the Snap package, if that matters.
Before I forget, new containers created on this server also lack IP addresses and internet connections.
root@tardis-01.madpony.host:~# lxc launch ubuntu:18.04 testing-network
Creating testing-network
Starting testing-network
root@tardis-01.madpony.host:~# lxc list
+-----------------+---------+--------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+--------------------+------+------------+-----------+
| smith-01 | RUNNING | 5.9.194.144 (eth0) | | PERSISTENT | 0 |
+-----------------+---------+--------------------+------+------------+-----------+
| testing-network | RUNNING | | | PERSISTENT | 0 |
+-----------------+---------+--------------------+------+------------+-----------+
Thanks in advance!