I got similar to Nictype=routed: network is not working with Ubuntu 18.04 (but ok with Centos 7) issue with centos 7 now on LXD 4.1 from snap:
ct=c991
ostemplate="centos/7"
ip="..."
lxc launch images:${ostemplate} ${ct}
lxc stop ${ct}
lxc config device add ${ct} root disk path=/ pool=lxd size=${disk}
lxc config device add $ct eth0 nic nictype=routed parent=eno1 ipv4.address=$ip
lxc start ${ct}
Inside ct:
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0@if31: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 22:3e:50:77:3d:9e brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 194.117.***.***/32 brd 255.255.255.255 scope global eth0
valid_lft forever preferred_lft forever
and network is not working, I’m unable to ping even 8.8.8.8.
I see:
# ps auxfw
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 223 0.0 0.0 115524 2192 ? Ss 00:01 0:00 bash
root 232 0.0 0.0 155448 2048 ? R+ 00:01 0:00 \_ ps auxfw
root 1 0.0 0.0 43284 3468 ? Ss 00:01 0:00 /sbin/init
root 38 0.0 0.0 39092 5368 ? Ss 00:01 0:00 /usr/lib/systemd/systemd-journald
root 51 0.0 0.0 41536 1908 ? Ss 00:01 0:00 /usr/lib/systemd/systemd-udevd
root 59 0.0 0.0 12100 2172 ? Ss 00:01 0:00 /bin/bash /etc/rc.d/init.d/network start
root 183 0.0 0.0 12096 2292 ? S 00:01 0:00 \_ /bin/bash /etc/sysconfig/network-scripts/ifup-eth ifcfg-eth0.deleted boot
root 199 0.0 0.0 102908 5812 ? S 00:01 0:00 \_ /sbin/dhclient -1 -q -lf /var/lib/dhclient/dhclient--eth0.lease -pf /var/run/dhcl
root 61 0.0 0.0 24268 1756 ? Ss 00:01 0:00 /usr/lib/systemd/systemd-logind
dbus 63 0.0 0.0 58012 2536 ? Ss 00:01 0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activatio
root 71 0.0 0.0 22704 1808 ? Ss 00:01 0:00 /usr/sbin/crond -n
root 76 0.0 0.0 6524 1116 console Ss+ 00:01 0:00 /sbin/agetty --noclear --keep-baud console 115200,38400,9600 linux
so it looks like some problem with dhclient, but unfortunately I was not able to force it not touching network like I did to Ubuntu.
Does anybody have this issue? How to fix it? Thanks
tomp
June 9, 2020, 7:53am
2
Take a look at the config files in /etc/sysconfig/network-scripts/ as that is where they are stored for CentOS and your ps output shows thats what is running dhclient.
tomp
June 9, 2020, 9:15am
3
It looks like its still trying to process ifcfg-eth0.deleted so suggest removing that file entirely.
lbg74
June 9, 2020, 12:32pm
4
Hi. @Nick_Knutov and @tomp
I also tested it.
@tomp , 2 questions.
When adding a nic device, if i add the device name as eth2, if eth1 does not exist in the container, is it automatically registered as eth1 in the container?
Is routed nic not available in vm?
Thank you.
tomp
June 9, 2020, 1:16pm
5
Seems to work OK, here’s my process:
lxc init images:centos/7 c1
lxc config device add c1 eth0 nic nictype=routed ipv4.address=192.168.1.200 parent=enp3s0
lxc start
lxc ls c1
+------+---------+----------------------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+---------+----------------------+------+-----------+-----------+
| c1 | RUNNING | 192.168.1.200 (eth0) | | CONTAINER | 0 |
+------+---------+----------------------+------+-----------+-----------+
OK good, so container has IP without me doing anything.
But lets check connectivity:
ing 192.168.1.200 -c2 -W1
PING 192.168.1.200 (192.168.1.200) 56(84) bytes of data.
--- 192.168.1.200 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1006ms
Oh dear, something isn’t right. Lets take a look at the container’s routes:
lxc exec c1 -- ip r
OK, no output, this means no default route. This is our problem.
lxc exec c1 -- cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
HOSTNAME=c1
NM_CONTROLLED=no
TYPE=Ethernet
MTU=
DHCP_HOSTNAME=c1
Change BOOTPROTO=dhcp to BOOTPROTO=none.
lxc restart c1
ping 192.168.1.200 -c2 -W1
PING 192.168.1.200 (192.168.1.200) 56(84) bytes of data.
64 bytes from 192.168.1.200: icmp_seq=1 ttl=64 time=0.403 ms
64 bytes from 192.168.1.200: icmp_seq=2 ttl=64 time=0.065 ms
--- 192.168.1.200 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1022ms
rtt min/avg/max/mdev = 0.065/0.234/0.403/0.169 ms
Working
tomp
June 9, 2020, 1:17pm
6
If you want to be sure what the device name will be inside the container, be sure to specify name in the device config, e.g.
lxc config device add c1 mynic nic nictype=routed name=eth2 ipv4.address=192.168.1.200 parent=enp3s0
tomp
June 9, 2020, 1:21pm
7
Ah, but the routing still isn’t quite right:
lxc exec c1 -- ip r
169.254.0.0/16 dev eth0 scope link metric 1002
That gets us to the default gateway, and thus the LXD host, but not onto the wider network.
tomp
June 9, 2020, 1:36pm
8
Try adding this to the container as well as the change above.
lxc exec c1 -- cat /etc/sysconfig/network-scripts/route-eth0
169.254.0.1/128 dev eth0
default via 169.254.0.1 dev eth0
1 Like
lbg74
June 9, 2020, 1:39pm
9
Thank you @tomp
Please take a look at my container’s configuration first.
What is wrong with my configuration?
tomp
June 9, 2020, 1:49pm
11
Also, which device are you wanting to be your default gateway device (assuming the static route gets added correctly, you cant have lxdbr0 provide a default route by DHCP and have one defined statically).
So aside from the particularities of CentOS 7 network setup, this configuration is unlikely to work properly as you have two competing NICs.
lbg74
June 9, 2020, 2:05pm
12
Thank you @tomp .
As you said, if i add routed nic as eth0 and write route-eth0, it will ping from the container to the outside (8.8.8.8).
However, adding routed-nic as eth1 doesn’t seem to work.
However, in case of centos 8, by default, only routed-nic is added, and external communication is good.
Please refer to the picture below.
tomp
June 9, 2020, 2:07pm
13
I’m not clear what you are trying to achieve.
Your container config shows two NICs; eth0 and mynic, which will appear as eth0 and eth1 respectively inside the container.
Which one of those should be the default gateway device?
lbg74
June 9, 2020, 2:40pm
14
Thank you @tomp .
Add routed-nic in centos 7 and 8 and look at netstat -an as below.
After creating /etc/sysconfig/network-scripts/route-eth1 in centos 7 and rebooting, 169.254.0.1 does not register as the default gateway.
So, i register 169.254.0.1 as the default gw with the route add command in centos7, external communication is good.
Thanks again for letting me know in detail.
1 Like
