Lxd vm has no internet

Hello Guys,

I am using my system bridge for my lxd setup;

devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: br0
    type: nic

This configs are the same on my setup for containers and vms.
All vms and containers gets an ip from my network.
Containers only seems to have full internet activities. But Vms do not. I.e vms can communicate with other ips on the same network, but not with external network/internet.

I wonder if anybody came accross same issue here, i appreciate help.

1 Like

Hi,
Can you share the output of those commands?
Regards.

lxc network ls
lxc network show lxdbr0
lxc info | grep -i firewall

I am using the host bridge br0 the profile entry for the bridge is in the first comment.

lxc info | grep -i firewall
- network_firewall_filtering
- firewall_driver
  firewall: nftables

All hosts and vms get ip, resolve dns its just the package manager+ browsers has no internet activities on the vms only.
I.e this vm get a regular ips from my bridge.

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:16:3e:9c:67:39 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.198/24 brd 10.0.0.255 scope global enp5s0
       valid_lft forever preferred_lft forever
    inet6 fd07:f139:3a65:0:735f:fe7c:66ef:60f8/64 scope global temporary dynamic 
       valid_lft 604013sec preferred_lft 85521sec
    inet6 fd07:f139:3a65:0:216:3eff:fe9c:6739/64 scope global dynamic mngtmpaddr 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe9c:6739/64 scope link 

Here is the vm profile, which is the same as other containers profile which like i said containers has no issue.

config:
  limits.cpu: "2"
  limits.memory: 1024MB
  limits.memory.swap: "false"
  limits.memory.swap.priority: "0"
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: br0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: default

And here are the vm configs;

architecture: x86_64
config:
  image.architecture: amd64
  image.description: Opensuse tumbleweed amd64 (20221104_04:21)
  image.os: Opensuse
  image.release: tumbleweed
  image.serial: "20221104_04:21"
  image.type: disk-kvm.img
  image.variant: desktop-kde
  limits.cpu: "2"
  limits.memory: 3GiB
  volatile.base_image: aa4c3adacc87e60978524446d6ebc21e6cc4b63bb2e849b9d745fb0cdcb6ca5c
  volatile.cloud-init.instance-id: a30c9df2-bd0e-4090-b661-8053ed94ffc6
  volatile.eth0.host_name: tap4b3a221c
  volatile.eth0.hwaddr: 00:16:3e:9c:67:39
  volatile.last_state.power: RUNNING
  volatile.uuid: ac71df1e-717a-4e89-a45d-ea7a75cc0203
  volatile.vsock_id: "82"
devices: {}
ephemeral: false
profiles:
- default
stateful: false
description: ""
created_at: 2022-11-06T17:21:55.564055869Z
name: susekde
status: Running
status_code: 103
last_used_at: 2022-11-06T17:22:16.520437181Z
location: none
type: virtual-machine

Please post the ip r command output on the vm?
Regards.

ip r                 
default via 10.0.0.1 dev enp5s0 
10.0.0.0/24 dev enp5s0 proto kernel scope link src 10.0.0.198 

Hi @seven , interesting, which distributions are those vms? And have you ever try with different linux distribution?
Regards.

Hi @cemzafer ,
That one is opensues/kde, i also tried arch, and ubuntu.
I have also tried creating a vm without descktop, same issue.

Humm, I havent figure it out, sorry. Maybe @stgraber and @tomp has any idea what could be the problem.
Regards.

Please show ip a and ip r on the LXD host and inside a container and a VM.

Thanks

HI @tomp ,
Apology for being late here;
Host;

ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether MAC brd ff:ff:ff:ff:ff:ff
    altname wlp7s0
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
    link/ether MAC brd ff:ff:ff:ff:ff:ff
    altname enp0s31f6
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default qlen 1000
    link/ether MAC brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.100/24 brd 10.0.0.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::2e56:dcff:fe3f:1c9d/64 scope link 
       valid_lft forever preferred_lft forever
5: lxdbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 00:16:3e:10:ce:8e brd ff:ff:ff:ff:ff:ff
    inet 11.11.11.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
7: veth7c504c27@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether 52:43:c2:a0:4d:a4 brd ff:ff:ff:ff:ff:ff link-netnsid 0
9: veth27578b56@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether 26:f3:75:7c:7f:91 brd ff:ff:ff:ff:ff:ff link-netnsid 1
11: veth83927a4a@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether 82:a9:3f:c6:1c:6d brd ff:ff:ff:ff:ff:ff link-netnsid 2
13: vethdc60dad7@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether 82:57:c9:3a:2c:78 brd ff:ff:ff:ff:ff:ff link-netnsid 3
15: veth0766c98f@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether 7e:1a:6c:71:b5:4d brd ff:ff:ff:ff:ff:ff link-netnsid 4
17: veth157d28a0@if16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether e2:d2:e8:64:6a:49 brd ff:ff:ff:ff:ff:ff link-netnsid 5
18: tap09a96cdd: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc mq master br0 state UP group default qlen 1000
    link/ether c2:c3:b4:0c:46:d2 brd ff:ff:ff:ff:ff:ff
19: tape50036f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc mq master br0 state UP group default qlen 1000
    link/ether 6a:cc:86:0c:43:f9 brd ff:ff:ff:ff:ff:ff
21: vetha87eeda2@if20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether 92:47:bb:66:15:f2 brd ff:ff:ff:ff:ff:ff link-netnsid 6
25: veth506a1ea4@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether b6:67:60:ee:98:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 9


ip r
default via 10.0.0.1 dev br0 
10.0.0.0/24 dev br0 proto kernel scope link src 10.0.0.100 
11.11.11.0/24 dev lxdbr0 proto kernel scope link src 11.11.11.1 linkdown

VM;
lxc exec ubuntu ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:16:3e:0b:8e:47 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.248/24 brd 10.0.0.255 scope global dynamic enp5s0
       valid_lft 41200sec preferred_lft 41200sec
    inet6 fd07:f139:3a65:0:216:3eff:fe0b:8e47/64 scope global mngtmpaddr noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe0b:8e47/64 scope link 
       valid_lft forever preferred_lft forever

lxc exec ubuntu ip r
default via 10.0.0.1 dev enp5s0 proto dhcp src 10.0.0.248 metric 100 
10.0.0.0/24 dev enp5s0 proto kernel scope link src 10.0.0.248 
10.0.0.1 dev enp5s0 proto dhcp scope link src 10.0.0.248 metric 100 

Container;
lxc exec fed35 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
24: eth0@if25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:c4:ba:58 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.0.0.246/24 metric 1024 brd 10.0.0.255 scope global dynamic eth0
       valid_lft 42437sec preferred_lft 42437sec
    inet6 fd07:f139:3a65:0:216:3eff:fec4:ba58/64 scope global mngtmpaddr noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fec4:ba58/64 scope link 
       valid_lft forever preferred_lft forever

lxc exec fed35 ip r
default via 10.0.0.1 dev eth0 proto dhcp src 10.0.0.246 metric 1024 
10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.246 metric 1024 
10.0.0.1 dev eth0 proto dhcp scope link src 10.0.0.246 metric 1024 

Can you ping 10.0.0.246 and 10.0.0.1 from the VM?

Yes i can.

Oh I spotted the issue. Your br0 has a lower MTU. Its set to 1400.

Containers just use a veth pair connected to the bridge and so inherit the MTU of the bridge on both ends of the veth pair and it gets passed through into the container.

Whereas VMs have their own kernel, this means the bridge’s MTU isn’t getting passed through.

For managed LXD bridges, LXD supplies the bridge’s MTU in the DHCP options. However as you’re using your own bridge, it is up to you to either manually set the MTU for the interface inside the VM or to add the relevant DHCP option to your DHCP server (if you are using one and don’t mind that being sent to all devices).

1 Like