IPv6 not working for containers from the outside, but works from host and other containers

Incus
,
1

I have a Scaleway host running Arch Linux with a /64 IPv6 network, but it’s broken other than internal communication between containers and host, and I am not very proficient with (IPv6) networking to be able to tell why.

# incus network show lxdbr0
config:
  ipv4.address: 10.49.174.1/24
  ipv4.nat: "true"
  ipv6.address: 2001:bc8:1640:1c45:dc00:ff:fe27:b697/64
  ipv6.nat: "false"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/bree
- /1.0/instances/mail
- /1.0/profiles/default
managed: true
status: Created
locations:
- none
project: default

I have Arch Linux containers that can ping between themselves.
I can ping containers from host.
I can ping host from outside.
I can access outside from the host.
I cannot ping containers from outside.
I cannot access outside from the containers.

Host sc1:

[0] # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether de:00:00:27:b6:97 brd ff:ff:ff:ff:ff:ff
    altname enp0s2
    inet 51.15.117.74/32 metric 1024 scope global dynamic ens2
       valid_lft 504sec preferred_lft 504sec
    inet6 2001:bc8:1640:1c45:dc00:ff:fe27:b697/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 86399sec preferred_lft 14399sec
    inet6 fe80::dc00:ff:fe27:b697/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none 
    inet 10.200.200.1/24 scope global wg0
       valid_lft forever preferred_lft forever
4: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:d0:18:92 brd ff:ff:ff:ff:ff:ff
    inet 10.49.174.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet6 2001:bc8:1640:1c45:dc00:ff:fe27:b697/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fed0:1892/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
6: veth99258b44@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether 56:4a:ec:0b:87:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0
8: veth4842d6cd@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether ce:7b:c6:96:d0:f6 brd ff:ff:ff:ff:ff:ff link-netnsid 1

[0] # ip r
default via 62.210.0.1 dev ens2 proto dhcp src 51.15.117.74 metric 1024 
10.49.174.0/24 dev lxdbr0 proto kernel scope link src 10.49.174.1 
10.200.200.0/24 dev wg0 proto kernel scope link src 10.200.200.1 
51.158.139.25 via 62.210.0.1 dev ens2 proto dhcp src 51.15.117.74 metric 1024 
51.158.139.28 via 62.210.0.1 dev ens2 proto dhcp src 51.15.117.74 metric 1024 
62.210.0.1 dev ens2 proto dhcp scope link src 51.15.117.74 metric 1024 

[0] # ip -6 r
2001:bc8:1640:1c45::/64 dev lxdbr0 proto kernel metric 256 pref medium
2001:bc8:1640:1c45::/64 dev ens2 proto ra metric 1024 expires 86393sec pref medium
fe80::/64 dev ens2 proto kernel metric 256 pref medium
fe80::/64 dev lxdbr0 proto kernel metric 256 pref medium
default via fe80::dc00:ff:fe27:b698 dev ens2 proto ra metric 1024 expires 23sec pref medium

Host is using a simple .network file for systemd-networkd:

[Match]
Name=ens2

[Network]
DHCP=yes

Container bree:

[0] # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host proto kernel_lo 
       valid_lft forever preferred_lft forever
5: eth0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:42:29:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.49.174.11/24 metric 1024 brd 10.49.174.255 scope global dynamic eth0
       valid_lft 2503sec preferred_lft 2503sec
    inet6 2001:bc8:1640:1c45:216:3eff:fe42:2947/64 scope global mngtmpaddr noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe42:2947/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever                                                                                                                                                                                                                   

[0] # ip r
default via 10.49.174.1 dev eth0 proto dhcp src 10.49.174.11 metric 1024 
10.49.174.0/24 dev eth0 proto kernel scope link src 10.49.174.11 metric 1024 
10.49.174.1 dev eth0 proto dhcp scope link src 10.49.174.11 metric 1024 

[0] # ip -6 r 
2001:bc8:1640:1c45::/64 dev eth0 proto kernel metric 256 pref medium
2001:bc8:1640:1c45::/64 dev eth0 proto ra metric 1024 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::216:3eff:fed0:1892 dev eth0 proto ra metric 1024 expires 1551sec hoplimit 64 pref medium

How I tested:

OUTSIDE-HOST->WORKS # ping -6 2001:bc8:1640:1c45:dc00:ff:fe27:b697
OUTSIDE-CONTAINER->BROKEN # ping -6 '2001:bc8:1640:1c45:216:3eff:fe42:2947'
2

Apparently I was missing ipv6.routing: "true".

Found out thanks to OP in Lxd ipv6 interface and bridge in same ipv6 network allowed or not? .

EDIT: Apparently something else is also at play because after a full reboot it stopped working, now to figure what else it is that I did from shell history…

EDIT2: So actually what helped was enabling NDP Proxy on host and manually registering the container, I have no clue what is going on or how to automate this. This works even after removing the above ipv6.routing config.

sysctl net.ipv6.conf.ens2.proxy_ndp=1
ip -6 neigh add proxy 2001:bc8:1640:1c45:216:3eff:fe42:2947 dev ens2

Additionally I changed the bridge to have a different IPv6 from the ens2 interface.

  ipv6.address: 2001:bc8:1640:1c45::1/64

EDIT 3: So I just added the proxy adds for each container to a shell script that runs on boot as a lazy solution, one can also use ndppd as per How to Assign IPv6 Addresses to LXD Containers on a VPS | Ryan Young but I do not get why I have to do this as my /64 block is supposedly routed to the host according to Scaleway?

2 Likes