ianm
July 21, 2021, 5:02pm
1
I’m setting up some containers on a host system that only has ipv6 link-local connectivity. I cannot connect from directly connected PC to the container’s global ipv6 address unless I add the host’s physical interface to the lxcbr0 bridge (but then I lose the ability to address the host).
I am able to communicate from the host to the containers global scope. ip route get tells me the proper route to get to the container’s IP address
ip -6 route get fc27::216:3eff:fe37:a04e from fe8
I’ve got a crude photo of how the network architecture looks
I tried to setup ipv6 NAT but it doesn’t seem to work properly. When I DNAT to an ip address inside my container the packet gets discarded after prerouting:
For example, here’s a trace of a packet traversing ip6tables with DNAT to a container’s IP:
root@root:/# ip6tables -t nat -I PREROUTING -p tcp --dport 808
It is discarded at the first routing decision after PREROUTING
And here’s a trace when the DNAT is set to the bridge IP:
It succesfully traverses the appropriate tables.
Any advice is appreciated
tomp
July 21, 2021, 6:27pm
2
Can you show ip a and ip r on the host and inside the container please.
ianm
July 21, 2021, 7:02pm
3
root@root:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
link/sit 0.0.0.0 brd 0.0.0.0
4: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master0
link/ether 00:04:9f:78:bc:06 brd ff:ff:ff:ff:ff:ff
inet6 fe80::204:9fff:fe78:bc06/64 scope link
valid_lft forever preferred_lft forever
16: lxcbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP q0
link/ether 00:16:3e:00:00:00 brd ff:ff:ff:ff:ff:ff
inet 10.0.3.1/24 scope global lxcbr0
valid_lft forever preferred_lft forever
inet6 fc27::216:3eff:fe00:1/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::216:3eff:fe00:0/64 scope link
valid_lft forever preferred_lft forever
24: vethDKVTO5@if23: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc no0
link/ether fe:c1:1b:c2:1d:74 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcc1:1bff:fec2:1d74/64 scope link
valid_lft forever preferred_lft forever
26: veth6D5UE7@if25: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc no0
link/ether fe:42:1a:41:29:63 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc42:1aff:fe41:2963/64 scope link
valid_lft forever preferred_lft forever
root@root:~# ip r
10.0.3.0/24 dev lxcbr0 scope link src 10.0.3.1
ip -6 r
anycast fc27:: dev lxcbr0 metric 0
fc27::/64 dev lxcbr0 metric 256
anycast fe80:: dev eth0 metric 0
anycast fe80:: dev lxcbr0 metric 0
anycast fe80:: dev vethDKVTO5 metric 0
anycast fe80:: dev veth6D5UE7 metric 0
fe80::/64 dev eth0 metric 256
fe80::/64 dev lxcbr0 metric 256
fe80::/64 dev vethDKVTO5 metric 256
fe80::/64 dev veth6D5UE7 metric 256
multicast ff00::/8 dev eth0 metric 256
multicast ff00::/8 dev lxcbr0 metric 256
multicast ff00::/8 dev vethDKVTO5 metric 256
multicast ff00::/8 dev veth6D5UE7 metric 256
tomp
July 21, 2021, 7:10pm
4
I don’t think you can do what you’re trying to achieve as link-local IPv6 addresses cannot be routed (even if you’re NATing it).
Why don’t you give your containers an address in the fc27::216:3eff:fe00::/64 range the same as your host’s lxcbr0 interface has?
ianm
July 21, 2021, 7:11pm
5
Oh sorry, I forgot to run the commands in the container
root@foo:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
link/sit 0.0.0.0 brd 0.0.0.0
23: eth0@if24: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 0
link/ether 00:16:3e:37:a0:4e brd ff:ff:ff:ff:ff:ff
inet6 fc27::216:3eff:fe37:a04e/64 scope global dynamic
valid_lft 3124sec preferred_lft 3124sec
inet6 fe80::216:3eff:fe37:a04e/64 scope link
valid_lft forever preferred_lft forever
root@foo:~# ip -r
BusyBox v1.31.1 () multi-call binary.
Usage: ip [OPTIONS] address|route|link|tunnel|neigh|rule [ARGS]
root@foo:~# ip r
root@foo:~# ip -6 r
fc27::/64 dev eth0 metric 256 expires 3118sec
fe80::/64 dev eth0 metric 256
multicast ff00::/8 dev eth0 metric 256
default via fe80::216:3eff:fe00:0 dev eth0 metric 1024 expires 1318sec
ianm
July 21, 2021, 7:23pm
6
If I bridge my eth0 host interface with lxcbr0 on the host and then on the connecting computer add a route for fc27::/64 I have connectivity to the system. It seems I can use lxcbr0’s ip address in the same way I was using eth0’s address previously.
Does this sound correct?
tomp
July 22, 2021, 10:53am
7
I’m still a little confused by your setup.
What is “host PC” and “client” in your original diagram, are they separate machines on the network?
Why are you using link-local everywhere, and why does your lxcbr0 have a non-link-local address though?
ianm
July 22, 2021, 2:13pm
8
Hi Tomas,
In the Diagram the ‘Client’ is an embedded linux device running LXC directly connected via ethernet to the ‘host PC’
lxcbr0 has a non-link-local address because I was experimenting with lxc-net script.
link-local is used I believe to keep the client from being reachable by the wider network.
I’ve found now that I can in fact use only a link-local address with my bridge, I was using the wrong scopeID in my previous tests.
If I add my eth0 interface to the bridge, I can address it’s link-local address from my ‘host pc’ in the diagram via its link-local address, as well as my containers’ link-local addresses.
I believe that is exactly what I need, at least for now
