Container and arp reply

Hello,

I’ve a raspberry pi running Raspbian Buster with a fresh installation of LXD (3.22).
This raspberry pi has a static ip adress configured into /etc/dhcpcd.conf

I did an LXD init, created a profile, and linked my storage pool to it :

lxc profile list
+----------+---------+
|   NAME   | USED BY |
+----------+---------+
| default  | 0       |
+----------+---------+
| rasticot | 3       |
+----------+---------+

My profile got a macvlan configuration

lxc profile show rasticot
config:
  boot.autostart: "0"
description: Rasticot's profile
devices:
  eth0:
    nictype: macvlan
    parent: eth0
    type: nic
  root:
    path: /
    pool: storage
    type: disk
name: rasticot
used_by:
- /1.0/instances/najedha
- /1.0/instances/endor
- /1.0/instances/tython

The LXD configuration is the same in my other hosts, the difference is the distribution used (Ubuntu 18.04 vs Raspbian)
All of my containers are getting an IP adress from my dhcp server, which is also running inside a container on the same network

Problem


All of my containers installed into this host (raspbian) are getting trouble replying to arp request.

Example

If I do a ping from another computer into the same network

Envoi d’une requête 'Ping'  192.168.0.55 avec 32 octets de données :
Réponse de 192.168.0.110 : Impossible de joindre l’hôte de destination.

The host can see the right traffic, but the container never answer to that broadcast request.

sudo tcpdump arp host 192.168.0.110
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:47:18.749542 ARP, Request who-has 192.168.0.55 tell 192.168.0.110, length 46
13:47:19.554767 ARP, Request who-has 192.168.0.55 tell 192.168.0.110, length 46
13:47:20.554783 ARP, Request who-has 192.168.0.55 tell 192.168.0.110, length 46
13:47:21.562626 ARP, Request who-has 192.168.0.55 tell 192.168.0.110, length 46
13:47:22.554941 ARP, Request who-has 192.168.0.55 tell 192.168.0.110, length 46
13:47:23.555344 ARP, Request who-has 192.168.0.55 tell 192.168.0.110, length 46

Now if I go on the container and capture the same trafic

$ lxc exec tython bash
$ root@tython:~# tcpdump arp host 192.168.0.110
13:51:22.808657 ARP, Request who-has tython tell 192.168.0.110, length 46
13:51:22.808783 ARP, Reply tython is-at 00:16:3e:2a:4d:3c (oui Unknown), length 28

Envoi d’une requête 'Ping'  192.168.0.55 avec 32 octets de données :
Réponse de 192.168.0.55 : octets=32 temps<1ms TTL=64
Réponse de 192.168.0.55 : octets=32 temps<1ms TTL=64

The container is able to answer as long as I’m connected on it and use tcpdump to display traffic.
Of course, if I send a ping to 192.168.0.110 from my container, arp table of my computer will be adjusted and the container will be reacheable.

Here is the configuration of this container

lxc config show --expanded tython
architecture: armv7l
config:
  boot.autostart: "0"
  image.architecture: armhf
  image.description: ubuntu 18.04 LTS armhf (release) (20200317)
  image.label: release
  image.os: ubuntu
  image.release: bionic
  image.serial: "20200317"
  image.type: squashfs
  image.version: "18.04"
  security.privileged: "true"
  volatile.base_image: 38acc14580aa421c9499dd64763c0ebed42b39b42612ca55bbd347ad488a0480
  volatile.eth0.host_name: macef311301
  volatile.eth0.hwaddr: 00:16:3e:2a:4d:3c
  volatile.eth0.last_state.created: "false"
  volatile.eth0.name: eth0
  volatile.idmap.base: "0"
  volatile.idmap.current: '[]'
  volatile.idmap.next: '[]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: STOPPED
devices:
  eth0:
    nictype: macvlan
    parent: eth0
    type: nic
  root:
    path: /
    pool: storage
    type: disk
ephemeral: false
profiles:
- rasticot
stateful: false
description: ""

Don’t really know where to look at, it seems to be a problem with my macvlan configuration, at least with this specific distribution.

When I use Wireshark to capture traffic on my LAN Network, there is no answer to the ARP request send by my computer. But why it works if I connect to the container and do any network manipulation ?

The hosts is blocking the traffic between the physical and the macvlan interface. How to debug the problem ?

Thank you !

Strange!

My first thought would be to check the firewall on the host, check for things like OUTPUT rules and NAT rules that may be altering the packets coming from your container.

I’m not quite clear on how you got ping to work from your container though, can you show a network capture of the ARP resolution occurring to allow that please.

My first thought would be to check the firewall on the host, check for things like OUTPUT rules and NAT rules that may be altering the packets coming from your container.

I’m not very comfortable with firewall rules under linux but , the rules applied in my container are :

root@tython:~# root@tython:~# ufw status
Status: inactive 

root@tython:~# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT A

root@tython:~# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

It seems that there is no rule that bloc the traffic, everything seems to be open.

I’m not quite clear on how you got ping to work from your container though, can you show a network capture of the ARP resolution occurring to allow that please.

When my container boot, it makes a DHCP request an got an IP from my dhcp-server.

root@tython:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.55  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::216:3eff:fe2a:4d3c  prefixlen 64  scopeid 0x20<link>
        ether 00:16:3e:2a:4d:3c  txqueuelen 1000  (Ethernet)
        RX packets 112844  bytes 149998407 (149.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 171054  bytes 168622733 (168.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@tython:~# ip route
default via 192.168.0.1 dev eth0 proto dhcp src 192.168.0.55 metric 100
192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.55
192.168.0.1 dev eth0 proto dhcp scope link src 192.168.0.55 metric 100

TCPDUMP from the host


Now, if I’m not remotely connected to the container, I wont be able to contact it, even though the arp request is reaching the host. 192.168.0.110 is my personal computer that trigger the request, rasticit is the host.

yanover@rasticit:~ $ sudo tcpdump arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:18:17.927708 ARP, Request who-has 192.168.0.55 tell 192.168.0.110, length 46
12:18:22.313884 ARP, Request who-has 192.168.0.55 tell 192.168.0.110, length 46
12:18:22.928232 ARP, Request who-has 192.168.0.55 tell 192.168.0.110, length 46

TCPDUMP from the container


Now, I do a capture of the arp packets directly from the container

yanover@rasticit:~ $ lxc exec tython bash
root@tython:~# tcpdump arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:22:08.557021 ARP, Request who-has tython tell 192.168.0.110, length 46
12:22:08.557163 ARP, Reply tython is-at 00:16:3e:2a:4d:3c (oui Unknown), length 28

And directly after this command, the container answer to the ARP request and it becomes reachable. It seems that as long as I’m not connected to the container, eth0 on the host doesn’t forward request to the container … Or the container is not listening to the network interface ?

I tried to run tcpdump in background mode and I closed the cli, the container still answer. Maybe this test can valide the fact that the host is forwarding packet to macvlan ?

I hope that explains it a little better.

Thank you for your reply !

Ok guys,

After a few more hours of research, I found these two thread :

It seems to be a kernel issu with macvlan and my Raspbian distribution, I’ve just updated the Kernel and it works !

rpi-update
reboot now

This thread can be mark as resolved !

Excellent good find. :slight_smile: