Ipvlan Interface Integration with LXC Containers and routing problem

LXC
,
1

Hello, I am having trouble understanding how to create an ipvlan interface and integrate it into an LXC container.

System: Debian 11
Host:

root@DUO:/srv/lxc # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0a:9d:f6:0f:d7 brd ff:ff:ff:ff:ff:ff
    inet 192.168.9.3/24 brd 192.168.9.255 scope global enp3s0
       valid_lft forever preferred_lft forever
    inet6 fe80::20a:9dff:fef6:fd7/64 scope link
       valid_lft forever preferred_lft forever
13: ipvl-jelly@enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 00:0a:9d:f6:0f:d7 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a:9d00:af6:fd7/64 scope link
       valid_lft forever preferred_lft forever
root@DUO:/srv/lxc #

The host system has no bridge, ipv4 forwarding is activated.

The LXC container is also Debian, and a sample config looks like this:

#vmJellyfin
lxc.hook.pre-start = /usr/bin/bash -c "ip link add link enp3s0 name ipvl-jelly type ipvlan mode l2"
lxc.hook.post-stop = /usr/bin/bash -c "ip link delete ipvl-jelly"
lxc.net.0.type = ipvlan
lxc.net.0.ipvlan.mode=l2
#lxc.net.0.ipvlan.flags=bridge
lxc.net.0.link = ipvl-jelly
#lxc.net.0.l2proxy = 1
lxc.net.0.ipv4.address = 192.168.9.82/32
lxc.net.0.ipv4.gateway = 192.168.9.2
lxc.net.0.flags = up
lxc.apparmor.profile = unconfined
lxc.apparmor.allow_nesting = 1
lxc.rootfs.path = dir:/var/lib/lxc/vmJellyfin/rootfs
lxc.uts.name = vmJellyfin
lxc.arch = amd64
lxc.pty.max = 1024

In lxc.hook.pre-start, the ipvlan device is created, then no matter what is assigned to lxc.net.0.type or ipvlan.mode, contianer works.

root@DUO:/srv/lxc # lxc-ls -f
NAME                       STATE   AUTOSTART GROUPS IPV4         IPV6 UNPRIVILEGED
vmJellyfin                 RUNNING 0         -      192.168.9.82 -    false
vmLeantime                 STOPPED 0         -      -            -    false

Question 1: Is my LXC network configuration correct?

Question 2: Why is there no network communication between the host and the container?
I’ve been playing around with namespaces, but it doesn’t work with the container.
See Example: https://www.kernel.org/doc/Documentation/networking/ipvlan.txt

2

Your LXC network configuration looks correct. However, it’s hard to determine the reason for the lack of network communication between the host and container without more information. Please can you do the followings:

  • Check if the ipvlan interface is up and has an IP address on the host by running ip addr show ipvl-jelly . If not, try bringing it up with sudo ip link set dev ipvl-jelly up
  • Check if the container has received the correct IP address and gateway by running lxc-info -n vmJellyfin and look for the ipv4 and gw fields under network.
  • Try pinging the container from the host and vice versa using their respective IP addresses. If you can’t ping the container, try disabling any firewalls on the host and container temporarily to see if that’s causing the issue.
  • Check if there are any error messages in the syslog or dmesg related to networking when starting the container.
  • Also, my last idea would be to ensure that your kernel has support for IPVLAN enabled.
3

1,2,3

root@DUO:/srv/lxc # uname -a
Linux DUO 6.1.0-0.deb11.7-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.20-2~bpo11+1 (2023-04-23) x86_64 GNU/Linux

set device up

root@DUO:/srv/lxc # ip link set dev ipvl-jelly up
root@DUO:/srv/lxc # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0a:9d:f6:0f:d7 brd ff:ff:ff:ff:ff:ff
    inet 192.168.9.3/24 brd 192.168.9.255 scope global enp3s0
       valid_lft forever preferred_lft forever
    inet6 fe80::20a:9dff:fef6:fd7/64 scope link
       valid_lft forever preferred_lft forever
20: ipvl-jelly@enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 00:0a:9d:f6:0f:d7 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a:9d00:11f6:fd7/64 scope link
       valid_lft forever preferred_lft forever
root@DUO:/srv/lxc #

in fact stat is UNKNOW , but i dont have idea how change them.

root@DUO:/srv/lxc # lxc-info -n vmJellyfin
Name:           vmJellyfin
State:          RUNNING
PID:            25874
IP:             192.168.9.82
Link:           ipvl-jelly
 TX bytes:      21.79 KiB
 RX bytes:      1.11 KiB
 Total bytes:   22.90 KiB
root@DUO:/srv/lxc #

The container works fine , but route to host System not works.
log from 2 times start/stop container

root@DUO:/srv/lxc # tail -f /var/log/syslog
2023-05-30T12:57:32.159700+02:00 DUO systemd[1]: Starting apt-daily.service - Daily apt download activities...
2023-05-30T12:57:32.889611+02:00 DUO systemd[1]: apt-daily.service: Deactivated successfully.
2023-05-30T12:57:32.890073+02:00 DUO systemd[1]: Finished apt-daily.service - Daily apt download activities.
2023-05-30T13:13:37.708800+02:00 DUO kernel: [99328.500168] nfsd: last server has exited, flushing export cache
2023-05-30T13:13:51.124822+02:00 DUO kernel: [99341.916701] eth0: renamed from ipSIu0mI
2023-05-30T13:13:51.188825+02:00 DUO kernel: [99341.982867] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
2023-05-30T13:13:52.941314+02:00 DUO kernel: [99343.731694] NFSD: attempt to initialize umh client tracking in a container ignored.
2023-05-30T13:13:52.941342+02:00 DUO kernel: [99343.731709] NFSD: attempt to initialize legacy client tracking in a container ignored.
2023-05-30T13:13:52.941345+02:00 DUO kernel: [99343.731711] NFSD: Unable to initialize client recovery tracking! (-22)
2023-05-30T13:13:52.941346+02:00 DUO kernel: [99343.731713] NFSD: starting 90-second grace period (net f00003a9)
2023-05-30T13:14:33.976843+02:00 DUO kernel: [99384.767497] nfsd: last server has exited, flushing export cache
2023-05-30T13:14:38.784812+02:00 DUO kernel: [99389.577141] eth0: renamed from ip1l5ZZp
2023-05-30T13:14:38.880820+02:00 DUO kernel: [99389.671431] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
2023-05-30T13:14:40.976834+02:00 DUO kernel: [99391.767152] NFSD: attempt to initialize umh client tracking in a container ignored.
2023-05-30T13:14:40.976880+02:00 DUO kernel: [99391.767168] NFSD: attempt to initialize legacy client tracking in a container ignored.
2023-05-30T13:14:40.976883+02:00 DUO kernel: [99391.767169] NFSD: Unable to initialize client recovery tracking! (-22)
2023-05-30T13:14:40.976885+02:00 DUO kernel: [99391.767171] NFSD: starting 90-second grace period (net f00003a9)

NFSD ( NFS server works fine ) , but im sure this is not important, other container without NFS , can’t connect to host too.

in example other VMserver syslog

root@DUO:/srv/lxc # tail -f /var/log/syslog
2023-05-30T13:29:47.846443+02:00 DUO kernel: [100298.630196] eth0: renamed from ip9KiHM2
2023-05-30T13:29:47.924868+02:00 DUO kernel: [100298.707984] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
2023-05-30T13:30:05.384808+02:00 DUO kernel: [100316.166509] audit: type=1400 audit(1685446205.378:36): apparmor="STATUS" operation="profile_remove" profile="/usr/bin/lxc-start" name="lxc-vmBookstack_</var/lib/lxc>" pid=29267 comm="apparmor_parser"
2023-05-30T13:30:09.324807+02:00 DUO kernel: [100320.105118] audit: type=1400 audit(1685446209.318:37): apparmor="STATUS" operation="profile_load" profile="/usr/bin/lxc-start" name="lxc-vmBookstack_</var/lib/lxc>" pid=29284 comm="apparmor_parser"
2023-05-30T13:30:09.408832+02:00 DUO kernel: [100320.189921] eth0: renamed from ipdGVWMw
2023-05-30T13:30:09.480830+02:00 DUO kernel: [100320.263702] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.

no firewall …

root@DUO:/srv/lxc # iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
root@DUO:/srv/lxc #

and kernel has support for IPVLAN enabled. If i create manual device , i shell for test, set IP , i can do all with them.