No IPV4 address for freebsd vm

zcatav · July 17, 2024, 11:15am

Hi,
I use Incus 6.0.1 on Debian Bookworm and Linux containers and VMs (cli or desktop) run without any problems.
However, when installing FreeBSD, it cannot get an IP address from DHCP. FreeBSD uses vtnet0 network structure. The installed containers and VMs cannot get Ipv4 but they have Ipv6 addresses. FreeBSD machines cannot have internet access. Virt-manager is also installed and running on my system.
How can I solve this problem? Thanks for your help.

incus ls

iptables -L -n -v

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 317K 1152M LIBVIRT_INP  0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   26  1976 LIBVIRT_FWX  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
   26  1976 LIBVIRT_FWI  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
   26  1976 LIBVIRT_FWO  0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 183K   16M LIBVIRT_OUT  0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain LIBVIRT_FWI (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     0    --  *      virbr0  0.0.0.0/0            192.168.122.0/24     ctstate RELATED,ESTABLISHED
    0     0 REJECT     0    --  *      virbr0  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain LIBVIRT_FWO (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     0    --  virbr0 *       192.168.122.0/24     0.0.0.0/0           
    0     0 REJECT     0    --  virbr0 *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain LIBVIRT_FWX (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     0    --  virbr0 virbr0  0.0.0.0/0            0.0.0.0/0           

Chain LIBVIRT_INP (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     17   --  virbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:53
    0     0 ACCEPT     6    --  virbr0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53
    0     0 ACCEPT     17   --  virbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:67
    0     0 ACCEPT     6    --  virbr0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:67

Chain LIBVIRT_OUT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     17   --  *      virbr0  0.0.0.0/0            0.0.0.0/0            udp dpt:53
    0     0 ACCEPT     6    --  *      virbr0  0.0.0.0/0            0.0.0.0/0            tcp dpt:53
    0     0 ACCEPT     17   --  *      virbr0  0.0.0.0/0            0.0.0.0/0            udp dpt:68
    0     0 ACCEPT     6    --  *      virbr0  0.0.0.0/0            0.0.0.0/0            tcp dpt:68

ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 80:fa:5b:42:3e:a4 brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 30:e3:7a:ae:4d:ef brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.165/24 brd 192.168.1.255 scope global dynamic noprefixroute wlan0
       valid_lft 84843sec preferred_lft 84843sec
    inet6 fe80::34f7:b8f7:8c1c:ad51/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:1e:9c:26 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever

ip link show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
    link/ether 80:fa:5b:42:3e:a4 brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DORMANT group default qlen 1000
    link/ether 30:e3:7a:ae:4d:ef brd ff:ff:ff:ff:ff:ff
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether 52:54:00:1e:9c:26 brd ff:ff:ff:ff:ff:ff

zcatav · July 21, 2024, 8:36pm

incus config device add <instance_name> eth0 nic nictype=bridged parent=virbr0

This command solved problem.

zcatav · October 10, 2024, 5:36pm

Hi again,
I use Incus 6.6 on Linux Mint Virginia, Linux containers and VMs (cli or desktop) run without any problems.
However, when installing FreeBSD, it cannot get an IP address from DHCP. FreeBSD uses vtnet0 network structure. The installed containers and VMs cannot get Ipv4 but they have Ipv6 addresses. FreeBSD machines cannot have internet access.

iptables -L -n -v | grep incus

                                                                                                                                                                                 
    0     0 ACCEPT     all  --  incusbr0 *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      incusbr0  0.0.0.0/0            0.0.0.0/0           
  278 79417 ACCEPT     all  --  incusbr0 *       0.0.0.0/0            0.0.0.0/0

ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp3s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 80:fa:5b:42:3e:a4 brd ff:ff:ff:ff:ff:ff
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 30:e3:7a:ae:4d:ef brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.104/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp2s0
       valid_lft 80288sec preferred_lft 80288sec
    inet6 fe80::925c:7894:5c13:9ac1/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
4: incusbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:95:29:de brd ff:ff:ff:ff:ff:ff
    inet 10.126.157.1/24 scope global incusbr0
       valid_lft forever preferred_lft forever
    inet6 fd42:df1b:7995:457c::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe95:29de/64 scope link 
       valid_lft forever preferred_lft forever
11: tap7109052a: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master incusbr0 state UP group default qlen 1000
    link/ether 02:6e:83:1e:62:59 brd ff:ff:ff:ff:ff:ff

ip link show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp3s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
    link/ether 80:fa:5b:42:3e:a4 brd ff:ff:ff:ff:ff:ff
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DORMANT group default qlen 1000
    link/ether 30:e3:7a:ae:4d:ef brd ff:ff:ff:ff:ff:ff
4: incusbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 00:16:3e:95:29:de brd ff:ff:ff:ff:ff:ff
11: tap7109052a: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master incusbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 02:6e:83:1e:62:59 brd ff:ff:ff:ff:ff:ff

In this case

incus config device add freebsd eth0 nic nictype=bridged parent=incusbr0

doesn’t solve my problem.

Any suggestion?
Thanks in advance.

candlerb · October 11, 2024, 8:51am

If it’s not firewalling then maybe it’s to do with UDP checksum. It could be that the FreeBSD VM is sending DHCP packets with no checksum set, on the assumption they will be added by the physical NIC on egress.

On linux guests you can do:

ethtool --offload eth0 tx off

but I don’t know what the equivalent is for FreeBSD.

According to this old post, lxd had a workaround to update the checksums using mangle rules.

However, on a pure incus 6.0.2 system here, iptables -L -n -v -t mangle doesn’t show anything. On a system with ufw and libvirt as well, I see libvirt adds one for itself:

Chain LIBVIRT_PRT (1 references)
 pkts bytes target     prot opt in     out     source               destination
  220 75251 CHECKSUM   udp  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            udp dpt:68 CHECKSUM fill

but not one for incus. Therefore, maybe incus’ dnsmasq has its own workaround now. But still, it would be worth doing a tcpdump on the incusbr0 interface, and an strace on the dnsmasq process, to see if the DHCP discover packets are being received.

zcatav · October 11, 2024, 5:29pm

Hi,

FreeBSD VM booting sequence;

Starting dhclient.

DHCPDISCOVER on vtneto to 255.255.255.255 port 67 interval 4

DHCPDISCOVER on vtneto to 255.255.255.255 port 67 interval 9

DHCPDISCOVER on vtnete to 255.255.255.255 port 67 interval 20 DHCPDISCOVER on vtneto to 255.255.255.255 port 67 interval 16

DHCPDISCOVER on vtnete to 255.255.255.255 port 67 interval 12

**5 bad udp checksums in 5 packets**

No DHCPOFFERS received.

No working leases in persistent database sleeping.

Waiting 30s for the default route interface:

add host 127.0.0.1: gateway 100 fib 0: route already in table

add host::1: gateway 100 fib 0: route already in table

add net fe80::: gateway:: 1

add net ff02::: gateway:: 1

add net::ffff: 0.0.0.0: gateway:: 1

add net: 0.0.0.0: gateway:: 1

Starting local_unbound.

It’s look like a checksum problem.

Tcpdump as follow;

tcpdump -i incusbr0 port 67 -vvv
tcpdump: listening on incusbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:33:58.105980 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0x4c1f21fd, secs 6, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:33:58.106514 IP (tos 0xc0, ttl 64, id 31759, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x5213!] BOOTP/DHCP, Reply, length 300, xid 0x4c1f21fd, secs 6, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:34:04.267288 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0x4c1f21fd, secs 12, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:34:04.267852 IP (tos 0xc0, ttl 64, id 34125, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x520d!] BOOTP/DHCP, Reply, length 300, xid 0x4c1f21fd, secs 12, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:34:16.281625 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0x4c1f21fd, secs 24, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:34:16.282210 IP (tos 0xc0, ttl 64, id 36379, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x5201!] BOOTP/DHCP, Reply, length 300, xid 0x4c1f21fd, secs 24, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:34:29.415223 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0x4c1f21fd, secs 37, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:34:29.415831 IP (tos 0xc0, ttl 64, id 44947, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x51f4!] BOOTP/DHCP, Reply, length 300, xid 0x4c1f21fd, secs 37, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:34:42.477172 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0x4c1f21fd, secs 50, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:34:42.477590 IP (tos 0xc0, ttl 64, id 55688, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x51e7!] BOOTP/DHCP, Reply, length 300, xid 0x4c1f21fd, secs 50, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:36:01.267924 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0xa26ac46d, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:36:01.268040 IP (tos 0xc0, ttl 64, id 37903, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x595d!] BOOTP/DHCP, Reply, length 300, xid 0xa26ac46d, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:36:05.295033 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0xa26ac46d, secs 4, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:36:05.295546 IP (tos 0xc0, ttl 64, id 40453, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x5959!] BOOTP/DHCP, Reply, length 300, xid 0xa26ac46d, secs 4, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:36:14.451926 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0xa26ac46d, secs 13, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:36:14.452485 IP (tos 0xc0, ttl 64, id 47241, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x5950!] BOOTP/DHCP, Reply, length 300, xid 0xa26ac46d, secs 13, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:36:22.469840 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0xa26ac46d, secs 21, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:36:22.470408 IP (tos 0xc0, ttl 64, id 51277, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x5948!] BOOTP/DHCP, Reply, length 300, xid 0xa26ac46d, secs 21, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:36:32.485485 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0xa26ac46d, secs 31, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:36:32.486082 IP (tos 0xc0, ttl 64, id 55872, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x593e!] BOOTP/DHCP, Reply, length 300, xid 0xa26ac46d, secs 31, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0
15:36:42.507927 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:16:3e:9b:fc:f4 (oui Unknown), length 300, xid 0xa26ac46d, secs 41, Flags [none] (0x0000)
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Discover
	    Client-ID (61), length 7: ether 00:16:3e:9b:fc:f4
	    Hostname (12), length 7: "freebsd"
	    Parameter-Request (55), length 10: 
	      Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121)
	      Default-Gateway (3), Domain-Name (15), Domain-Name-Server (6), Hostname (12)
	      Unknown (119), MTU (26)
	    END (255), length 0
	    PAD (0), length 0, occurs 26
15:36:42.508540 IP (tos 0xc0, ttl 64, id 58096, offset 0, flags [none], proto UDP (17), length 328)
    medimint.catav.cyou.bootps > 10.126.157.22.bootpc: [bad udp cksum 0x5059 -> 0x5934!] BOOTP/DHCP, Reply, length 300, xid 0xa26ac46d, secs 41, Flags [none] (0x0000)
	  Your-IP 10.126.157.22
	  Server-IP medimint.catav.cyou
	  Client-Ethernet-Address 00:16:3e:9b:fc:f4 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message (53), length 1: Offer
	    Server-ID (54), length 4: medimint.catav.cyou
	    Lease-Time (51), length 4: 3600
	    RN (58), length 4: 1800
	    RB (59), length 4: 3150
	    Subnet-Mask (1), length 4: 255.255.255.0
	    BR (28), length 4: 10.126.157.255
	    Default-Gateway (3), length 4: medimint.catav.cyou
	    Domain-Name-Server (6), length 4: medimint.catav.cyou
	    Domain-Name (15), length 5: "incus"
	    END (255), length 0
	    PAD (0), length 0

candlerb · October 11, 2024, 6:24pm

Interesting: so it’s the replies going back to FreeBSD being rejected because of bad checksums.

Then I guess it’s worth trying:

# does this show tx-checksumming: on ?
ethtool --show-offload incusbr0 | grep -i checksum

# if so, try this:
ethtool --offload incusbr0 tx off

zcatav · October 12, 2024, 7:09am

Hi,

ethtool --show-offload incusbr0 | grep -i checksum
rx-checksumming: off [fixed]
tx-checksumming: on
	tx-checksum-ipv4: off [fixed]
	tx-checksum-ip-generic: on
	tx-checksum-ipv6: off [fixed]
	tx-checksum-fcoe-crc: off [fixed]
	tx-checksum-sctp: off [fixed]

ethtool --offload incusbr0 tx off
Actual changes:
tx-checksum-ip-generic: off
tx-tcp-segmentation: off [not requested]
tx-tcp-ecn-segmentation: off [not requested]
tx-tcp-mangleid-segmentation: off [not requested]
tx-tcp6-segmentation: off [not requested]

freebsd | RUNNING | 10.126.157.22 (eth0) | fd42:df1b:7995:457c:216:3eff:fe9b:fcf4 (eth0) | VIRTUAL-MACHINE | 0 |

Thanks for your help.

PS: Does this pose a security vulnerability?

candlerb · October 12, 2024, 8:03am

I can’t see how. It just makes the kernel do a bit more work to calculate checksums properly instead of deferring them until later.

zcatav · October 12, 2024, 8:31am

How can I make this command permanent?

candlerb · October 12, 2024, 9:54am

Incus itself creates the incusbr0 network, and I don’t see a config hook in the docs.

You could try creating /etc/networkd-dispatcher/routable.d/incus containing

#!/bin/bash -eu
if [ "$IFACE" = "incusbr0" ]; then
  ethtool --offload incusbr0 tx off
fi

(and make it executable: chmod +x). But I don’t know if systemd-networkd will invoke it for an interface it didn’t create or initialize itself. Try it anyway. You could also try carrier.d or configured.d directories (man networkctl for descriptions)

zcatav · October 12, 2024, 11:52am

This solution did not work. I added the command to the KDE startup applications and that worked. Thanks again.