Is it possible DHCP gateways are setting a fixed /24?

Incus
1

I have a network set up with an address with a /22, however I only appear to be able to route traffic via the gateway within the /24.

# incus network show private
config:
  bridge.mtu: "1300"
  dns.nameservers: 10.4.0.14
  ipv4.address: 10.4.0.1/22
  ipv4.dhcp.routes: 0.0.0.0/0,10.4.0.1
  ipv4.nat: "true"
  ipv6.address: none
  network: UPLINK
  volatile.network.ipv4.address: 192.168.1.16

Addresses are allocated by DHCP from 10.4.0.2 upwards and generally work fine. So I’d like to allocate a static of 10.5.0.10 to an instance on this network. Should be Ok as it’s within the routed range (?) So my first test instance using DHCP looks like this;

# ip a|grep global|cut -c1-22
    inet 10.4.0.17/22 
# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.4.0.1        0.0.0.0         UG    1024   0        0 eth0
10.4.0.0        0.0.0.0         255.255.252.0   U     1024   0        0 eth0
10.4.0.1        0.0.0.0         255.255.255.255 UH    1024   0        0 eth0
10.4.0.14       0.0.0.0         255.255.255.255 UH    1024   0        0 eth0

My second instance I set up with a static IP address looks like this;

# ip a|grep global|cut -c1-22
    inet 10.5.0.10/22 
# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.4.0.1        0.0.0.0         UG    0      0        0 eth0
10.5.0.0        0.0.0.0         255.255.252.0   U     0      0        0 eth0

My initial expectation is that they can ping each other - they cannot, when routing to the 10.5 via the gateway it pushes traffic out of the default route.

# traceroute -n 10.5.0.10
traceroute to 10.5.0.10 (10.5.0.10), 30 hops max, 60 byte packets
 1  10.4.0.1  1.982 ms  2.030 ms  2.097 ms
 2  192.168.1.254  3.487 ms  3.745 ms  3.460 ms^C

Kinda looks like the netmask on the gateway set up at 10.4.0.1 is a /24 rather than a /22. Just to confirm a gateway issue, if I add a static route to both sides;

node A:
ip route add 10.5.0.0/24 dev eth0
node B:
ip route add 10.4.0.0/24 dev eth0
..
# traceroute -n 10.5.0.10
traceroute to 10.5.0.10 (10.5.0.10), 30 hops max, 60 byte packets
 1  10.5.0.10  1.952 ms  1.899 ms  1.884 ms

The overall problem being, node B on 10.5 can’t see or get to the gateway on 10.4.0.1, hence it can’t route to other networks over the OVN-IC (as it has no access to a gateway).
Any idea what I’m doing wrong?

2

So I’d like to allocate a static of 10.5.0.10 to an instance on this network. Should be Ok as it’s within the routed range (?)

It’s not. 10.4.0.0/22 covers the range 10.4.0.0 to 10.4.3.255 inclusive.

I think you maybe want 10.4.0.0/14 which would cover 10.4.0.0 … 10.7.255.255

3

Gah! yes … many thanks, I knew it would be staring me in the face. I have no idea why I thought 10.5 would fit in a /22 … will try again with 10.4.3 … this is what happens when you get old, stupid stuff keeps coming to bite your rear end.

4

Yeah, that seems to have resolved all the issues …

# cat /etc/systemd/network/10-cloud-init-eth0.network
[Match]
Name=eth0

[Network]
DHCP=ipv4

[Link]
MTUBytes=1280
ConfigureWithoutCarrier=yes

[Network]
Address=10.4.3.1/22
Gateway=10.4.0.1

[DHCPv4]
UseRoutes=no

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.4.0.1        0.0.0.0         UG    0      0        0 eth0
10.4.0.0        0.0.0.0         255.255.252.0   U     0      0        0 eth0
10.4.0.14       0.0.0.0         255.255.255.255 UH    1024   0        0 eth0

It means needing network specific network configuration per instance, although I guess I could use cloud-init, however it does seem to fix all my problems …

# speedtest
Testing download speed................................................................................
Download: 776.81 Mbit/s
Testing upload speed......................................................................................................
Upload: 106.46 Mbit/s

… which is UK <=> Germany

1 Like
5

Possibly a more elegant solution for cloud instances, in my case Debian bookworm. Add the following to the default profile (assuming the default profile hands out eth-1 attached to your OVN network);

config:
  cloud-init.network-config: |
    # cloud-init
    network:
      version: 2
      ethernets:
        eth0:
          dhcp4: true
          dhcp4-overrides:
            use-mtu: false
            use-routes: true
          mtu: 1280
          routes:
          - to: 10.103.0.0/24
            via: 10.4.0.1
            on-link: true

In this case 10.4.0.1 is my gateway and 10.103.0.0/24 is the other end of the OVN-IC link. So we’re still getting default routes and an IP address via DHCP, but the MTU is fixed at a lower setting and we’re installing a static to our remote IC network.

# incus create debian demo3
Creating demo3
# incus config device override demo3 eth-1 ipv4.address=10.4.3.5
Device eth-1 overridden for demo3
# incus start demo3
# incus shell demo3
# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1280
        inet 10.4.3.5  netmask 255.255.252.0  broadcast 10.4.3.255
        inet6 fe80::1266:6aff:febe:855d  prefixlen 64  scopeid 0x20<link>
        ether 10:66:6a:be:85:5d  txqueuelen 1000  (Ethernet)
        RX packets 56  bytes 3248 (3.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 28  bytes 2672 (2.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.4.0.1        0.0.0.0         UG    1024   0        0 eth0
10.4.0.0        0.0.0.0         255.255.252.0   U     1024   0        0 eth0
10.4.0.1        0.0.0.0         255.255.255.255 UH    1024   0        0 eth0
10.4.0.14       0.0.0.0         255.255.255.255 UH    1024   0        0 eth0
10.103.0.0      10.4.0.1        255.255.255.0   UG    0      0        0 eth0

I’m still unsure whether it’s a good idea to let an instance acquire an address when it first boots, then pin that address, or to statically allocate an address and immediately pin it before first boot.