Container can't get IPv4 address on Ubuntu 24.04 LTS (on oracle cloud)

Incus
1

I am having a similar issue to others’ where the default Incus network of incusbr0 created with the incus admin init isn’t giving containers IPv4 addresses.

The incusbr0 has an IPv4 address, but it’s not getting it out to its containers, specifically one named whoogle.

It’s definitely some routing issue, but I can’t make sense of it. docker, ufw, and firewalld are not installed. As far as I can tell there is no firewall controller per-say.

Now I am trying this on the free oracle cloud Ubuntu 24.04 LTS which clearly brings with it weird rules, one of which is likely ruining it. But I can’t find what would be skewering the incusbr0 bridge. I have installed wireguard on port 1024, but I don’t think it’s gonna mess with it since I’ve done that on other servers without issue. Incus is installed via apt without zabbly.

If anyone can help, that’d be great! Semi-appologies for the dump of outputs, it’s commands I saw others ask for when solving this problem.

/proc/sys/net/bridge does not exist. Dump of

incus network show incusbr0:

config:
  ipv4.address: 10.120.169.1/24
  ipv4.firewall: "true"
  ipv4.nat: "true"
  ipv6.address: fd42:70db:9779:2f39::1/64
  ipv6.firewall: "true"
  ipv6.nat: "true"
description: ""
name: incusbr0
type: bridge
used_by:
- /1.0/instances/whoogle
- /1.0/profiles/default
managed: true
status: Created
locations:
- none

incus list:

+---------+---------+------+-----------------------------------------------+-----------+-----------+
|  NAME   |  STATE  | IPV4 |                     IPV6                      |   TYPE    | SNAPSHOTS |
+---------+---------+------+-----------------------------------------------+-----------+-----------+
| whoogle | RUNNING |      | fd42:70db:9779:2f39:216:3eff:fe90:ecda (eth0) | CONTAINER | 0         |
+---------+---------+------+-----------------------------------------------+-----------+-----------+

incus config show whoogle -e:

architecture: x86_64
config:
  image.architecture: amd64
  image.description: Debian bookworm amd64 (20240922_05:24)
  image.os: Debian
  image.release: bookworm
  image.serial: "20240922_05:24"
  image.type: squashfs
  image.variant: cloud
  volatile.base_image: bd0b794ef3c2d9d61c9b2ee8ab0f4b40c95b9beecd917d5239b4fd68ffaf361f
  volatile.cloud-init.instance-id: 395a5c74-1813-45d9-baad-ccb51b856594
  volatile.eth0.host_name: veth468b55e1
  volatile.eth0.hwaddr: 00:16:3e:90:ec:da
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.uuid: df9ebdc4-2e38-49c1-80b6-017d459922f8
  volatile.uuid.generation: df9ebdc4-2e38-49c1-80b6-017d459922f8
devices:
  eth0:
    name: eth0
    network: incusbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

ip link show:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 02:00:17:05:2f:1c brd ff:ff:ff:ff:ff:ff
    altname enp0s3
3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 8920 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/none
4: incusbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 00:16:3e:75:d8:0d brd ff:ff:ff:ff:ff:ff
6: veth468b55e1@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master incusbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 86:c2:90:9b:53:58 brd ff:ff:ff:ff:ff:ff link-netnsid 0

iptables -L -n -v:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     17   --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:1024
 1611  489K ACCEPT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            0.0.0.0/0
   74  7112 ACCEPT     0    --  lo     *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     17   --  *      *       0.0.0.0/0            0.0.0.0/0            udp spt:123
   26  1464 ACCEPT     6    --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW tcp dpt:22
   22  7005 REJECT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     0    --  wg0    *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  ens3   wg0     0.0.0.0/0            0.0.0.0/0
    0     0 REJECT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT 1166 packets, 1033K bytes)
 pkts bytes target     prot opt in     out     source               destination
  464 40061 InstanceServices  0    --  *      *       0.0.0.0/0            169.254.0.0/16

Chain InstanceServices (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.0.2          owner UID match 0 tcp dpt:3260 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.2.0/24       owner UID match 0 tcp dpt:3260 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.4.0/24       owner UID match 0 tcp dpt:3260 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.5.0/24       owner UID match 0 tcp dpt:3260 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.0.2          tcp dpt:80 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
   82  8274 ACCEPT     17   --  *      *       0.0.0.0/0            169.254.169.254      udp dpt:53 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.169.254      tcp dpt:53 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.0.3          owner UID match 0 tcp dpt:80 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.0.4          tcp dpt:80 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
  376 31331 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.169.254      tcp dpt:80 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 ACCEPT     17   --  *      *       0.0.0.0/0            169.254.169.254      udp dpt:67 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 ACCEPT     17   --  *      *       0.0.0.0/0            169.254.169.254      udp dpt:69 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    6   456 ACCEPT     17   --  *      *       0.0.0.0/0            169.254.169.254      udp dpt:123 /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */
    0     0 REJECT     6    --  *      *       0.0.0.0/0            169.254.0.0/16       tcp /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */ reject-with tcp-reset
    0     0 REJECT     17   --  *      *       0.0.0.0/0            169.254.0.0/16       udp /* See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule */ reject-with icmp-port-unreachable

nft list ruleset:

# Warning: table ip filter is managed by iptables-nft, do not touch!
table ip filter {
        chain INPUT {
                type filter hook input priority filter; policy accept;
                udp dport 1024 counter packets 0 bytes 0 accept
                ct state related,established counter packets 1694 bytes 502202 accept
                ip protocol icmp counter packets 0 bytes 0 accept
                iifname "lo" counter packets 76 bytes 7322 accept
                udp sport 123 counter packets 0 bytes 0 accept
                ip protocol tcp ct state new tcp dport 22 counter packets 29 bytes 1644 accept
                counter packets 22 bytes 7005 reject with icmp type host-prohibited
        }

        chain FORWARD {
                type filter hook forward priority filter; policy accept;
                iifname "wg0" counter packets 0 bytes 0 accept
                iifname "ens3" oifname "wg0" counter packets 0 bytes 0 accept
                counter packets 0 bytes 0 reject with icmp type host-prohibited
        }

        chain OUTPUT {
                type filter hook output priority filter; policy accept;
                ip daddr 169.254.0.0/16 counter packets 469 bytes 40484 jump InstanceServices
        }

        chain InstanceServices {
                ip daddr 169.254.0.2 ip protocol tcp skuid 0 tcp dport 3260  counter packets 0 bytes 0 accept
                ip daddr 169.254.2.0/24 ip protocol tcp skuid 0 tcp dport 3260  counter packets 0 bytes 0 accept
                ip daddr 169.254.4.0/24 ip protocol tcp skuid 0 tcp dport 3260  counter packets 0 bytes 0 accept
                ip daddr 169.254.5.0/24 ip protocol tcp skuid 0 tcp dport 3260  counter packets 0 bytes 0 accept
                ip daddr 169.254.0.2 tcp dport 80  counter packets 0 bytes 0 accept
                ip daddr 169.254.169.254 udp dport 53  counter packets 85 bytes 8593 accept
                ip daddr 169.254.169.254 tcp dport 53  counter packets 0 bytes 0 accept
                ip daddr 169.254.0.3 ip protocol tcp skuid 0 tcp dport 80  counter packets 0 bytes 0 accept
                ip daddr 169.254.0.4 tcp dport 80  counter packets 0 bytes 0 accept
                ip daddr 169.254.169.254 tcp dport 80  counter packets 378 bytes 31435 accept
                ip daddr 169.254.169.254 udp dport 67  counter packets 0 bytes 0 accept
                ip daddr 169.254.169.254 udp dport 69  counter packets 0 bytes 0 accept
                ip daddr 169.254.169.254 udp dport 123  counter packets 6 bytes 456 accept
                ip daddr 169.254.0.0/16 ip protocol tcp   counter packets 0 bytes 0 reject with tcp reset
                ip daddr 169.254.0.0/16 ip protocol udp   counter packets 0 bytes 0 reject
        }
}
# Warning: table ip nat is managed by iptables-nft, do not touch!
table ip nat {
        chain PREROUTING {
                type nat hook prerouting priority dstnat; policy accept;
        }

        chain INPUT {
                type nat hook input priority srcnat; policy accept;
        }

        chain OUTPUT {
                type nat hook output priority dstnat; policy accept;
        }

        chain POSTROUTING {
                type nat hook postrouting priority srcnat; policy accept;
                oifname "ens3" counter packets 162 bytes 13293 masquerade
                oifname "ens3" counter packets 0 bytes 0 masquerade
        }
}
table ip6 filter {
        chain INPUT {
                type filter hook input priority filter; policy accept;
        }

        chain FORWARD {
                type filter hook forward priority filter; policy accept;
                iifname "wg0" counter packets 0 bytes 0 accept
        }

        chain OUTPUT {
                type filter hook output priority filter; policy accept;
        }
}
# Warning: table ip6 nat is managed by iptables-nft, do not touch!
table ip6 nat {
        chain PREROUTING {
                type nat hook prerouting priority dstnat; policy accept;
        }

        chain INPUT {
                type nat hook input priority srcnat; policy accept;
        }

        chain OUTPUT {
                type nat hook output priority dstnat; policy accept;
        }

        chain POSTROUTING {
                type nat hook postrouting priority srcnat; policy accept;
                oifname "ens3" counter packets 0 bytes 0 # Warning: XT target MASQUERADE not found
xt target "MASQUERADE"
                oifname "ens3" counter packets 0 bytes 0 # Warning: XT target MASQUERADE not found
xt target "MASQUERADE"
        }
}
table inet incus {
        chain pstrt.incusbr0 {
                type nat hook postrouting priority srcnat; policy accept;
                ip saddr 10.120.169.0/24 ip daddr != 10.120.169.0/24 masquerade
                ip6 saddr fd42:70db:9779:2f39::/64 ip6 daddr != fd42:70db:9779:2f39::/64 masquerade
        }

        chain fwd.incusbr0 {
                type filter hook forward priority filter; policy accept;
                ip version 4 oifname "incusbr0" accept
                ip version 4 iifname "incusbr0" accept
                ip6 version 6 oifname "incusbr0" accept
                ip6 version 6 iifname "incusbr0" accept
        }

        chain in.incusbr0 {
                type filter hook input priority filter; policy accept;
                iifname "incusbr0" tcp dport 53 accept
                iifname "incusbr0" udp dport 53 accept
                iifname "incusbr0" icmp type { destination-unreachable, time-exceeded, parameter-problem } accept
                iifname "incusbr0" udp dport 67 accept
                iifname "incusbr0" icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, nd-router-solicit, nd-neighbor-solicit, nd-neighbor-advert, mld2-listener-report } accept
                iifname "incusbr0" udp dport 547 accept
        }

        chain out.incusbr0 {
                type filter hook output priority filter; policy accept;
                oifname "incusbr0" tcp sport 53 accept
                oifname "incusbr0" udp sport 53 accept
                oifname "incusbr0" icmp type { destination-unreachable, time-exceeded, parameter-problem } accept
                oifname "incusbr0" udp sport 67 accept
                oifname "incusbr0" icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, echo-request, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert, mld2-listener-report } accept
                oifname "incusbr0" udp sport 547 accept
        }
}
2

You’ve got iptables rules installed, probably picked up by the iptables-persistent package which reads /etc/iptables/rules.v4

As you don’t appear to have a rule to allow DHCP, this is most likely why your containers don’t get an IP address.

Try adding “accept” rules on the FORWARD and OUTPUT chains for packets received from interface incusbr0.

Background:

3

Thanks for the words - didn’t quite know what to center my search on but that helped a lot!

I was able to get it working by mimicking the rules that Incus (I assume) made for incusbr0 per nft list ruleset.

This got DHCP to work and gave the container an ipv4 address, but no internet access still.
iptables -I INPUT -i incusbr0 -p udp --dport 67:68 --sport 67:68 -j ACCEPT

These were from mimicking the nftables more:

iptables -I INPUT -i incusbr0 -p udp --dport 53 -j ACCEPT
iptables -I INPUT -i incusbr0 -p tcp --dport 53 -j ACCEPT
iptables -I INPUT -i incusbr0 -p udp --dport 547 -j ACCEPT

These sealed the deal, ipv4 did require the callout to the original network interface ens3 to finally work.

iptables -I FORWARD -i ens3 -o incusbr0 -j ACCEPT
iptables -I FORWARD -i incusbr0 -j ACCEPT
ip6tables -I FORWARD -i incusbr0  -j ACCEPT

I’m not sure if any are superfluous, but the container can access apt now. I wasn’t able to make output rules in iptables that tied to incusbr0, so skipped that but it didn’t matter. Guessing output is good to go due to its unrestricted rule. Also not sure why nftables is ignored for iptables, everything I read says they should “work together to bridge old and new” or something.

Thanks again!!