Routed public ip, lag on 443 connection

Hi, experiencing this odd behavior and i cannot wrap my Head around it.
With disabled ufw https is fast. With enabled ufw it lags by a lot while http remains fast.
Experimenting with the routed profile with just one additional IP on a hetzner bare bone Server.

my lxd host configs:

root@test ~ # uname -a
Linux test 5.4.0-139-generic #156-Ubuntu SMP Fri Jan 20 17:27:18 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
root@test ~ # cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.5 LTS"

root@test ~ # lxd --version
5.10

root@test ~ # cat /etc/network/interfaces
source-directory /etc/network/interfaces.d

auto lo
iface lo inet loopback

auto enp2s0
iface enp2s0 inet static
  address 213.239.210.243
  netmask 255.255.255.224
  gateway 213.239.210.225
  pointtopoint 213.239.210.225
  dns-nameserver 185.12.64.1 185.12.64.2 8.8.8.8 8.8.4.4

root@test ~ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 44:8a:5b:5d:d5:32 brd ff:ff:ff:ff:ff:ff
    inet 213.239.210.243/27 brd 213.239.210.255 scope global enp2s0
       valid_lft forever preferred_lft forever
3: lxdbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 00:16:3e:00:c8:17 brd ff:ff:ff:ff:ff:ff
    inet 10.126.175.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
5: mail@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5a:25:f6:c6:9e:62 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 169.254.0.1/32 scope global mail
       valid_lft forever preferred_lft forever
root@test ~ # ip r
default via 213.239.210.225 dev enp2s0 onlink 
10.126.175.0/24 dev lxdbr0 proto kernel scope link src 10.126.175.1 linkdown 
213.239.210.224/27 dev enp2s0 proto kernel scope link src 213.239.210.243 
213.239.211.94 dev mail scope link 

root@test ~ # lxc network list
+--------+----------+---------+-----------------+------+-------------+---------+---------+
|  NAME  |   TYPE   | MANAGED |      IPV4       | IPV6 | DESCRIPTION | USED BY |  STATE  |
+--------+----------+---------+-----------------+------+-------------+---------+---------+
| enp2s0 | physical | NO      |                 |      |             | 2       |         |
+--------+----------+---------+-----------------+------+-------------+---------+---------+
| lxdbr0 | bridge   | YES     | 10.126.175.1/24 | none |             | 1       | CREATED |
+--------+----------+---------+-----------------+------+-------------+---------+---------+


root@test ~ # cat /etc/sysctl.conf 
net.ipv4.conf.enp2s0.forwarding=1

root@test ~ # lxc profile show routed
config:
  user.network-config: |
    version: 2
    ethernets:
        eth0:
            addresses:
            - 213.239.211.94/32
            nameservers:
                addresses:
                - 185.12.64.1
                - 185.12.64.2
                - 8.8.8.8
                search: []
            routes:
            -   to: 0.0.0.0/0
                via: 169.254.0.1
                on-link: true
description: Routed LXD profile
devices:
  eth0:
    host_name: mail
    ipv4.address: 213.239.211.94
    nictype: routed
    parent: enp2s0
    type: nic
name: routed
used_by:
- /1.0/instances/mail

root@test ~ # lxc list
+------+---------+-----------------------+------+-----------+-----------+
| NAME |  STATE  |         IPV4          | IPV6 |   TYPE    | SNAPSHOTS |
+------+---------+-----------------------+------+-----------+-----------+
| mail | RUNNING | 213.239.211.94 (eth0) |      | CONTAINER | 0         |
+------+---------+-----------------------+------+-----------+-----------+

root@test ~ # lxc info | grep firewall:
  firewall: xtables

root@test ~ # grep -Ev ^'(#|$)' /etc/default/ufw 
IPV6=no
DEFAULT_INPUT_POLICY="DROP"
DEFAULT_OUTPUT_POLICY="ACCEPT"
DEFAULT_FORWARD_POLICY="DROP"
DEFAULT_APPLICATION_POLICY="SKIP"
MANAGE_BUILTINS=no
IPT_SYSCTL=/etc/ufw/sysctl.conf
IPT_MODULES=""


root@test ~ # ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22/tcp                    ALLOW IN    Anywhere                  
[ 2] 213.239.211.94 80 on mail  ALLOW FWD   Anywhere on enp2s0        
[ 3] 213.239.211.94 8080 on mail ALLOW FWD   Anywhere on enp2s0        
[ 4] 213.239.211.94 443 on mail ALLOW FWD   Anywhere on enp2s0      

Inside container,

root@test ~ # lxc shell mail
root@mail:~# ip r
default via 169.254.0.1 dev eth0 
default via 169.254.0.1 dev eth0 proto static onlink 
169.254.0.1 dev eth0 scope link 
root@mail:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
4: eth0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:c8:ce:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 213.239.211.94/32 scope global eth0
       valid_lft forever preferred_lft forever

Now when the firewall is enabled the https connection to port 8080 takes much longer then without ufw while the http connection to port 80 is instant, which is voodoo witchcraft to me.

What am i missing?

Does accessing a service via IP work quickly? I’m wondering if you’ve blocked DNS or IPv6.

Thanks you are right, turns out the routed container could not resolve DNS.
Adding ufw route allow out on enp2s0 allowed the container to resolve names again.


Meanwhile i also learned how i could create an additional non lxd managed virtual interface and use it with a modified routed profile. I believe to have seen this or something like this in this forum a long time ago but could be useful to play around on hetzner servers.

Might be of interest to someone, change IPs and hostname in profile and specify the network with -n when creating,
`lxc launch images:ubuntu/20.04 c1 -p default -p routed_virbr1 -n virbr1

root@test ~ # cat /etc/network/interfaces
source-directory /etc/network/interfaces.d

auto lo
iface lo inet loopback

auto enp2s0
iface enp2s0 inet static
  address 213.239.210.243
  netmask 255.255.255.224
  gateway 213.239.210.225
  pointtopoint 213.239.210.225
  dns-nameserver 185.12.64.1 185.12.64.2 8.8.8.8 8.8.4.4


auto virbr1
iface virbr1 inet static
   address 213.239.210.243
   netmask 255.255.255.224
   bridge_ports none
   bridge_stp off
   bridge_fd 0
   pre-up brctl addbr virbr1
   up ip route add 213.239.211.94/32 dev virbr1
   down ip route del 213.239.211.94/32 dev virbr1

root@test ~ # lxc profile show routed_virbr1 
config:
  user.network-config: |
    version: 2
    ethernets:
        eth0:
            addresses:
            - 213.239.211.94/32
            nameservers:
                addresses:
                - 185.12.64.1
                - 185.12.64.2
                - 8.8.8.8
                search: []
            routes:
            -   to: 0.0.0.0/0
                via: 213.239.210.243
                on-link: true
description: Routed LXD profile
devices:
  eth0:
    host_name: mail
    ipv4.address: 213.239.211.94
    nictype: routed
    parent: virbr1
    type: nic
name: routed_virbr1