Unable to reach floating ip from outside haproxy lxc container(keepalived/VRRP setup)

I am testing LXD for Virtual Router Redundancy Protocol using keepalived and haproxy. To test the service, I am using webserver as the target application server.
Below is a list of the applicable IPs and nodes:
host machine: 192.168.1.117
Floating IP=192.168.1.240
haproxy-01:192.168.1.245
haproxy-01:192.168.1.246
webserver-01: 192.168.1.238
webserver-01: 192.168.1.239

Everything worked ok when I setup with vagrant and virtualbox. I am more keen on deploying using LXD. But when I setup with LXD, everything work as expected inside haproxy-01 or haproxy-02.
From the lxd host, everything works ok except connection via the floating IP.
The floating IP is unreachable when pinged from outside haproxy LXC containers and cannot curl to the haproxy nor to webservers.

haproxy-01 configuration:

global
    log 127.0.0.1 local0
    log 127.0.0.1 local1 notice
    daemon
    maxconn 2000

defaults
    log global
    mode http
    option dontlognull
    retries 3
    option redispatch
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

frontend vrrp-request
    bind *:80
    default_backend nodes
    
backend nodes
    mode http
    stats enable
    stats auth admin:admin
    stats uri /haproxy?stats
    balance roundrobin
    option httpchk
    option forwardfor
    option http-server-close
    server webserver-01 192.168.1.238:80 check
    server webserver-02 192.168.1.239:80 check

keepalived config:

vrrp_script chk_haproxy {
    script "killall -0 haproxy"
    interval 2
    weight 2
}
vrrp_instance VI_1 {
    interface eth0
    state MASTER
    virtual_router_id 51
    priority 101
    virtual_ipaddress {
        192.168.1.240
    }
    track_script {
        chk_haproxy
    }
}

Below is a network profile (each node has seperate based on same template):

lxc profile show routed_192.168.1.245
config:
  user.network-config: |
    version: 2
    ethernets:
        eth0:
            addresses:
            - 192.168.1.245/32
            nameservers:
                addresses:
                - 8.8.8.8
                search: []
            routes:
            -   to: 0.0.0.0/0
                via: 192.168.1.1
                on-link: true
description: Default LXD profile
devices:
  eth0:
    ipv4.address: 192.168.1.245
    nictype: routed
    parent: wlp2s0
    type: nic
name: routed_192.168.1.245
used_by:
- /1.0/instances/haproxy-01

LXD version is 5.10.
I hope someone can guide me to resolve this issue.

1 Like

You cannot use a floating IP managed inside the container with routed NIC type.
As the name suggests, it uses routing to achieve network connectivity with the external network, rather than sharing the layer 2 domain (such as with macvlan or bridge NIC types).

Infact one of the important features of the routed NIC is that it prevents containers from announcing (and thus attracting traffic for) IP addresses that have not been explicitly allowed by LXD.

You probably want to be using a bridged NIC connected to an unmanaged (manually setup) bridge that is also connected to your external network.

This will then allow your container to send APR/NDP packets to advertise it is responsible for additional IPs.

Hi,
I have managed like that and it works. Set lxc network set lxdbr0 dns.mode=none and add network interface to each ha instances. lxc config device add <ha_instances> eth1 nic nictype=bridged parent=lxdbr0 name=eth1```
Configuration file changed a little bit.

vrrp_script chk_haproxy {
    script "killall -0 haproxy"
    interval 2
    weight 2
}
vrrp_instance VI_1 {
    interface eth0
    state MASTER
    virtual_router_id 51
    priority 100

    unicast_src_ip <your network hapr ip> #IP of this device 
    unicast_peer {
	<your secondary hapr ip> #IP of peer device
    }

    virtual_ipaddress {
        <floating ip> dev eth1
    }
    track_script {
        chk_haproxy
    }
}

Regards.

Thanks tomp for the very informative response. I must admit network is an area I am learning every day. It took me a while to identify a method that allowed me to assign static ips based on my local network. Routed worked very well. My ealier studies and test for briged did not work. The container would start but the nic would hung leaving the container without ip.
Kindly guide on some instructions that can allowe me to set container with desired static ip and accessible to the floating ip through the haproxy. I am open to any advice that is considered best prcrice

Chemzafer, I really appreciate you taking time to look at the issue. I will follow your guide and give an update how it goes.

This is to report that I eventually got the VRRP setup working as expected.
The game changer was moving away from routed network as suggested by @tomp then assigning ip as in the example below:

lxc launch ubuntu:22.04 a1
lxc network attach lxdbr0 a1 eth0 eth0 
lxc config device set a1 eth0 ipv4.address 192.168.2.237
lxc exec a1 -- reboot

LXD is great! and thanks guyz for input.

1 Like