Accessing SQL in another servers container

LXD
1

I have an application that runs on several servers in a cluster. I am looking to access container in A from many containers in A, B , C, D etc… Containers have different IP class than others obviously. I can’t even ping the Database Container from the other server. In the past I have been able to do this with a NOSQL via Nginx. Anyway looking for the easiest and best way for container 240.19.0.95 to be able to access Mysql in 240.18.0.90.

Thanks in advance

tony

2

The subnets you’re using suggest you’re using a fan nework, is that correct?

If so, all containers would be expected to have direct IP connectivity to one another, if they don’t (as you’re indicating), then it’d probably be best to figure out what’s preventing the fan from doing its job, once resolved, that will allow direct connectivity between containers.

3

Yes, the seem to be fan. I can not reach a container via a ping from another server. Not sure what that means or why I can or if I should be able to?

4

Can you show lxc network show lxdfan0 (assuming that’s its name).
And then ip -4 a on each of the hosts too.

Do you have firewalling going on between your hosts? I believe the fan uses VXLAN by default which would require UDP 4789 to be allowed in between the servers.

5

I did not allow it in Firewall - I added port 4789 - however ping still does not work.

lxc network show lxdfan0
config:
bridge.mode: fan
fan.underlay_subnet: 84.17.40.0/24
ipv4.nat: “true”
description: “”
name: lxdfan0
type: bridge
used_by:

  • /1.0/instances/AI-GENIE-2020-mar6
  • /1.0/instances/AI-GENIE-2020-mar6-bk-Jul20-2020
  • /1.0/instances/CHAT
- /1.0/instances/X-PROSODY
- /1.0/instances/amusing-doe
- /1.0/instances/lxdMosaic2020B2
- /1.0/instances/move-8eaeeeb7-804d-44ca-8622-2b25c34baea0
- /1.0/instances/u1
- /1.0/profiles/default
managed: true
status: Created
locations:
- Q1
- Q3
- Q2
- Q4

====================

: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 84.17.40.18/26 brd 84.17.40.63 scope global noprefixroute enp1s0
       valid_lft forever preferred_lft forever
4: lxdfan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    inet 240.18.0.1/8 scope global lxdfan0
       valid_lft forever preferred_lft forever
===================
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 84.17.40.19/26 brd 84.17.40.63 scope global noprefixroute enp1s0
       valid_lft forever preferred_lft forever
4: lxdfan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    inet 240.19.0.1/8 scope global lxdfan0
       valid_lft forever preferred_lft forever
================
ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 84.17.40.20/26 brd 84.17.40.63 scope global noprefixroute enp1s0
       valid_lft forever preferred_lft forever
6: lxdfan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    inet 240.20.0.1/8 scope global lxdfan0
       valid_lft forever preferred_lft forever
==========================
ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 84.17.40.21/26 brd 84.17.40.63 scope global noprefixroute enp1s0f0
       valid_lft forever preferred_lft forever
4: wlx9cefd5fb69ac: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 10.0.1.148/24 brd 10.0.1.255 scope global dynamic noprefixroute wlx9cefd5fb69ac
       valid_lft 85675sec preferred_lft 85675sec
5: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100
    inet 10.8.0.1/24 brd 10.8.0.255 scope global tun0
       valid_lft forever preferred_lft forever
6: lxdfan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    inet 240.21.0.1/8 scope global lxdfan0
       valid_lft forever preferred_lft forever
6

Ok, the fan config looks correct, so chances are all your problems are firewall related.
For the fan to work, I’d expect you at the very least need:

  • VXLAN to be allowed through (udp/4789)
  • Firewall on each of the hosts (if present) to allow FORWARD to/from the fan subnet (240.0.0.0/8) over the underlay interface (enp1s0 or enp1s0f0 in your case)

You’ll probably want to run some tcpdump on both lxdfan0 and enp1s0 to see what’s being received and sent.

ip -4 route show on the different servers may also be useful (though I remember most of the fan magic being hidden in the kernel).

7

I have UFW in each server and added ufw allow proto udp from any to any port 4789 to each server. Not sure if this fulfills your rule: ufw allow proto udp from any to any port 4789
I found that if disable UFW it seems to work.
Any idea for what rule to add.

8

You still need to add a rule which allows forwarding to/from the 240.0.0.0/8 subnet.