Advice on how to access service running in LXD container on different host on LAN

LXD
#1

I have 2 hosts on my LAN as illustrated below. On the host A i can access svc_2.lxd. I would like to be able to do the same from host B and i am wondering how to go about. I would like to have name resolution for .lxd managed by the dnsmasq attached to lxdbr0 on host A and I am wondering if it is possible. Any kind of advice would be most welcomed.

+---------------------+             +------------------+
| host A              |             | Ubuntu host B    |
| 192.168.0.10        |<--- LAN --->| 192.168.0.20     |
| +-----------------+ |             |                  |
| | LXD - svc_1.lxd | |             | $ curl svc_2.lxd |
| | 10.0.0.10       | |             +------------------+
| +-----------------+ |
| +-----------------+ |
| | LXD - svc_2.lxd | |
| | 10.0.0.11       | |
| +-----------------+ |
+---------------------+

Thanks

#2

You need to create a bridge interface on host to access the container over LAN.

Below worked for me,
I am running LXD in ubuntu 21.10 host,
I created a bridge interface in /etc/netplan config file.

network:
ethernets:
eth0:
dhcp4: no

bridges:
br01:
interfaces: [eth0]
addresses: [192.168.0.11/24]
gateway4: 192.168.0.1
mtu: 1500
nameservers:
addresses: [192.168.0.3, 192.168.0.1]
parameters:
stp: true
forward-delay: 4
dhcp4: no
dhcp6: no

Then attach the interface to container
lxc config device add <CONTAINER_NAME> eth0 nic name=eth0 nictype=bridged parent=br01

After this your container should pick up IP via DHCP or static if configured.

For local DNS resolution I use pi hole in a separate container.

#3

Thanks. From a routing point of view I could create a route from host A to host B and since IPv4 Forwarding is enabled on the later. Thus, I would be able to access the service with IP addresses. However, I would like to leverage the DNS resolver on lxdbr0 on host A instead of adding another DNS resolver somewhere else. Would you have an idea on how I could achieve that?

#4

No super easy solutions :slight_smile:

You could run another dnsmasq, bind9 or unbound DNS server on the system which then forwards the .lxd domain to the local dnsmasq that LXD operates while sending the rest of the traffic to resolve on a normal upstream DNS server.

#5

Thanks @stgraber :slight_smile:

I am unable to run another dnsmasq process on host A as the port 53 is already in use. Example:

$ dnsmasq --listen-address=192.168.0.10 

dnsmasq: failed to create listening socket for port 53: Address already in use

And so, I tried to leverage resolved to achieve my goal but was not successful. Example of my approach:

Since,

$ # Host A
$ cat /etc/systemd/resolved.conf.d/lxd.conf 
[Resolve]
DNS=10.0.0.1
Domains=lxd

$ # Host A
$ curl svc_2.lxd
hello

Then on host B I configured resolved as well given host A is reachable from B as they are on the same LAN. So the configuration in this example:

$ # Host B
$ cat /etc/systemd/resolved.conf.d/lxd.conf 
[Resolve]
DNS=192.168.0.10 # IP Address of host A
Domains=lxd

From host B if i try to resolve svc_2.lxd, I can see that it is going towards the configured IP address. Example:

12:57:36.657138 IP (tos 0x0, ttl 64, id 21227, offset 0, flags [DF], proto UDP (17), length 56)
    gl-pc.41778 > 192.168.0.10.domain: 44594+ A? svc_2.lxd. (28)

And on host A i can see incoming request from host B. But unfortunately I do not have an resolver bound to 192.168.0.10 and the resolve request dies. I was hoping whether the dnsmasq used by LXD could be adjusted accordingly. For instance, I was thinking whether I could append the IP address of host A to --listen-address flag of dnsmasq and thus there would be a correct DNS server to respond to the query? If this could be a way to try, would you have an idea how i could add this configuration? For instance now i can see that the process is launched to listen only on the gateway of the bridge:

2057 dnsmasq --keep-in-foreground --strict-order --bind-interfaces --except-interface=lo --pid-file= --no-ping --interface=lxdbr0 --dhcp-rapid-commit --quiet-dhcp --quiet-dhcp6 --quiet-ra --listen-address=10.0.0.1 --dhcp-no-override --dhcp-authoritative --dhcp-leasefile=/var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.leases --dhcp-hostsfile=/var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.hosts --dhcp-range 10.0.0.2,10.0.0.254,1h -s lxd --interface-name _gateway.lxd,lxdbr0 -S /lxd/ --conf-file=/var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.raw -u lxd -g lxd