How to access container alex as alex.lxd from host?

The initial wait, during the first DNS request of every container’s name, makes me want to replace systemd-resolved with dnsmasq on the host. I wonder if it’s a good idea.

You have a typo in your tutorial: You say:

ExecStop=/usr/local/bin/lxdhostdnsi_stop.sh

(there’s an extra i)

Thanks for reporting the typo. I just fixed it.

Indeed, there is an initial wait for the timeout of the default DNS server.
I updated the blog post with a troubleshooting section, which describes the issue.
There should be a way for systemd-resolved to just ask the lxdbr0 DNS server and not need to also consult the default DNS server. systemd-resolved can deduce that because we specified that the .lxd domains are served by the lxdbr0 DNS server.

There should either be some special configuration in systemd for this, or there is a need for a bug report on systemd.

@simos, I tried your suggestion to do: systemctl | grep lxd, in order to find a suitable unit to place in the After= field.

I tried placing all of them in After=, did systemctl daemon-reload after each one, but your service always failed during boot time. The error message is this:

Ιουν 25 09:45:50 karjala-vmware lxdhostdns_start.sh[1403]: Device "lxdbr0" does not exist. Ιουν 25 09:45:50 karjala-vmware lxdhostdns_start.sh[1403]: Unknown interface lxdbr0: No such device

So we still haven’t solved the problem for Snap users (which is the most important), only for Apt users.

I wonder if it’s a good solution to replace systemd-resolved with dnsmasq on the host, like I had in Ubuntu 16.04. What do you think about this?

However, when I systemctl start your service after boot, and after I have logged in, that works, and the LXD containers hostnames resolve.

First of all, thank for trying out with the different options for After.

Ideally, it should be better to figure out a way to make it work reliably on Ubuntu 18.04 without additional system changes.
I’ll look into this a bit later on a fresh 18.04 install. If you need to get a system working at once, then go ahead and switch to dnsmasq.

P.S. I didn’t use StartAfter= option, but the After= option. Did I do it wrong?

No, you did right. There is no StartAfter, it is After. I just fixed my typo above.

Hey @simos, I keep having problems with this when using the fan networking and snap. I reported this initially under https://github.com/lxc/lxd/issues/4631 but after two weeks to and fro of tweaking and testing, I couldn’t move forward at all. The problem is, that lxd snap starts before network is ready, here the relevant syslog entry

Jun 27 07:20:13 kai02 lxd.daemon[978]: err="listen tcp 192.168.88.102:8443: bind: cannot assign requested address" lvl=
Jun 27 07:20:14 kai02 systemd-networkd[741]: enp0s31f6: Gained carrier
Jun 27 07:20:14 kai02 kernel: e1000e: enp0s31f6 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Jun 27 07:20:14 kai02 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): enp0s31f6: link becomes ready
Jun 27 07:20:15 kai02 systemd-networkd[741]: enp0s31f6: Gained IPv6LL
Jun 27 07:20:15 kai02 systemd-timesyncd[719]: Network configuration changed, trying to establish connection.
Jun 27 07:20:18 kai02 lxd.daemon[978]: lvl=warn msg="Raft: Heartbeat timeout from \"\" reached, starting election" t=20
Jun 27 07:20:21 kai02 systemd-networkd[741]: enp0s31f6: DHCPv4 address 192.168.88.102/24 via 192.168.88.1

@stgraber suggested on github to block lxd from starting until network is up, so what I did was to start with adding an ExecStartPre=/bin/sleep 20 … this helped but this doesnt really solve anything, right? So I ended up with ExecStartPre=/lib/systemd/systemd-networkd-wait-online --interface=enp0s31f6 … here the full unit file:

$ cat snap.lxd.daemon.service 
[Unit]
# Auto-generated, DO NOT EDIT
Description=Service for snap application lxd.daemon
Requires=snap-lxd-7586.mount
Wants=network-online.target
After=snap-lxd-7586.mount network-online.target
X-Snappy=yes

[Service]
ExecStartPre=/lib/systemd/systemd-networkd-wait-online --interface=enp0s31f6
ExecStart=/usr/bin/snap run lxd.daemon
SyslogIdentifier=lxd.daemon
Restart=always
WorkingDirectory=/var/snap/lxd/7586
ExecStop=/usr/bin/snap run --command=stop lxd.daemon
ExecReload=/usr/bin/snap run --command=reload lxd.daemon
TimeoutStopSec=100
Type=simple

[Install]
WantedBy=multi-user.target

but this actually fails to do the trick. I also modified the unit to wait for lxdfan0 to start, because I had the same problem with the unit starting too early. I changed the after to snap.lxd.daemon.service (not sure where you get the lxd-containers.service, but that probably applies if you build from source, right?)

$ cat /lib/systemd/system/lxd-host-dns.service 
[Unit]
Description=LXD host DNS service
After=snap.lxd.daemon.service

[Service]
Type=oneshot
ExecStartPre=/lib/systemd/systemd-networkd-wait-online --interface=lxdfan0
ExecStart=/usr/local/bin/lxdhostdns_start.sh
RemainAfterExit=true
ExecStop=/usr/local/bin/lxdhostdns_stop.sh
StandardOutput=journal

[Install]
WantedBy=multi-user.target

i naturally modified the start/stop scripts to use lxdfan0. Again, this runs for me if i use a sleep here, but using systemd-networkd-wait-online fails with

-- Unit lxd-host-dns.service has begun starting up.
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: veth3XRAOP
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: vethPB44BR
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: enp0s31f6
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: vethM1P8JU
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: veth7KP1IC
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: managing: lxdfan0
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: lo
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: lxdfan0-mtu
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: veth21DABR
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: lxdfan0-fan
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: vethXHDGFH
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: wlp1s0
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: vethECXIQM
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: veth0BGHQS
Jun 27 07:36:28 kai02 lxdhostdns_start.sh[6692]: The specified interface lxdfan0 is managed by systemd-networkd. Operation refused.
Jun 27 07:36:28 kai02 lxdhostdns_start.sh[6692]: Please configure DNS settings for systemd-networkd managed interfaces directly in their .network files.
Jun 27 07:36:28 kai02 systemd[1]: lxd-host-dns.service: Main process exited, code=exited, status=1/FAILURE
Jun 27 07:36:28 kai02 systemd[1]: lxd-host-dns.service: Failed with result 'exit-code'.
Jun 27 07:36:28 kai02 systemd[1]: Failed to start LXD host DNS service.
-- Subject: Unit lxd-host-dns.service has failed

Could you kindly help me out of this mess? My lxd init is basically the defaults for the first cluster machine. I cant really imagine adding more machines to the cluster, if I can’t even run one host straight.

Hey, @simos, I think that this document about Ubuntu 18.04 might provide the solution to our problem.

https://www.linuxjournal.com/content/have-plan-netplan

It talks about his struggle to set up split-DNS on his Ubuntu Desktop. I didn’t follow the instructions (will do so when I get home) but just by looking at it sounds like what we need.

Hey @simos, I followed your instructions (creating a lxd-host-dns.service file in /etc/systemd/system, etc), but now when that service is enabled, I get the problem that containers cannot translate a domain name (eg microsoft.com) to an IP address. And as soon as I stop that service (the lxd-host-dns) with systemd, I can doing DNS requests again. Do you know why this is happening? And how it can be fixed?

@akarelas-pt so are you suggesting to revert to ifupdown (or better yet ifupdown2) inside the containers? I already noticed a number of shortcomings of netplan.io since I encountered it (on a host with a fresh Ubuntu 18.04 install instead of an upgraded one).

No @Hvergelmir, everything I wrote in this forum thread is rubbish. That’s because I don’t know much about Linux Networking.

However THIS here suggestion (also by me) seems very good to me, and even though I don’t know enough about Linux Network to suggest it, it has worked excellently at my setups (both linux servers and desktops, and also both snap and apt-based installations): A way to resolve container.lxd from host, in all cases

Thanks for the response. Guess I have some more reading to do.

Hi @simos
When i use this instruction, the container name .lxd is ready in host but when go inside of container i can’t ping domain names like google.com

what should i do at this situation?

@alizowghi,

One way (that worked great for me) is to follow the very simple instructions at the top of this page: A way to resolve container.lxd from host, in all cases - you’ll need to undo anything you’ve done until now though.

1 Like

Thank you @akarelas-pt
lxdbr0 address with reboot doesn’t change?

I don’t know, @alizowghi. But I’ve had this set-up on 2 PCs and it’s never changed on me.

@alizowghi,
It probably doesn’t change because if you type: lxc network show lxdbr0 you get this, which maybe implies that the IP address is part of the network’s configuration:

config:
  ipv4.address: 10.100.12.34/24
  ipv4.nat: "true"
  ipv6.address: faaa:faa:7777:bbbb::1/64
  ipv6.nat: "true"
description: ""
name: lxdbr0
type: bridge
used_by:
1 Like

I tested your solution and till now is working very well.
I think that’s right.