How to access container alex as alex.lxd from host?


(Alexander Karelas) #21

By the way, I never execute the “temporary” solution that you provide in your blogpost, just the “permanent” one. I hope I’m doing that fine.


#22

I have done some more testing. My overall view is that if you get DNS in Current Scopes, then the per-link configuration for lxdbr0 works.

me@my-VirtualBox:~$ systemd-resolve --status lxdbr0
Link 3 (lxdbr0)
Current Scopes: DNS
LLMNR setting: yes
MulticastDNS setting: no
DNSSEC setting: no
DNSSEC supported: no
DNS Servers: 10.10.10.1
DNS Domain: lxd
me@my-VirtualBox:~$ ping mycontainer.lxd
PING mycontainer.lxd(mycontainer.lxd (fd42:8196:99f3:52ad:216:3eff:fe0f:bacb)) 56 data bytes
64 bytes from mycontainer.lxd (fd42:8196:99f3:52ad:216:3eff:fe0f:bacb): icmp_seq=1 ttl=64 time=0.053 ms

The permanent solution in my blog post is foolproof, because when systemd-networkd starts before lxd.service is completed, there is no lxdbr0 and the per-link network configuration is not applied anyway.

For now, the temporary configuration seems to be the most reliable to work.

What I suggest:

  1. Try the temporary solution and make sure that it works reliably for you.
  2. Create a systemd Unit that runs after lxd.service (once lxdbr0 gets configure),
    and this systemd Unit will run the temporary command.

#23

I have updated my blog post with a more robust solution. Running the temporary command through systemd.


(Alexander Karelas) #24

Your service file does not work on my computer during boot (Ubuntu MATE).

Here’s the error I get:

$ systemctl status lxd-host-dns.service

Ιουν 22 17:01:03 karjala-vmware systemd[1]: Starting LXD host DNS service...
Ιουν 22 17:01:03 karjala-vmware lxdhostdns_start.sh[1201]: Device "lxdbr0" does not exist.
Ιουν 22 17:01:03 karjala-vmware lxdhostdns_start.sh[1201]: Unknown interface lxdbr0: No such device
Ιουν 22 17:01:03 karjala-vmware systemd[1]: lxd-host-dns.service: Main process exited, code=exited, status=1/FAILURE
Ιουν 22 17:01:03 karjala-vmware systemd[1]: lxd-host-dns.service: Failed with result 'exit-code'.
Ιουν 22 17:01:03 karjala-vmware systemd[1]: Failed to start LXD host DNS service.

The reason is that the lxd-containers service does not exist on my computer. Should it?

root@karjala-vmware:~# systemctl status lxd-containers.service 
Unit lxd-containers.service could not be found.

(Alexander Karelas) #25

I don’t have the apt package of lxd installed (which contains the lxd-containers.service file)

Should I install it?

Doesn’t it defeat the whole purpose of snap, if I install lxd apt?


#26

Indeed, you have the snap version of LXD.
You can attach then the new service to the corresponding LXD snap service.
Have a look at

systemctl | grep lxd

to get a list.


(Alexander Karelas) #27

The initial wait, during the first DNS request of every container’s name, makes me want to replace systemd-resolved with dnsmasq on the host. I wonder if it’s a good idea.


(Alexander Karelas) #28

You have a typo in your tutorial: You say:

ExecStop=/usr/local/bin/lxdhostdnsi_stop.sh

(there’s an extra i)


#29

Thanks for reporting the typo. I just fixed it.

Indeed, there is an initial wait for the timeout of the default DNS server.
I updated the blog post with a troubleshooting section, which describes the issue.
There should be a way for systemd-resolved to just ask the lxdbr0 DNS server and not need to also consult the default DNS server. systemd-resolved can deduce that because we specified that the .lxd domains are served by the lxdbr0 DNS server.

There should either be some special configuration in systemd for this, or there is a need for a bug report on systemd.


(Alexander Karelas) #30

@simos, I tried your suggestion to do: systemctl | grep lxd, in order to find a suitable unit to place in the After= field.

I tried placing all of them in After=, did systemctl daemon-reload after each one, but your service always failed during boot time. The error message is this:

Ιουν 25 09:45:50 karjala-vmware lxdhostdns_start.sh[1403]: Device "lxdbr0" does not exist. Ιουν 25 09:45:50 karjala-vmware lxdhostdns_start.sh[1403]: Unknown interface lxdbr0: No such device

So we still haven’t solved the problem for Snap users (which is the most important), only for Apt users.

I wonder if it’s a good solution to replace systemd-resolved with dnsmasq on the host, like I had in Ubuntu 16.04. What do you think about this?


(Alexander Karelas) #31

However, when I systemctl start your service after boot, and after I have logged in, that works, and the LXD containers hostnames resolve.


#32

First of all, thank for trying out with the different options for After.

Ideally, it should be better to figure out a way to make it work reliably on Ubuntu 18.04 without additional system changes.
I’ll look into this a bit later on a fresh 18.04 install. If you need to get a system working at once, then go ahead and switch to dnsmasq.


(Alexander Karelas) #33

P.S. I didn’t use StartAfter= option, but the After= option. Did I do it wrong?


#34

No, you did right. There is no StartAfter, it is After. I just fixed my typo above.


#35

Hey @simos, I keep having problems with this when using the fan networking and snap. I reported this initially under https://github.com/lxc/lxd/issues/4631 but after two weeks to and fro of tweaking and testing, I couldn’t move forward at all. The problem is, that lxd snap starts before network is ready, here the relevant syslog entry

Jun 27 07:20:13 kai02 lxd.daemon[978]: err="listen tcp 192.168.88.102:8443: bind: cannot assign requested address" lvl=
Jun 27 07:20:14 kai02 systemd-networkd[741]: enp0s31f6: Gained carrier
Jun 27 07:20:14 kai02 kernel: e1000e: enp0s31f6 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Jun 27 07:20:14 kai02 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): enp0s31f6: link becomes ready
Jun 27 07:20:15 kai02 systemd-networkd[741]: enp0s31f6: Gained IPv6LL
Jun 27 07:20:15 kai02 systemd-timesyncd[719]: Network configuration changed, trying to establish connection.
Jun 27 07:20:18 kai02 lxd.daemon[978]: lvl=warn msg="Raft: Heartbeat timeout from \"\" reached, starting election" t=20
Jun 27 07:20:21 kai02 systemd-networkd[741]: enp0s31f6: DHCPv4 address 192.168.88.102/24 via 192.168.88.1

@stgraber suggested on github to block lxd from starting until network is up, so what I did was to start with adding an ExecStartPre=/bin/sleep 20 … this helped but this doesnt really solve anything, right? So I ended up with ExecStartPre=/lib/systemd/systemd-networkd-wait-online --interface=enp0s31f6 … here the full unit file:

$ cat snap.lxd.daemon.service 
[Unit]
# Auto-generated, DO NOT EDIT
Description=Service for snap application lxd.daemon
Requires=snap-lxd-7586.mount
Wants=network-online.target
After=snap-lxd-7586.mount network-online.target
X-Snappy=yes

[Service]
ExecStartPre=/lib/systemd/systemd-networkd-wait-online --interface=enp0s31f6
ExecStart=/usr/bin/snap run lxd.daemon
SyslogIdentifier=lxd.daemon
Restart=always
WorkingDirectory=/var/snap/lxd/7586
ExecStop=/usr/bin/snap run --command=stop lxd.daemon
ExecReload=/usr/bin/snap run --command=reload lxd.daemon
TimeoutStopSec=100
Type=simple

[Install]
WantedBy=multi-user.target

but this actually fails to do the trick. I also modified the unit to wait for lxdfan0 to start, because I had the same problem with the unit starting too early. I changed the after to snap.lxd.daemon.service (not sure where you get the lxd-containers.service, but that probably applies if you build from source, right?)

$ cat /lib/systemd/system/lxd-host-dns.service 
[Unit]
Description=LXD host DNS service
After=snap.lxd.daemon.service

[Service]
Type=oneshot
ExecStartPre=/lib/systemd/systemd-networkd-wait-online --interface=lxdfan0
ExecStart=/usr/local/bin/lxdhostdns_start.sh
RemainAfterExit=true
ExecStop=/usr/local/bin/lxdhostdns_stop.sh
StandardOutput=journal

[Install]
WantedBy=multi-user.target

i naturally modified the start/stop scripts to use lxdfan0. Again, this runs for me if i use a sleep here, but using systemd-networkd-wait-online fails with

-- Unit lxd-host-dns.service has begun starting up.
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: veth3XRAOP
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: vethPB44BR
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: enp0s31f6
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: vethM1P8JU
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: veth7KP1IC
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: managing: lxdfan0
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: lo
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: lxdfan0-mtu
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: veth21DABR
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: lxdfan0-fan
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: vethXHDGFH
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: wlp1s0
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: vethECXIQM
Jun 27 07:36:28 kai02 systemd-networkd-wait-online[6684]: ignoring: veth0BGHQS
Jun 27 07:36:28 kai02 lxdhostdns_start.sh[6692]: The specified interface lxdfan0 is managed by systemd-networkd. Operation refused.
Jun 27 07:36:28 kai02 lxdhostdns_start.sh[6692]: Please configure DNS settings for systemd-networkd managed interfaces directly in their .network files.
Jun 27 07:36:28 kai02 systemd[1]: lxd-host-dns.service: Main process exited, code=exited, status=1/FAILURE
Jun 27 07:36:28 kai02 systemd[1]: lxd-host-dns.service: Failed with result 'exit-code'.
Jun 27 07:36:28 kai02 systemd[1]: Failed to start LXD host DNS service.
-- Subject: Unit lxd-host-dns.service has failed

Could you kindly help me out of this mess? My lxd init is basically the defaults for the first cluster machine. I cant really imagine adding more machines to the cluster, if I can’t even run one host straight.


(Alexander Karelas) #37

Hey, @simos, I think that this document about Ubuntu 18.04 might provide the solution to our problem.

https://www.linuxjournal.com/content/have-plan-netplan

It talks about his struggle to set up split-DNS on his Ubuntu Desktop. I didn’t follow the instructions (will do so when I get home) but just by looking at it sounds like what we need.


(Alexander Karelas) #38

Hey @simos, I followed your instructions (creating a lxd-host-dns.service file in /etc/systemd/system, etc), but now when that service is enabled, I get the problem that containers cannot translate a domain name (eg microsoft.com) to an IP address. And as soon as I stop that service (the lxd-host-dns) with systemd, I can doing DNS requests again. Do you know why this is happening? And how it can be fixed?