Quick question regarding managed bridge configuration and when it runs

Hi All,

Just a quick question.
Basically I have a “frankenstein” of a server where I have installed LXD and allowed LXD to manage and setup some basic bridges with IP addresses.
I then have used saltstack to configure ifupdown2 on the same server and two new bridges are created in that by way of the /etc/network/interfaces file. These bridges have different names than the lxd controlled ones.

On first deployment it all seems to work perfectly. However, on rebooting the server, it seems something is wiping the LXD bridge configuration, (possibly over-riding it somehow), even though LXD still thinks the bridges exist, they have actually gone.

I was wondering at what point on boot up sequence the lxd networking config kicks and creates the bridges so that they persist, this may help me stop whatever it is overwriting the bridge configuration by delaying things etc.

Many thanks,

Jon.

OK figured out if I then manually run lxc-net.service “systemctl start lxc-net” then the bridges come back.

OK thought it worked but it seems that after a reboot and running the service command again the bridges don’t appear? I’m confused lol.

Please can you clarify what you mean by:

Do the bridge interfaces still exist, but with no IPs, or are they being disconnected from the containers?

The bridge interfaces are not there at all. But they are still showing in the LXD db as they show in lxc network list

However there are no LXD containers existing at this time. Wondering if there needs to be containers for the actual bridges to survive a reboot.

They are there initially on creation with the command lxc network create blah…, but after the reboot they don’t re-appear. I’m assuming something else is wiping the bridge config.
I’m going to see if I get the same behaviour using openvswitch bridge driver instead.

So systemctl start lxc-net is related to “lxc” only, not LXD, so you shouldn’t be using that at all (unless you’re mixing LXD and LXC on the same system).

LXD creates the bridges on LXD start up, so it doesn’t persist them anywhere.

You can do sudo systemctl reload snap.lxd.daemon to trigger it to re-create network bridges if needed.

1 Like

OK I will try that in future. I’ve also figured out the bridges seem to be recreated on startup if I use the openvswitch bridge driver so I will do that for now and leave my custom bridges that I create with salt using linuxbridge. That way they don’t seem to conflict.

Just noticed the ovs bridges didn’t seem to appear until about 5 mins or more after startup? Is this some kind of timer or wait condition?

Thanks! :slight_smile:

Jon.

OVS bridges are persisted by the underlying OVS system. But no there shouldnt be a 5 minute wait, unless you are not marking any of your LXD containers to start on boot, and LXD isn’t actually starting up until the first time you initiate a lxc command (which is also when it would setup the bridge networks).

1 Like

I was just gonna say, yeah the bridges appear when I ran “lxc network list”, but yeah I have no lxc containers running at the moment so that is why they are not appearing until I prompt with a command.

I will just have to make sure there is a container in existence before I start my Virtual machines which are also using the same bridges (but they are proxmox VM’s).

Cheers,
Jon.

Ah yes that would make sense.

@stgraber is there a way to make LXD always start at boot even if no instances are configured to start at boot, so the network is created?

I’ve just worked around it by using a startup state in salt which runs “lxc network show” when the server boots.

That should be fine.

The plan is that would only be used if there were no containers at that point in time but generally there should be a few on the servers anyway.

1 Like