Managing the LXD snap

Introduction

The LXD snap produced by the LXD team is the preferred way to consume LXD. This allows the LXD team to distribute all of LXD’s dependencies in one package and have LXD run in a consistent environment whilst allowing it to be installed on many different Linux distributions. This greatly simplifies support as other than the kernel version, everything else is going to be identical on all systems.

The snapd packaging system uses the concept of “tracks” and “channels” to allow the user to choose a specific series of the software to install based on their appetite for stability vs new features.

By default snapd will automatically update LXD to the most recent version in the user’s chosen channel up to four times a day.

However as LXD is often used to run critical services some users may prefer to exercise a greater level of control over when LXD is updated. There are several approaches that can be taken to achieve this goal and this guide will go through each approach available.

LXD Tracks and Channels

When installing the LXD snap, you can specify the channel as follows:

sudo snap install lxd --channel=latest/stable

The channel specified is made up of two components; the track and the risk level.

So in this case, we are specifying the latest track with a risk level of stable, meaning that your LXD version will contain all of the latest features, but will be updated when the LXD team decide a feature is ready and no issues have been revealed by users running same revision on the more riskier branches (edge and candidate).

In addition to the latest track, the LXD team also maintains 3 LTS (long term support) tracks that receive only bug fixes and do not receive any new features. These are much ‘slower’ tracks that do not change frequently.

The stable tracks are currently: 5.0, 4.0, 3.0 and 2.0.

Just like the latest track, you can also specify your risk preference on these tracks by specifying stable, candidate or edge. However these risk levels are still relative to the overall channel’s risk (i.e the 5.0/edge channel is less risky than the latest/edge channel as it will only include bug fixes and not new features).

E.g.

sudo snap install lxd --channel=5.0/stable

Pinned Feature Channels

Although snapd does allow some control over when updates are applied (see Refresh Windows below), it is not currently possible to disable automatic updates entirely.

However you may need a feature from one of the feature releases available in the “latest” track, but do not want to take on the risk profile of even the latest/stable channel with automatic updates.

To allow more fine grained control over the version of LXD you are running, the LXD team also provides “pinned” channels for the current and previous releases that do not get updated.

E.g.

sudo snap install lxd --channel=5.n/stable

This will install the specific version and no further automatic updates will occur, but only if the version picked is not the latest version. If, however, you pick the latest release version it will receive the same cherry-picked updates as the latest/stable channel until the next release is made, at which time it will receive no further updates.

When you want to upgrade to the next feature version you can run:

sudo snap refresh lxd --channel=5.n/stable

You can see all of the available channels by running:

snap info lxd

System Refresh Windows

Snapd allows some degree of control over when updates are applied so that the time frame when updates are applied can be restricted to times that are suitable for dealing with any unexpected issues that may arise from the update.

These settings apply to all snaps installed on the system, not just LXD.

You can set specific time ranges when refreshes can occur using the refresh.timer setting.

The following example asks the system to only refresh snaps between 4.00am and 7.00am, and 7.00pm and 10:10pm:

sudo snap set system refresh.timer=4:00-7:00,19:00-22:10

You can also delay the refresh from happening until a certain date by setting the refresh.hold setting.

E.g.

sudo snap set system refresh.hold="$(date --date=tomorrow +%Y-%m-%dT%H:%M:%S%:z)"

However after 90 days a refresh will occur irrespective of the value of refresh.hold.

Please see Managing updates | Snapcraft documentation for more information on tuning refresh windows.

Refresh behavior in clusters

LXD clusters must all run the same LXD version. It is also recommended that they run the exact same amount of bug fixes as the API could get very inconsistent if that’s not the case.

All LXD servers in a cluster should therefore be configured to be on the same track/channel and if cohorts are in use, must also be part of the same cohort (see more on cohorts below).

As soon as one of the servers refreshes and detects it’s now running on a newer version than the rest, all cluster traffic will be held until things are consistent again. All the other servers will detect this and trigger a self-refresh so that the entire cluster is refreshed as soon as possible.

Cohorts Pinning

Snapd supports the concept of “cohorts”. This is a snapshot of a package’s current revisions at a point in time. You can then install that package on one or more systems using the cohort’s snapshot combined with the usual channel/risk pair to ensure all systems use the same revision. Additionally the cohort snapshot is retained for 90 days after which a new snapshot of the package’s revision list is made and then retained for a further 90 days. In this way you can limit the amount of updates that snapd will install to once per 90 days.

This combined with the refresh windows discussed above allow you to both limit the frequency of automatic updates as well as specify the time range when they can occur.

NOTE: This isn’t recommended by the LXD team unless someone is paying very close attention to what’s released in the original channel and refreshes the cohort whenever a security or critical bugfix has been pushed. Otherwise, such long term version pinning can result in critical security issues being present on the systems.

snap create-cohort lxd

Which will output:

cohorts:
  lxd:
    cohort-key: <generated key>

Then use the generated key displayed to install lxd using the specific cohort snapshot:

snap install --cohort=<generated key> lxd

Snap Store Proxy

If you manage multiple LXD nodes in a large deployment and cohorts are not sufficient because you need absolute control over when updates are applied, then the Snap Store Proxy may provide the solution you need.

The Snap Store Proxy is a separate application that sits between the snap client command on your nodes where you want to install LXD and the Internet. With this running you can then specify that the Snap Store Proxy only makes a specific revision of LXD available to install for each architecture.

There is a detailed guide on how to set up the Snap Store Proxy here Introduction | Snap Store Proxy documentation.

Once your snap clients are configured to use your the proxy, you can then instruct it to only make a specific LXD revision available using:

sudo snap-proxy override lxd <channel>=<revision>

E.g.

sudo snap-proxy override lxd stable=15457

You can see a list of current revisions by running:

snap info lxd

The number in brackets beside the date is the revision number.

There is more information on snap proxy overrides here Snap overrides | Snap Store Proxy documentation.

Snap store proxy has a free tier with a limit on the number of nodes that can be connected to it. See Registration | Snap Store Proxy documentation for more information on device limits and if you require more then please contact Canonical using Contact Canonical | Support | Ubuntu.

Recommendations

Development system:

  • Use the latest/stable channel.
  • Set the refresh window to be outside of work hours.

Production server:

  • Use the latest/stable channel if you need the latest features and can specify a frequent refresh window.
  • Use a $VERSION/stable channel if you want to avoid the automatic release upgrade and have time to do the channel change once a month.
  • Use an $LTS/stable channel if you don’t need any of the features that were added since the last LTS release.
  • Set a refresh window to match your system maintenance window.

Staging server:

  • Same as production server, but use the matching “candidate” channel so you can identify any breaking change ahead of it hitting production.

Large deployments and/or air-gapped environments:

  • Set up a Snap Store Proxy.
  • Run test systems on the upstream stable channels.
  • Pin your proxy to the upstream revisions you’ve tested.
  • Frequently re-evaluate and update your pinning.
7 Likes
Lxd snap stuck in 4.0/stable/ubuntu-20.04
Repeatable LXD installations
"ZFS doesn't support restoring from snapshots other than the latest one"
LXD latest version issue
How does LXC interact with e.g. lvm?
Lxc live snapshot
Macvlan networking suddenly stopped working; cloud-init not setting up interfaces and DHCP failing
Policy for backporting fixes to LTS releases
LXD and Snap Auto-Update
Error: Failed importing MACHINE_NAME_HERE: Failed creating instance record: Unknown configuration key: volatile.cloud-init.instance-id
VM's frozen at boot after snap LXD update (4.24) io_uring problems
Guide for upgrading?
4.21 can't start containers mapping /tmp/.X11-unix/X0
4.21 can't start containers mapping /tmp/.X11-unix/X0
LXD cluster broken since last snap update: ( lxd 4.19 21723 )
SOLVED: LXD packages for Ubuntu 20.04
Repeatable LXD installations
Failures on installing LXD on Ubuntu20.04 and systemctl errors
Migrate deb 3.0 to snap 4.0
LXD 3.0.3: Failed to create proxy devices listening on host port: Failed setns to container user namespace: Invalid argument
Fs layout for lxd hypervisor
Cluster upgraded automatically to 4.4 a few minutes ago, and now all my containers have no IPs
Can not start bash on a container
Snap fix channel
Some or the other error with lxd snap auto upgrades
Lxd snap stuck in 4.0/stable/ubuntu-20.04
LXD wont start after spontaneous snap update
LXD stable channel policy
LXD wont start after spontaneous snap update
Snap auto refresh broke LXD clusters (VMs not reachable in LXD 5.0.1)
Snap auto refresh broke LXD clusters (VMs not reachable in LXD 5.0.1)
Pending SNAP update

Typo here, should be /snap-store-proxy i guess Discourse might have nagged you about posting the same link more than once?

Thanks, fixed, not sure what happened there.

1 Like

As you had below, one doesn’t need to sudo to run snap info <snap>

1 Like

If you first sudo snap login, then you do not need sudo in any of the subsequent snap commands.

3 Likes