LXD 3.17 has been released

Introduction

The LXD team is very excited to announce the release of LXD 3.17!

The main new feature for this release is the ability to use LXD storage pools to store container tarballs and images rather than the host’s root filesystem.

Most of the rest of the changes in this release are refactoring and porting existing logic to new internal APIs. This work while invisible to our users has let us uncover a number of long time issues, improve our testing and simplify a lot of complex logic.

Enjoy!

New features

Storage pool backed image tarballs and backups

Two new server configuration keys have been introduced:

  • storage.backups_volume
  • storage.images_volume

They can each be set to a different, unused, storage volume using a POOL-NAME/VOLUME-NAME syntax. Existing data stored on the system will be moved to the storage volume.

This allows for systems with very limited available space on their root filesystem to run LXD with almost all data stored onto LXD managed storage pools.

stgraber@castiana:~$ lxc storage volume create default backups
Storage volume backups created
stgraber@castiana:~$ lxc storage volume create default images
Storage volume images created
stgraber@castiana:~$ lxc config set storage.backups_volume default/backups
stgraber@castiana:~$ lxc config set storage.images_volume default/images

Container configuration as YAML on lxc init and lxc launch

It is now possible to pass complex configuration and devices right at container creation time by having those stored in a YAML file which is read from standard input during lxc init and lxc launch.

stgraber@castiana:~$ cat gui.yaml 
config:
  environment.DISPLAY: :0
  environment.PULSE_LATENCY_MSEC: "30"
  environment.PULSE_SERVER: /mnt/.pulse-native
  environment.QT_X11_NO_MITSHM: "1"
devices:
  gpu:
    type: gpu
  pulse:
    bind: container
    connect: unix:/run/user/1000/pulse/native
    listen: unix:/mnt/.pulse-native
    mode: "0666"
    security.gid: "1000"
    security.uid: "1000"
    type: proxy
  x11:
    bind: container
    connect: unix:@/tmp/.X11-unix/X0
    listen: unix:@/tmp/.X11-unix/X0
    security.gid: "1000"
    security.uid: "1000"
    type: proxy

stgraber@castiana:~$ lxc launch ubuntu:18.04 gui-steam < gui.yaml 
Creating gui-steam
Starting gui-steam

stgraber@castiana:~$ lxc config show gui-steam
architecture: x86_64
config:
  environment.DISPLAY: :0
  environment.PULSE_LATENCY_MSEC: "30"
  environment.PULSE_SERVER: /mnt/.pulse-native
  environment.QT_X11_NO_MITSHM: "1"
  image.architecture: amd64
  image.description: ubuntu 18.04 LTS amd64 (release) (20190813.1)
  image.label: release
  image.os: ubuntu
  image.release: bionic
  image.serial: "20190813.1"
  image.version: "18.04"
  volatile.base_image: 2dd611e2689a8efc45807bd2a86933cf2da0ffc768f57814724a73b5db499eac
  volatile.eth0.host_name: vethe8c1ff8b
  volatile.eth0.hwaddr: 00:16:3e:65:36:88
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},    {"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},    {"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
devices:
  gpu:
    type: gpu
  pulse:
    bind: container
    connect: unix:/run/user/1000/pulse/native
    listen: unix:/mnt/.pulse-native
    mode: "0666"
    security.gid: "1000"
    security.uid: "1000"
    type: proxy
  x11:
    bind: container
    connect: unix:@/tmp/.X11-unix/X0
    listen: unix:@/tmp/.X11-unix/X0
    security.gid: "1000"
    security.uid: "1000"
    type: proxy
ephemeral: false
profiles:
- default
stateful: false
description: ""

Notes

Backup API consistency

This release contains a small API breakage which is very unlikely to affect any of our users as it’s not in a part of the LXD API which is used by any of the existing tools.

This renames:

  • creation_date to created_at (CreatedAt in Go binding)
  • expiry_date to expires_at (ExpiresAt in Go binding)

This makes that API consistent with the other endpoints which expose timestamps.

New documentation website

A new documentation website is now being published at: https://linuxcontainers.org/lxd/docs/master/

Effort was put into better titles and grouping of topics to be easier to get started with LXD.
All content still comes straight from the doc directory in the upstream repository, making it easy for anyone to send updates on Github.

Improved initial user experience on Windows/MacOS

Windows and MacOS builds of the LXD client still default to a local remote, despite the fact that the LXD daemon cannot work on those operating systems.

This caused some confusion to users in the past, so rather than showing a confusing error about a missing unix socket, those users are now greeted with:

C:\Users\stgraber>lxc list
This client hasn't been configured to use a remote LXD server yet.
As your platform can't run native Linux containers, you must connect to a remote LXD server.

If you already added a remote server, make it the default with "lxc remote switch NAME".
To easily setup a local LXD server in a virtual machine, consider using: https://multipass.run

Ported to final Dqlite 1.0

Dqlite 1.0 final has now been released and as part of the release effort a number of improvements and small design changes went in.
LXD 3.17 is now based on that final 1.0 version.

Database rework

This release comes with some internal database reshuffling.
All the containers tables have now been renamed to instances and snapshots have now been split out to their own set of instances_snapshots tables, making it easier to enforce consistency through the schema.

If you have custom scripts that directly look for data in the database, they may very well need updating for this.

Container devices rework

LXD 3.17 concludes our effort to split out all our device handling code, making it much more modular and improve coding patterns and testing. LXD 3.16 had the nic, infiniband and proxy devices ported, 3.17 now ports unix-char, unix-block, usb, gpu and disk, completing the set.

Storage rework

A first set of commit has been included in this release which cleans up some of our storage backends and similar to the devices rework will begin us moving every storage backend to a new cleaner structure.

Bugs fixed

  • client: Use PathEscape rather than QueryEscape for URL part parts
  • doc: Add documentation metadata
  • doc: Add required property to disk device type
  • doc: Update infiniband hwaddr docs
  • doc: Update NIC MTU descriptions
  • doc/server: Add missing key namespaces
  • doc/server: Add scope column
  • doc/server: Fix defaults for rbac
  • doc/storage: Clarify defaults for size
  • global: Remove accidentally included lxc.exe
  • i18n: Update translations from weblate
  • i18n: Update translation templates
  • lxc: Better error handling on non-Linux
  • lxc: Don’t show header in CSV output
  • lxc: Just show help on missing subcommand
  • lxc: Update for fixed backup fields
  • lxc/config: Detect non-Linux systems
  • lxc/copy: Really always strip volatile.last_state.power
  • lxc/file: Fix error handling
  • lxc/file: Intercept user cancelation
  • lxc/init: Fix usage with no args
  • lxc/remote: Add basic auth example
  • lxc-to-lxd: Remove dependency of lxd/device/config package
  • lxc-to-lxd: Remove dependency on deviceConfig package
  • lxd: Add call to devicesRegister function on startup
  • lxd: Add unit test for creating a snapshot
  • lxd: Check RemoveAll() error properly
  • lxd: Remove MS_LAZYTIME definition
  • lxd: Reorganised the uevent and inotify event handler startup
  • lxd/apparmor: Prevent writes to /proc/acpi/**
  • lxd/backup: Call tar with --numeric-owner
  • lxd/containers: Add DeviceEventHandler
  • lxd/containers: Add DeviceEventHandler to interface
  • lxd/containers: Add deviceHandleMounts to handle mount and unmount
  • lxd/containers: Add safety net for deviceStop() in case no device returned
  • lxd/containers: Fix description
  • lxd/containers: Fix inotify dynamic hotplug on existing container start
  • lxd/containers: Further switchover to lxd/device/config Device types
  • lxd/containers: Improve error messages
  • lxd/containers: Modify deviceStop and deviceRemove to skip validation errors
  • lxd/containers: Move device folder creation before device setup during boot
  • lxd/containers: Move disk device to use device package
  • lxd/containers: Move missing device type validation into device package
  • lxd/containers: Remove deviceAttachMounts
  • lxd/containers: Remove device Register() after device start
  • lxd/containers: Remove gpu validation as moved to device package
  • lxd/containers: Remove old moved rootfs quota code
  • lxd/containers: Remove reference to non-existent infiniband nictype
  • lxd/containers: Remove unused arg from deviceAddCgroupRules
  • lxd/containers: Remove unused setupUnixDevice()
  • lxd/containers: Remove unused unix-char and unix-block code
  • lxd/containers: Remove unused unix-char and unix-block validation
  • lxd/containers: Remove unused USB code
  • lxd/containers: Remove unused USB related code
  • lxd/containers: Remove volatile device keys when device is actually removed
  • lxd/containers: Rename runConfig vars to runConf for consistency
  • lxd/containers: Simplify device validation now all devices are ported
  • lxd/containers: Supply all old devices to device Update() function
  • lxd/containers: Update all device major/minor parsing to use uint32
  • lxd/containers: Update deviceShiftMounts to ignore unmount requests
  • lxd/containers: Update deviceStart to call device’s Register() function
  • lxd/containers: Update gpu device support to use device package
  • lxd/containers: Update use of shared.RunCommandSplit
  • lxd/containers: Use shared.GetRootDiskDevice for dup root device detection
  • lxd/db: Adapt lxd/cluster to new dqlite sub-packages
  • lxd/db: Adapt lxd/cluster to Server -> Node rename
  • lxd/db: Adapt lxd/db/cluster to Server -> Node rename
  • lxd/db: Adapt lxd/db package to new dqlite driver import
  • lxd/db: Adapt lxd/db to new dqlite driver package
  • lxd/db: Adapt lxd/db to Server -> Node rename
  • lxd/db: Adapt main package to new dqlite sub packages
  • lxd/db: Adapt main package to Server -> Node rename
  • lxd/db: Adapt to changed dqlite.New() signature, not requiring NodeInfo
  • lxd/db: Adapt to new Server.Leader() API
  • lxd/db: Add basic unit tests for generated snapshot code
  • lxd/db: Add copy of cluster schema version 14
  • lxd/db: Add Delete and Rename methods to Snapshot db model
  • lxd/db: Add Dump parameter to db.OpenCluster()
  • lxd/db: Add initial Snapshot db model
  • lxd/db: Add project column to views
  • lxd/db: Add schema update 16 adding the instances_snapshots table
  • lxd/db: Add schema update function to rename containers-related tables
  • lxd/db: Add type column to images table
  • lxd/db: Amend InstanceFilter docstring
  • lxd/db: At shutdown, wait a bit for the in-flight force request to settle
  • lxd/db: Change container special-casing in db code generator
  • lxd/db: Change db code generator to comply with Go naming standards
  • lxd/db: Convert containerCreateInternal to use InstanceSnapshotCreate()
  • lxd/db: Convert dump to new client API
  • lxd/db: Create instances_snapshots_config in schema update 16
  • lxd/db: Create instances_snapshots_config_ref view
  • lxd/db: Create instances_snapshots_devices_config in schema update 16
  • lxd/db: Create instances_snapshots_devices in schema update 16
  • lxd/db: Create instances_snapshots_devices_ref view
  • lxd/db: Detect possible leadership change through ougoing dqlite connections
  • lxd/db: Drop obsolete snapshot-related tests
  • lxd/db: Drop Parent filter from Instance
  • lxd/db: Drop unneeded logic to relink snapshots to new nodes
  • lxd/db: Drop unused Container and ContainerFilter structs
  • lxd/db: Extract configUpdate method from ContainerConfigUpdate
  • lxd/db: Fix failing unit test
  • lxd/db: Fix selecting NULL description columns
  • lxd/db: Generate Snapshot db mapping code
  • lxd/db: Improve dqlite proxy error messages and abort both sides on error
  • lxd/db: Invoke data migration from db.OpenCluster, before schema updates
  • lxd/db: Make the db code generator handle indirect foreign key
  • lxd/db: Migrate snapshots to the new tables
  • lxd/db: No need to manually bootstrap
  • lxd/db: Pass a context to server.Cluster()
  • lxd/db: Properly account for project when moving ceph-based containers
  • lxd/db: Regenerate db code
  • lxd/db: Re-generate db mapper code
  • lxd/db: Regenerate db mappers code
  • lxd/db: Regenerate db schema
  • lxd/db: Regenerate global db schema
  • lxd/db: Remove legacy unit test making use of old snapshot apis
  • lxd/db: Replace references to the “containers” table with “instances”
  • lxd/db: Skip clustering-related unit tests, see issue #6122
  • lxd/db: Update db code generator to handle composite entity names
  • lxd/db: Update top-level daemon package to new auto-generated method names
  • lxd/db: Update unit tests after containers -> instances conversion
  • lxd/db: Use Client.Add() API
  • lxd/db: Use Client.Cluster() API
  • lxd/db: Use Client.Leader() API
  • lxd/db: Use Client.Remove() API
  • lxd/db: Use correct db APIs depending on the container type
  • lxd/db: Use new LeaderAddress() api
  • lxd/db: Use new snapshot db APIs in ClusterTx.SnapshotIDsAndNames()
  • lxd/db: Use new snapshots APIs in ContainerGetSnapshotsFull
  • lxd/db: Use new snapshots tables in ContainerNextSnapshot()
  • lxd/db: Use new snapshot tables in Cluster.ContainerGetSnapshots()
  • lxd/db: Use new snapshot tables in daemon patches
  • lxd/db: Use query.Transaction instead of manual tx management
  • lxd/db: Use WithServerBindAddress
  • lxd/devices: Add Register function and links USB implementation
  • lxd/devices: Change Update() to accept all old devices
  • lxd/devices: Fix dynamic hotplug of unix devices when they exist on startup
  • lxd/devices: Link gpu device
  • lxd/devices: Link up disk device
  • lxd/devices: Link up unix-char and unix-block devices
  • lxd/devices: Modify New function to return device even if validation fails
  • lxd/devices: Move empty device type validation into device package
  • lxd/devices: Move USB event handling into device package
  • lxd/devices: Remove disk related functions
  • lxd/devices: Remove gpu related code moved to device package
  • lxd/devices: Remove inotify code
  • lxd/devices: Rename USBDevice to USBEvent
  • lxd/devices: Update sorted devices usage
  • lxd/devices: Update use of Device type
  • lxd/devices/config: Change Devices type to map[string]Device
  • lxd/devices/config: Make device set sorting exported
  • lxd/devices/config: Replace DeviceNames() with Sorted() and Reversed()
  • lxd/devices/config/validate: Move function to be attached to Device type
  • lxd/devices/device: Link up none device type
  • lxd/devices/device/utils/usb/events: Add USB event handler functions
  • lxd/devices/device/utils/usb: Move bits into usb and device_utils_usb_events
  • lxd/devices/disk: Add disk device implementation
  • lxd/devices/disk: Add StorageVolumeMount and StorageVolumeUmount functions
  • lxd/devices/disk: Add validation for root disk having a pool property
  • lxd/devices/disk: Link StorageRootFSApplyQuota
  • lxd/devices/gpu: Add gpu implementation
  • lxd/devices/gpu: Move nvidia device loading to use resources package
  • lxd/devices/gpu: Update all device major/minor parsing to use uint32
  • lxd/devices/gpu: Update unix function usage
  • lxd/devices/gpu: Use device package validation functions
  • lxd/devices/gpu: Validate vendorid and productid
  • lxd/devices/infiniband: Add IB MAC functions
  • lxd/devices/infiniband: Remove unused code after switch to resources package
  • lxd/devices/infiniband: Update use of unix functions
  • lxd/devices/infiniband: Workaround weird sysfs behavior
  • lxd/devices/infiniband/physical: Improve MAC address support
  • lxd/devices/infiniband/physical: Switch to use resources package
  • lxd/devices/infiniband/physical: Update unix function usage
  • lxd/devices/infiniband/physical: Update use of unix device functions
  • lxd/devices/infiniband/sriov: Improve MAC address support
  • lxd/devices/infiniband/sriov: Switches to use resources package
  • lxd/devices/infiniband/sriov: Update unix function usage
  • lxd/devices/infiniband/sriov: Update use of unix device functions
  • lxd/devices/inotify: Move inotify functions to device package
  • lxd/devices/instance: Add DeviceEventHandler function
  • lxd/devices/instance: Add LocalDevices() to interface
  • lxd/devices/instance: Add RootfsPath() to InstanceIdentifier interface
  • lxd/devices/network: Add networkValidMAC
  • lxd/devices/network: MTU inheriting from parent on bridged devices
  • lxd/devices/network: Remove NetworkSRIOVGetFreeVFInterface
  • lxd/devices/nic: Update bridged and p2p types to new Update signature
  • lxd/devices/nic: Update nic validation of hwaddr
  • lxd/devices/nic/bridged: DHCP release fixes
  • lxd/devices/nic/sriov: Add getFreeVFInterface after moving from shared utils
  • lxd/devices/none: Add none device type
  • lxd/devices/proxy: Update validation to use d.instance.ExpandedDevices()
  • lxd/devices/proxy: Use device package validation functions
  • lxd/devices/runconfig: Add RootFS support
  • lxd/devices/runconfig: Add Uevents slice
  • lxd/devices/runconfig: Change mount Shift to OwnerShift
  • lxd/devices/runconfig: Fix typo in comment
  • lxd/devices/unix: Add implementation for unix-char and unix-block devices
  • lxd/devices/unix: Add unix event handling functions
  • lxd/devices/unix: Clarify required property logic
  • lxd/devices/unix: Comment clarification
  • lxd/devices/unix: Device management function rework
  • lxd/devices/unix: Ensure unix devices are mounted with MountOwnerShiftStatic
  • lxd/devices/unix: Fix double device name encoding in file name
  • lxd/devices/unix: Make unixDeviceAttributes unexported
  • lxd/devices/unix: Move some config validation functions into device package
  • lxd/devices/unix: Update all device major/minor parsing to uin32
  • lxd/devices/unix: Update device removal functions to accept file filter
  • lxd/devices/unix: Update use of unixDeviceAttributes
  • lxd/devices/unix: Various small improvements
  • lxd/devices/usb: Add unexported usbIsOurDevice and switch to USBEvent
  • lxd/devices/usb: Add USB device implementation
  • lxd/devices/usb: Add USB event handling functions
  • lxd/devices/usb: Clarify required property logic
  • lxd/devices/usb: Remove unused function
  • lxd/devices/usb: Update Register() to be called by post start hook
  • lxd/dnsmasq: Update version check to use shared.RunCommandCLocale
  • lxd/main_forkuevent: Fix error when >3 arguments used (normal case)
  • lxd/migration: Remove unused Snapshots() function from interface
  • lxd/networks: Handle error from dnsmasq version check
  • lxd/networks: Remove old dnsmasq.leases file on network start
  • lxd/patches: Update sorted devices usage
  • lxd/projects: Remove dependency on deviceConfig package
  • lxd/response: Show wrapped errors
  • lxd/seccomp: Update use of shared.RunCommandSplit
  • lxd/storage: Add storageRootFSApplyQuota
  • lxd/storage: Add storageVolumeMount and storageVolumeUmount
  • lxd/storage: Fix bad UsedBy check
  • lxd/storage: Move btrfs migration code
  • lxd/storage: Move ceph migration code
  • lxd/storage: Move ContainerGetParentAndSnapshotName to shared
  • lxd/storage: Move ContainerPath() to storage package
  • lxd/storage: Move Create{Container,Snapshot}Mountpoint to storage
  • lxd/storage: Move get*MountPoint() to storage package
  • lxd/storage: Move storage cgo to storage package
  • lxd/storage: Move storage_utils to storage/utils
  • lxd/storage: Move zfs migration code
  • lxd/storage: Remove ContainerCanRestore from storage interface
  • lxd/storage: Remove Image{Umount,Mount} from storage interface
  • lxd/storage: Remove shared code from backends
  • lxd/storage/lvm: Log actual error
  • lxd/storage/quota: Move use of Major and Minor functions to unix package
  • lxd/storage/zfs: Fix error reporting
  • lxd/sys: Add CGroupBlkioWeightController check
  • lxd: Update for fixed backup fields
  • lxd: Update tests to use updated Devices type
  • lxd: Update top-level unit-tests for DB changes
  • lxd: Update use of Devices type
  • lxd: Use unix.MS_LAZYTIME
  • Makefile: Fix sqlite manifest path
  • Makefile: Fix update-schema target
  • shared: Handle symlinks in FileCopy()
  • shared/api: Fix backup timestamps
  • shared/container: Add IsDeviceID validation
  • shared/container: Improve comments on IsRootDiskDevice
  • shared/container: Move global hex regex into specific function
  • shared/container: Remove device related validation functions
  • shared/containerwriter: Updates use of GetFileStat
  • shared/simplestreams: Make golint clean
  • shared/simplestreams: Record all images
  • shared/simplestreams: Remove dead code
  • shared/simplestreams: Rename index structs
  • shared/simplestreams: Rename internal functions
  • shared/simplestreams: Rename product structs
  • shared/simplestreams: Rename SimpleStreamsFile
  • shared/simplestreams: Rename ssDefaultOS
  • shared/simplestreams: Split index/manifest out
  • shared/simplestreams: Split out sortedImages
  • shared/util: Add RunCommandCLocale() and update RunCommandSplit()
  • shared/util: Remove Major and Minor functions
  • tests: Add golint for storage package
  • tests: Add infiniband MAC tests
  • tests: Add more bridged DHCP release tests
  • tests: Add proxy tests for invalid config
  • tests: Add GPU tests
  • tests: Add simplestreams to golint
  • tests: Add storage quota tests
  • tests: Add test for bridged MTU parent inheritance
  • tests: Add test for ipvlan MTU parent inheritance
  • tests: Add test for macvlan MTU parent inheritance
  • tests: Add tests for unix-char and unix-block devices
  • tests: Enable quota checks for ceph engine
  • tests: Fix broken lxd import integration test
  • tests: Fix CEPH RBD leakage
  • tests: Fix typo in comment
  • tests: Remove MAC tests from infiniband tests
  • tests: Remove the attached testvolume
  • tests: Remove tmpfs references from gpu tests
  • tests: Remove use of uppercase chars in MAC tests
  • tests: Split server tests
  • tests: Update backup test for new error
  • tests: Update integration tests for DB changes

Try it for yourself

This new LXD release is already available for you to try on our demo service.

Downloads

The release tarballs can be found on our download page.

Binary builds are also available for:

  • Linux: snap install lxd
  • MacOS: brew install lxc
  • Windows: choco install lxc
7 Likes

Available in the candidate channel.

Given the database and devices refactoring/rework, it’s reasonably likely that there will be some upgrade issues hitting a small subset of our users. I would strongly recommend you update test/staging systems over to 3.17 as soon as possible and report any upgrade issues so we can fix those issues ahead of a more general roll out early next week.

1 Like

Amazing! :clap:

LXD 3.17 has been promoted to the stable channel.

No issues were reported by our candidate users over the past 3 days so hopefully things are working well.
We will be keeping a close eye on the forum and github for any upgrade issue so we can provide a short turnaround on any fix that may be needed as more people upgrade.

It would be very much appreciated if updates–especially those for the stable channel–would be accompanied by a short post linked to the original release notes:

--- /home/sys-maint/tmp/desktop01.last    2019-09-19 04:30:01.783928437 +0200
+++ /home/sys-maint/tmp/desktop01.current    2019-09-20 04:30:02.025091652 +0200
@@ -1,17 +1,17 @@
 Name                Version                     Rev    Tracking  Publisher   Notes
[...]
-lxd                 3.17                        11964  stable    canonical*  -
+lxd                 3.17                        11985  stable    canonical*  -

Thanks in advance!

Not particularly practical to do as most of that stuff is auto-generated and released and the store API to provide release notes is rather sub-optimal.

Our weekly status updates do include the list of changes we’ve made to the snap.
Alternatively, you could look at https://github.com/lxc/lxd-pkg-snap/commits/latest-candidate for a list of changes to the candidate build ahead of that hitting stable.