LXD 3.20 has been released

News
,
1

Introduction

The LXD team is very excited to announce the release of LXD 3.20!

We purposefully accelerated our normal monthly release cycle to just two weeks as a way to quickly integrate the many enhancements and bugfixes that followed the release of LXD 3.19. We intend to keep up this pace until the release of LXD 4.0 at which point we’ll resume the monthly cadence.

This release includes 3 features/improvements contributed by students at the University of Texas in Austin:

  • Server side support of API collections
  • New unix-hotplug device type
  • Reworked background process management

On top of integrating those, we’ve also been focusing on filling in some of the current gaps in our VM story, fixing many bugs since 3.19 came out and now completing its network interface handling, adding support for ppc64le and support for boot device ordering.

Enjoy!

Highlights

Server side support of API collections

As our users are dealing with increasingly larger sets of instances and images, filtering all those records on the client side can become quite expensive. With this release, we’re now adding the infrastructure and initial implementation of server side filtering.

This looks like:

stgraber@castiana:~/data/code/lxc/lxd (lxc/master)$ lxc query '/1.0/instances?filter=config.image.os%20eq%20ubuntu'
[
    "/1.0/instances/snapcraft",
    "/1.0/instances/ups-monitor",
    "/1.0/instances/v1",
    "/1.0/instances/maas01",
    "/1.0/instances/steam",
    "/1.0/instances/lxd-build"
]

This is using config.image.os eq ubuntu as a filter using URL encoding. More details on the filtering options can be found here.

New unix-hotplug device type

This device type is a bit of a cross between usb and unix-char and unix-block.

It allows specifying a specific vendorid/productid and have any resulting unix-char/unix-block devices be automatically passed to the container.

An example with a USB drive:

stgraber@castiana:~$ lxc config device add c1 kingston unix-hotplug vendorid=0951 productid=1666
Device kingston added to c1

stgraber@castiana:~$ lxc exec c1 bash
root@c1:~# ls -lh /dev/
total 0
crw--w---- 1 root   tty     136,   0 Jan 30 23:00 console
lrwxrwxrwx 1 root   root          11 Jan 30 22:59 core -> /proc/kcore
lrwxrwxrwx 1 root   root          13 Jan 30 22:59 fd -> /proc/self/fd
crw-rw-rw- 1 nobody nogroup   1,   7 Jan 13 03:59 full
crw-rw-rw- 1 nobody nogroup  10, 229 Jan 30 22:59 fuse
lrwxrwxrwx 1 root   root          25 Jan 30 22:59 initctl -> /run/systemd/initctl/fifo
lrwxrwxrwx 1 root   root          28 Jan 30 22:59 log -> /run/systemd/journal/dev-log
drwxr-xr-x 2 nobody nogroup       60 Jan 30 22:46 lxd
drwxrwxrwt 2 nobody nogroup       40 Jan 13 03:59 mqueue
drwxr-xr-x 2 root   root          60 Jan 30 22:59 net
crw-rw-rw- 1 nobody nogroup   1,   3 Jan 13 03:59 null
crw-rw-rw- 1 root   root      5,   2 Jan 30 22:59 ptmx
drwxr-xr-x 2 root   root           0 Jan 30 22:59 pts
crw-rw-rw- 1 nobody nogroup   1,   8 Jan 13 03:59 random
drwxrwxrwt 2 root   root          40 Jan 30 22:59 shm
lrwxrwxrwx 1 root   root          15 Jan 30 22:59 stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root   root          15 Jan 30 22:59 stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root   root          15 Jan 30 22:59 stdout -> /proc/self/fd/1
crw-rw-rw- 1 nobody nogroup   5,   0 Jan 30 21:23 tty
crw-rw-rw- 1 nobody nogroup   1,   9 Jan 13 03:59 urandom
crw-rw-rw- 1 nobody nogroup   1,   5 Jan 13 03:59 zero

root@c1:~# ls -lh /dev/
total 1.0K
drwxr-xr-x 3 root   root          60 Jan 30 23:01 bus
crw--w---- 1 root   tty     136,   0 Jan 30 23:00 console
lrwxrwxrwx 1 root   root          11 Jan 30 22:59 core -> /proc/kcore
lrwxrwxrwx 1 root   root          13 Jan 30 22:59 fd -> /proc/self/fd
crw-rw-rw- 1 nobody nogroup   1,   7 Jan 13 03:59 full
crw-rw-rw- 1 nobody nogroup  10, 229 Jan 30 22:59 fuse
lrwxrwxrwx 1 root   root          25 Jan 30 22:59 initctl -> /run/systemd/initctl/fifo
lrwxrwxrwx 1 root   root          28 Jan 30 22:59 log -> /run/systemd/journal/dev-log
drwxr-xr-x 2 nobody nogroup       60 Jan 30 22:46 lxd
drwxrwxrwt 2 nobody nogroup       40 Jan 13 03:59 mqueue
drwxr-xr-x 2 root   root          60 Jan 30 22:59 net
crw-rw-rw- 1 nobody nogroup   1,   3 Jan 13 03:59 null
crw-rw-rw- 1 root   root      5,   2 Jan 30 22:59 ptmx
drwxr-xr-x 2 root   root           0 Jan 30 22:59 pts
crw-rw-rw- 1 nobody nogroup   1,   8 Jan 13 03:59 random
brw-rw---- 1 root   root      8,   0 Jan 30 23:01 sda
brw-rw---- 1 root   root      8,   1 Jan 30 23:01 sda1
drwxrwxrwt 2 root   root          40 Jan 30 22:59 shm
lrwxrwxrwx 1 root   root          15 Jan 30 22:59 stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root   root          15 Jan 30 22:59 stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root   root          15 Jan 30 22:59 stdout -> /proc/self/fd/1
crw-rw-rw- 1 nobody nogroup   5,   0 Jan 30 21:23 tty
crw-rw-rw- 1 nobody nogroup   1,   9 Jan 13 03:59 urandom
crw-rw-rw- 1 nobody nogroup   1,   5 Jan 13 03:59 zero
root@c1:~# exit

Above you’ll notice sda and sda1 showing up when I plug my USB key.

And with a yubikey:

stgraber@castiana:~$ lxc config device add c1 yubikey unix-hotplug vendorid=1050
Device yubikey added to c1

stgraber@castiana:~$ lxc exec c1 bash
root@c1:~# ykman info
Device type: YubiKey 5C
Serial number: 11576019
Firmware version: 5.2.4
Form factor: Keychain (USB-C)
Enabled USB interfaces: FIDO+CCID

Applications
OTP     	Disabled	
FIDO U2F	Enabled 	
OpenPGP 	Enabled 	
PIV     	Enabled 	
OATH    	Disabled	
FIDO2   	Enabled 	
root@c1:~#

Support for standby cluster members

The dqlite-based cluster database logic was extended to make use of newly supported standby and idle roles in dqlite.

With this change in place, all cluster members will now be integrated with dqlite, they will then each hold one of the following three roles:

  • voting member
  • standby member
  • idle member

The configuration is for 3 voting members, followed by 2 standby members with the rest of the members being idle. In the event of a loss of a voting member, after a short timeout a non-voting member will get auto-promoted to voting. Idle members will be promoted to non-voting as needed to keep the set number of voting and standby members.

The end result is a much more resilient LXD cluster which will better handle cluster members going offline for maintenance or power failure.

VM: Extended NIC support

With this release, nic devices for virtual machines now support:

  • macvlan (using macvtap)
  • sriov
  • physical
  • p2p

Both SR-IOV and physical NIC passthrough rely on PCI passthrough to the virtual machines. This requires a properly configured IOMMU setup as well as adequate PCI setup.

VM: boot priority

A new boot.priority option is now present on all disk and nic type devices for use with virtual machines. The higher the priority, the more likely the VM is to boot from the device.

This is of particular use when one wants a VM to always boot from the network rather than use the default behavior of booting from its root disk.

VM: ppc64le host support

Initial support for ppc64le hosts has been added. This behaves in much the same way as x86_64 and aarch64 except for the fact that a regular PCI bus is used rather than PCIe and the firmware is SLOF rather than UEFI.

Complete changelog

Here is a complete list of all changes in this release:

Full commit list
  • Fix typo
  • Add role column to raft_nodes table
  • Make db.RaftNode an alias for dqlite.NodeInfo, which has a Role field
  • lxd/cluster: rename raft.go to info.go
  • Drop legacy newRaft() function, leftover from the hashicorp/raft time
  • Replace raftAddressProvider intefrace with a simple raftAddress() method
  • Gateway.currentRaftNodes(): return only voting nodes
  • Use the ID from the cluster nodes table as raft ID
  • Make RaftNodesReplace() skip non-voters
  • Include role when exchanging nodes in join/rebalance internal APIs
  • Always join the dqlite cluster, possibly as non-voter
  • Attempt to probe to a member which is considered offline
  • Connect to the target node before spanwing the image replication goroutine
  • Only contact voter nodes when searching for the cluster leader
  • Configure the dqlite client store to only connect to voter nodes
  • Load the role column of raft_nodes in db.RaftNodes()
  • Store the role of a node in RaftNodesReplace
  • Only start the dqlite engine for voter nodes
  • Include role information in heartbeat messages
  • Return also non-voter nodes in currentRaftNodes
  • Fix ineffective heartbeat upon join
  • Drop unused target variable in Join/Promote/Leave cluster functions
  • Start the dqlite engine also on non-voter nodes
  • Add cluster.Handover which finds a node eligible to become voter
  • Extract logic to POST a promote request into a new changeMemberRole() function
  • Attempt to transfer leadership to another member when shutting down
  • Add /internal/cluster/handover endpoint to transfer voter role
  • Add handoverMemberRole() helper to transfer the responsibilities of a member
  • Update gateway’s identify info upon role change
  • Rename cluster.Promote to cluster.Assign
  • Rename /internal/cluster/promote to /internal/cluster/assign
  • Remove database role when demoting
  • When a voter is shutdown, handover the role to another member
  • Assign up to 2 StandBy roles beyond the initial 3 voters
  • Return only voter nodes in cluster.ListDatabaseNodes()
  • When leaving, use currentRaftNodes() instead of querying the raft_nodes table
  • Redirect member delete requests to the leader
  • Serialize membership-related requests on the leader
  • Drop clusterRebalance helper, since it is used only once
  • Drop ineffective post-join heartbeat
  • Add rebalanceMemberRoles() helper and use it in the delete member API handler
  • Drop unused tryClusterRebalance()
  • Test shutting down two members concurrently
  • Close dqlite clients after use
  • Automatically demote offline nodes when running cluster.Rebalance
  • Automatically promote spare nodes if a voter goes offline
  • Assign roles to members not part of the raft configuration
  • Export MaxVoters and MaxStandBys
  • Trigger rebalance also if there are not enough voters or standbys
  • Downgrade rebalance error to warning, it should not block node removal
  • lxd/container: Improves error messages in instanceValidDevices
  • lxd/container: instance.ValidDevices usage
  • lxd/container/lxc: instance.ValidDevices usage
  • lxd/device/config/devices: Improves error messages
  • lxd/device/disk: Adds support for VM disk devices
  • lxd/instance/instance/interface: Comment ending consistency
  • lxd/instance/qemu/vm/qemu: Fixes driver index loop bug
  • lxd/instance/instance/utils: Introduces constant to indicate profile validation in instance name
  • lxd/profiles: Switches to use instance.ProfileValidationName during profile validation
  • lxd/device/disk: Updates Stop device to understand VM disks
  • Make cluster.Rebalance fail immediately if not leader
  • Export cluster.ErrNotLeader
  • Silence warning about failing to rebalance when not leader
  • lxd/storage/drivers/driver/common: Removes generic vfs functions as not common to all driver types
  • Re-enable clustering upgrade test
  • lxd/storage/drivers/generic/vfs: Moves generic VFS drivers into standalone file
  • lxd/storage/drivers: Updates usage of generic VFS functions
  • Add upgrade test for an 8-member cluster
  • Make upgrade notifications more robust
  • Wait for leadership to settle before running lxc cluster list
  • lxd/instance: Moves vm qemu pkg into instance/drivers pkg
  • lxd/instance/drivers/container/lxc: Adds placeholder for future lxc implementation
  • lxd/instance/drivers/load: Adds instance load functions
  • lxd/container: Removes unused functions
  • lxd/db/containers: Renames ContainerToArgs to InstanceToArgs
  • lxd/container: db.InstanceToArgs usage
  • lxd/profiles/utils: db.InstanceToArgs usage
  • lxd/profiles/utils: Updates use of containerLXCInstantiate
  • lxd/container/lxc: Push containerLXC load functions into instance/drivers package
  • lxd/container/lxc: containerLXCInstantiate usage
  • lxd/container/lxc: Makes containerLXCInstantiate compatiable with generic instance load functions
  • lxd/containers: instance.Load usage
  • lxd/containers/post: instance.Load usage
  • lxd/instance/drivers/vm/qemu: Unexport and rename load functions
  • lxd/instance/instance/utils: Load function comments
  • lxd/instance/instance/utils: Adds Create instance function placeholder
  • lxd/instance/instance/utils: db.InstanceToArgs usage
  • lxd/instance/drivers/vm/qemu: Unexports qemu implementation
  • lxd/exec: Pass full req through
  • lxd/exec: Forward control messages
  • lxd/containers: Fix error handling on stop
  • lxd/vm: Fix stop race condition
  • lxd/vm: Add locking for stop and shutdown
  • lxd/vm: Don’t crash on vm-initiated reboots
  • lxd/storage: Remove legacy volume.size check
  • lxc/init: Consider image type for instance type
  • i18n: Update translation templates
  • tests: Update volume.size tests
  • lxd/vm: Store qemu log
  • [Makefiles] Whitelist ldflags in libcap pkgconfig
  • lxd/vm: Fix incorrect bootindex
  • lxd/vm: Implement snapshot restore
  • lxd/instance: Move LoadAllInternal
  • lxd/vm: Implement Snapshots
  • lxd/storage/drivers/utils: Updates ensureVolumeBlockFile to use minimum block boundary size of 8192 bytes
  • lxd/storage/drivers/driver/lvm/utils: Avoid repetition of 512 bytes in roundedSizeBytesString
  • doc: Corrects lvm striping options
  • lxd/instance: Renames driver files for consistency
  • lxd/instance: Comment clarification
  • lxd-agent: Fixes bug when agent not seen as started if LXD restarted
  • lxd/device/device/utils/network: Adds host MTU support for VM tap devices
  • lxd/device/nic/bridged: Makes VM host side TAP interface name prefixed with “tap”
  • lxd/instance/drivers/qemu: Switch to template pkg to generate qemu conf
  • doc/api-extensions: Fix syntax
  • api: vm_boot_priority
  • lxd/vm: Add boot.priority
  • lxd/container/logs: Makes log file retrieval project aware
  • lxd/container/lxc: Adds devName skipping for startCommon
  • lxd/device/config/device/runconfig: Adds DevName to MountEntryItem
  • lxd/device/disk: Adds DevName to MountEntryItem
  • lxd/device: Adds devName property to network interface run config
  • lxd/instance/drivers/driver/qemu: Adds support for Disk and NIC device boot.priority setting
  • Use a light TCP/TLS connection attempt instead of a client request
  • lxd/container/exec: Removes duplication of env map now its being stored back into post data
  • Revert “lxd/exec: Forward control messages”
  • lxd/instance/drivers/driver/qemu/cmd: Makes qemu cmd struct qemu specific
  • lxd/instance/drivers/driver/qemu: Simplifies Exec with revert
  • lxd/container/exec: Cleaned up logging
  • lxd/container/exec: Switches to use instance command for resizing window
  • lxd/container/lxc/exec/cmd: Adds WindowResize
  • lxd/instance/instance/exec/cmd: Adds WindowResize function to signature
  • lxd/instance/drivers/driver/qemu: Reworks command control
  • lxd/instance/drivers/driver/qemu/cmd: Adds WindowResize support
  • lxd/instance/drivers/driver/qemu: Sets PID to 0 for VM commands
  • lxd/instance/drivers/driver/qemu: comment on forwardControlCommand
  • lxd/device/nic/p2p: Adds VM support
  • Fix translation of hard-coded address of first node
  • Close http transports since they might keep connections around
  • lxd/daemon: Ignore SIGHUP
  • lxd/instance/drivers/driver/qemu: Switch to unsafe async I/O mode on ZFS pools backed by loop files
  • lxd/storage: Improves pool init failure messages
  • lxd/storage/drivers: Indicates DirectIO support for most storage drivers
  • lxd/storage/drivers/driver/types: Adds DirectIO indicator to driver info struct
  • shared/version/version: Quotes malformed version string in error message
  • lxd/storage/drivers/driver/zfs: Adds DirectIO detection based on version
  • lxd/instance/drivers/driver/qemu: Unmounts volume on start failure if needed
  • lxd/device: Relaxes requirement for name property when not using containers
  • lxd/device/nic/macvlan: Clean up valid fields
  • lxd/device/nic/macvlan: Adds VM support and improves revert
  • lxd/instance/drivers/driver/qemu: Adds macvtap support
  • lxd/instance/drivers/driver/qemu/templates: Moves templates to separate file
  • lxd/instance/drivers/driver/qemu: Updates template usage
  • lxd/storage/drivers/driver/dir: Adds HostPath support
  • storage: Fix xfs_growfs command for older versions
  • shared/simplestreams: Fix architecture filtering
  • lxd/patches: Reset ZFS mountpoint/canmount
  • shared/simplestreams: Fix inconsistent sorting
  • lxd/instances: Don’t rquire type on copy
  • lxc/config: Tweak argument processing
  • lxd/vm: Prevent attaching directory as disk
  • lxd/storage/zfs: Ignore bookmarks
  • lxd/storage/btrfs: Skip missing quota
  • doc/instance: Clarifies disk path not available for VMs
  • lxd/instance/drivers/qmp/monitor: Prevent crashes with races closing closed channel
  • lxd/instance/drivers/driver/qemu: Improve clean up on start failure
  • Fix request redirect when removing a cluster member
  • lxd/storage/backend/lxd: Only detect volume.block.filesystem changes on block backed pool FS volumes
  • lxd/migration/migration/volumes: Adds support for pre-bidirectional negotiation targets
  • lxd/container/lxc: Removes VM specific NIC config ignoring
  • lxd/device: Only return devName NIC config item for VMs
  • lxd/device/nic/physical: Improves revert and deletion of created VLAN devices
  • lxd/instance/drivers/driver/qemu/templates: Clarifies qemuNetdevPhysical variables
  • lxd/device/nic/macvlan: Differentiates config parent from actual parent
  • lxd/device/device/utils/network: Adds networkGetDevicePCIDevice function
  • lxd/device/nic/sriov: Updates networkGetVFDevicePCISlot to use networkGetDevicePCIDevice
  • lxd/instance/drivers/driver/qemu: Adds physical NIC passthrough support
  • shared/instance: Updates config key checker to allow “.driver” keys
  • doc/instance: Documents which device types can be used with which instance types
  • lxd/device/device/utils/network: Adds generic PCI device bind/unbind functions
  • lxd/device/device/utils/network: Adds networkVFIOPCIRegister
  • lxd/device/nic/sriov: Switches PCI device bind/unbind to generic functions
  • lxd/device/nic/physical: Adds VM PCI passthrough support
  • lxd/device: Unexports NetworkRemoveInterfaceIfNeeded
  • lxd/instance: Add NetworkUpdateStatic
  • Add maasRename to VM
  • lxd/storage/generic: Don’t fail rename on missing path
  • lxd/storage/zfs: Fix block mounts
  • lxd/storage/zfs: Fix renames
  • lxd/vm: Implement Rename
  • lxd/device/nic/sriov: Adds VM support
  • lxd/instance/drivers/driver/qemu: Mount VM config vol before generating NVRAM file
  • lxd/device: Add unix_hotplug device type
  • lxd/device: Add support for listening to unix char and block udev events
  • lxd/storage: Pass config when deleting images
  • lxd/devices: Remove dead xtables code
  • lxd/iptables: Fix matching of IPv6 link-local
  • lxd: Updates usage of migration.MatchTypes
  • lxd/migration/migration/volumes: Updates MatchTypes to return all supported migration types
  • lxd/migration/migration/volumes: Break after first rsync transport features extracted
  • shared/subprocess: Cleanup test script
  • shared/subprocess: Fix Wait, tty and ignore stdin
  • shared/subprocess: Better handle not running
  • lxd/networks: Avoid dnsmasq reload on start
  • lxd: Switch to using the new subprocess module
  • lxd/patches: Convert PID files
  • shared/subprocess: Cleanup tests
  • shared/subprocess: Use channel for Wait
  • lxd/apparmor: Allow ro,remount,noatime,bind
  • lxd/storage/drivers: Pass mountPath to xfs_growfs
  • lxd/container: Removes containerValidName function
  • lxd/container: Switches to instance.ValidName
  • lxd/instance/instance/utils: Adds ValidName function
  • shared/util: Modifies ValidHostname to return specific error
  • shared/instance: InstanceGetParentAndSnapshotName comments
  • lxd/storgage/locking/lock: Fixes concurrent access race to map
  • global: Replace Fatalf by Errorf in tests
  • shared/generate: Fix regression caused by Fatalf fix
  • devices: retrieve vendor and product for hidraw devices
  • lxd/db: Fix for new testify
  • lxd/main: Adds cmdGlobal.rawArgs function
  • lxd: Adds forklimits command
  • lxd/instance/drivers/driver/qemu: Switches to launching qemu via forklimits
  • devices: substract libudev header
  • lxd/db: adds unix-hotplug device type to database
  • lxd/instance/drivers/driver/qemu: Adds qemu binary path lookup
  • lxd/main/forklimits: Switches forklimits to use syscall.Exec
  • shared/cert: Replace default IPs with localhost
  • shared/subprocess: Improve error in test
  • spelling: yaml should be YAML
  • spelling: Busybox should be BusyBox
  • i18n: Update translation templates
  • doc/storage: Update for snap package
  • api: Add extension for new device type unix hotplug
  • doc/instances: added new device type unix hotplug
  • doc: Add libudev-dev dependency
  • lxd/vm: Record architecture name
  • lxd/vm: Cleanup qemu config
  • lxd/vm: Add ppc64el support
  • lxd/device/device/common: Splits device common into own file
  • lxd/device/device: Removes original device.go file
  • lxd/device/device/interface: Splits device interfaces into own file
  • lxd/device/device/load: Separates device load functions into own file
  • lxd/instance/drivers/driver/common: Adds common driver type
  • lxd/instance/instance/interface: Adds ConfigRead interface
  • lxd/instance/drivers/load: Updates validDevices() to use device.Validate function
  • lxd/instance/instance/utils: Removes instanceName from validateDevices function
  • lxd/instance/drivers/driver/qemu: Embeds common type and removes dupe functionality
  • lxd: instance.ValidDevices usage
  • lxd/device/device/utils/instance: Adds instanceSupported function
  • lxd/device: Updates device validateConfig to support instance.ConfigReader argument
  • api: Add api_filtering extension
  • lxd/filter: Add API filtering package
  • lxd/instance: Add instance list filtering functions
  • lxd: Make use of filtering for instances and images
  • doc/rest-api: Document filtering
  • tests: Add tests for API filtering
  • lxd/filter: Workaround gofmt bug
  • lxd/device/disk: Adds a check for mkisofs tool for qemu config drive
  • lxd/device/nic/sriov: Loads vfio-pci module
  • tests: Fix BusyBox spelling for filtering
  • lxd/vm: Fix bad bus name on ppc64el
  • lxd/vm: Don’t specify addresses for pci on ppc64
  • i18n: Update translations from weblate

Try it for yourself

This new LXD release is already available for you to try on our demo service.

Downloads

The release tarballs can be found on our download page.

Binary builds are also available for:

  • Linux: snap install lxd
  • MacOS: brew install lxc
  • Windows: choco install lxc
3 Likes