LXD 3.23 has been released

News
,
1

Introduction

The LXD team is very excited to announce the release of LXD 3.23!

This should be the last release of the 3.x series with LXD 4.0 planned to be released next week with very minimal changes on top of that (a few backward-incompatible CLI tweaks).

This is also a rather feature packed release, especially for those using custom storage volumes, projects or virtual machines.

Enjoy!

Highlights

Custom storage volumes in projects

A new project feature (features.storage-volumes) is now available to all new projects and ties custom storage volumes to the project.

This allows projects to have their own separate set of custom storage volumes without risk of conflicts. When combined with Canonical RBAC, this also now properly isolates storage between different projects.

stgraber@castiana:~$ lxc storage volume list default | grep custom
+--------+----------+-------------+---------+
|  TYPE  |   NAME   | DESCRIPTION | USED BY |
+--------+----------+-------------+---------+
| custom | backups  |             | 1       |
+--------+----------+-------------+---------+
| custom | blah     |             | 0       |
+--------+----------+-------------+---------+
| custom | images   |             | 1       |
+--------+----------+-------------+---------+

stgraber@castiana:~$ lxc project create blah
Project blah created
stgraber@castiana:~$ lxc project switch blah

stgraber@castiana:~$ lxc storage volume create default foo
Storage volume foo created
stgraber@castiana:~$ lxc storage volume list default | grep custom
+--------+------+-------------+---------+
|  TYPE  | NAME | DESCRIPTION | USED BY |
+--------+------+-------------+---------+
| custom | foo  |             | 0       |
+--------+------+-------------+---------+
stgraber@castiana:~$

Schedule snapshots for custom storage volumes

Similar to instances, the snapshots.schedule and snapshots.pattern configuration keys are now available to custom volumes too.

They can be set directly using lxc storage volume set POOL VOL KEY VALUE.

Expiry for custom storage volumes

With automatic snapshots now being possible on custom storage volumes, an expiry mechanism is a good idea. Matching what’s available in instances, this can be configured through snapshots.expiry.

Editing the expiry on existing snapshots can be done with lxc storage volume edit POOL VOL/SNAP.

Limits for projects

Some limits can now be applied on a per-project basis.
The limits available at this time are:

  • limits.containers for the total number of containers allowed
  • limits.virtual-machines for the total number of virtual-machines allowed
  • limits.cpu for the number virtual CPUs that may be used
  • limits.memory for the total amount of memory that can be given
  • limits.processes for the total number of processes that can be used

Note that the last 3 require all instances in the project to have the matching configuration key set on them. The limit applies to the total configured limit on the instances rather than to the live usage.

Restrictions for projects

Additionally, some feature restrictions can now be applied to projects too.

The full list of options can be found at Projects | LXD

This is designed so that marking a project as restricted using restricted=true should default to it being safe for untrusted users. Restrictions can then be relaxed to allow potentially more dangerous or less confined configuration and devices.

Combined with Canonical RBAC, this can be used to run a shared LXD server or cluster with mostly untrusted users having the ability to spawn containers and virtual-machines without effectively having to trust them with full privileges on the hosts.

Improved backup/export logic

The backup/export logic as used by lxc export has been updated to reduce the amount of disk space needed during an export. The container files are now directly written to the compressed tarball, without any intermediate copy being made on disk.

This should significantly reduce the amount of disk space used by an export as well as speed up the process quite a bit.

VM: Support for migration

Virtual machines can now be copied and moved between local storage pools as well as to remote LXD servers.

Note that this is only “cold” migration, that is, virtual machines must be stopped prior to being moved. Live migration is planned for a later stage.

VM: Support for publishing

It is now possible to lxc publish a virtual machine, resulting in a functional image which can be used to spawn more virtual machines or transfer it to another server.

While this all works like it does for containers, we do have to highlight the fact that virtual machine disks are much larger than containers and as our images require the disk be repacked to qcow2 during the publishing process, you will need a significant amount of free disk space on the system in order to handle large virtual machines.

Complete changelog

Here is a complete list of all changes in this release:

Full commit list
  • lxd/storage/zfs: Fix usage calculation
  • Add go 1.14.x check
  • lxd: Cleanup error messages
  • lxd: Rename container files to instance
  • tests: Update for rename
  • production-setup: add net.core.bpf_jit_limit and kernel.keys.maxbytes
  • doc/instances: Adds missing host_name key on routed nic device
  • doc/instances: Documents ipv4.gateway and ipv6.gateway routed NIC keys
  • lxd/device/device/utils/network: Adds NetworkValidGateway helper
  • lxd/device/nic: Adds ipv4.gateway and ipv6.gateway validation
  • lxd/device/nic/routed: Adds support for not adding automatic default gateway
  • api: Adds extension container_nic_routed_gateway
  • lxd/util/fs: Fixes go vet conversion from int64 to string yields a string of one rune error
  • lxd/device/disk: Only unmounts non-root volumes attached
  • lxd/daemon: Adds comment to AllowAuthenticated
  • lxc/storage/volumes: Adds API permission check for permission “manage-storage-volumes”
  • lxd/project/project: Comment tweak to Instance()
  • lxd/project/project: Adds StorageVolume()
  • lxd/project/project/test: Adds StorageVolume() test
  • lxd/project/project: Adds StorageVolumeParts function
  • lxd/project/project: Adds StorageVolumeProject function
  • lxc/project: Adds STORAGE VOLUMES col to projects list
  • doc/projects: Documents features.storage.volumes flag
  • lxd/api/project: Adds features.storage.volumes to API
  • lxd/db/migration: Adds features.storage.volumes true to default project on importPreClusteringData
  • lxd/db/cluster/open: Adds features.storage.volumes true to default project in EnsureSchema
  • scripts/bash/lxd-client: Adds features.storage.volumes to bash autocomplete
  • lxd/daemon/storage: Error message quoting
  • lxd/daemon/storage: Updates storage custom volume functions to pass project.Default
  • lxd/daemon/storage: Adds support for custom volume projects
  • lxd/device/disk: Updates custom volume disks to support projects
  • lxd/patches: Updates patchStorageApiPermissions to use project.Default for custom volumes
  • lxd/storage/backend/mock: Updates custom volume signatures to support projectName
  • lxd/storage/pool/interface: Updates custom volume functions to support projectName
  • lxd/storage/volumes: Error message quoting and comment tweaks
  • lxd/storage/volumes: Improve volume type checks
  • lxd/storage/volumes: Add custom volume project support
  • lxd/storage/volumes: Migration project aware
  • lxd/storage/volumes/snapshots: Improve volume type validation
  • lxd/storage/volumes/snapshot: Error message quoting
  • lxd/storage/volumes/snapshot: Adds project support for custom volumes
  • lxd/storage/backend/lxd: Updates custom volume functions to support projects
  • lxd/db/storage/pools: Removes incorrect assumption about custom vol projects in storagePoolVolumeGetType
  • lxd/db/storage/pools: Comment and error msg tweaks
  • lxd/db/storage/pools: Removes incorrect filter for project default when vol type is StoragePoolVolumeTypeCustom
  • lxd/db/storage/pools: Updates StoragePoolVolumeSnapshotsGetType to filter by project
  • lxd/db/storage: Removes StoragePoolNodeVolumeGetType
  • lxd/db/storage: Fixes StoragePoolVolumeSnapshotsGetType to be project aware
  • lxd/patches: Switches to using storageDrivers.GetVolumeMountPath
  • lxd/storage/storage: Removes unused GetStoragePoolVolumeMountPoint
  • lxd/api: Updates projectParam to use project.Default
  • lxd: project.Default usage
  • lxd/images: Comment weaks
  • lxd/images: golint fixes
  • lxd/project/limits: Default const usage
  • lxd/storage/load: Adds support for custom vol projects to volIDFuncMake
  • lxd/storage/load: Error msg quoting tweaks
  • lxd/storage/volumes/utils: Error msg tweaks
  • lxd/storage/volumes/utils: Removes unused supportedVolumeTypesExceptImages
  • lxd/storage/volumes/utils: Updates storagePoolVolumeUpdateUsers to be project aware
  • lxd/storage/volumes/utils: Removes storagePoolVolumeUsedByRunningInstancesWithProfilesGet and old link var
  • lxd/container: Removes unused function instanceLoadAll
  • lxd/storage/backend/lxd: Updates use of database functions to use projectName
  • lxd/storage/utils: Adds VolumeUsedByRunningInstancesWithProfilesGet and removes old link var
  • lxd/storage/utils: Makes VolumeSnapshotsGet project aware
  • lxd/storage/utils: Makes VolumeUsedByInstancesGet project aware
  • lxd/migrate/storage/volumes: Makes custom volume project aware
  • lxd/storage/backend/lxd: b.state.Cluster.StoragePoolNodeVolumeGetTypeByProject usage
  • lxd/storage/backend/lxd: Updates migration functions to be project aware
  • lxd/storage/backend/mock: Updates migration functions to be storage aware
  • lxd/storage/pool/interface: Updates migration functions to be project aware
  • test: Updates tests for custom storage volume projects
  • lxd/db/storage/pools: Makes StoragePoolNodeVolumesGetType project aware
  • lxd/db/storage/pools: Removes StoragePoolNodeVolumeGetTypeID function
  • lxd/patches: StoragePoolNodeVolumeGetTypeIDByProject usage
  • lxd/patches: Improves error messages context
  • lxd/storage/backend/lxd/patches: Adds custom volume rename patch to add project prefix
  • lxd/storage/drivers/utils: Captures error context from e2fsck
  • lxd/storage/drivers/utils: Dont use TryCommand when resizing
  • i18n: Update translation templates
  • lxd: Replaces == “true” with shared.IsTrue() for projects and profiles
  • lxc: Replaces == “true” with shared.IsTrue() for project features
  • lxd/firewall: Don’t create zombies
  • lxd/patches: Adds concept of stage to patch system
  • lxd/daemon: Applies pre daemon storage patches
  • lxd/storage/backend/lxd/patches: Skip already renamed volumes
  • lxd/db/images: Removes unnecessary whitespace
  • lxd/db/images: Updates ImagesGetExpired to return ExpireImage struct with projectName
  • lxd/images: Updates pruneExpiredImages to support removing expired images from non-default projects
  • driver_qemu: delete vm id from vmConsole
  • ExecReaderToChannel: Prevent endless loops
  • lxd/daemon/storage: Removes daemonStorageUsed function
  • lxd/storage/utils: Adds VolumeUsedByDaemon function
  • lxd: storagePools.VolumeUsedByDaemon usage
  • lxd/storage/backend/lxd/patches: Adds daemon storage symlink update to lxdPatchStorageRenameCustomVolumeAddProject
  • lxd/firewall/nft: Flush chain on delete
  • lxd/firewall/nft: Handle json errors
  • lxd/firewall/nft: Refuse to run on old kernels
  • lxd/project: Rename limits.go to permissions.go
  • shared/util/linux: Updates ExecReaderToChannel to accept a finisher chan as struct{}
  • lxd-agent/exec: Updates usage of ExecReaderToChannel channel definitions
  • shared/network: Removes logging internal state of websocket in WebsocketRecvStream
  • shared/netutils/network/linux: Updates WebsocketExecMirror to use struct{} exited indicator channel
  • lxd/instance/exec: Fixes VM read loop when agent not started
  • lxd/project: Rename CheckLimitsUponInstanceCreation to AllowInstanceCreation
  • lxd/project: Rename CheckLimitsUponInstanceUpdate to AllowInstanceUpdate
  • lxd/project: Rename CheckLimitsUponProfileUpdate to AllowProfileUpdate
  • lxd/project: Rename ValidateLimitsUponProjectUpdate to AllowProjectUpdate
  • lxd/project: Rename checkAggregateInstanceLimits to checkRestrictionsAndAggregateLimits
  • lxd/project: Extract checkAggregateLimits from checkRestrictionsAndAggregateLimits
  • lxd/project: Honor the “restricted.containers.nesting” config
  • lxd/project: Prevent using low-level container options
  • lxd/project: Check if restrictions are consistent when updating a project config
  • lxd/project: Honor the “restricted.containers.lowlevel” config
  • lxd/project: Honor the “restricted.containers.privilege” config
  • lxd/project: Also expand instance devices
  • lxd/project: Add machinery to perform checks on instance devices
  • lxc/project: Honor the “restricted.devices.unix-char” config
  • lxd/project: Perform restrictions checks also on profiles config and devices
  • lxc/project: Honor the “restricted.devices.unix-block” config
  • lxc/project: Honor the “restricted.devices.unix-hotplug” config
  • lxc/project: Honor the “restricted.devices.infiniband” config
  • lxc/project: Honor the “restricted.devices.nic” config
  • lxd/project: Honor the “restricted.devices.disk” config
  • lxc/project: Honor the “restricted.devices.gpu” config
  • lxc/project: Honor the “restricted.devices.usb” config
  • lxc/project: Honor the “restricted.virtual-machines.lowlevel” config
  • lxd/project: Pass current configuration to AllowInstanceUpdate
  • lxd/project: Check restrictions for volatile config keys
  • lxd/project: Adjust import order
  • lxd/project: Drive-by lint fixes
  • api: Add new restrict.* config keys to projects
  • shared/version: Add “projects_restrictions” API extension
  • doc/projects.md: Document project restrictions
  • test: Add projects restrictions tests
  • scripts: Update bash completion profile with new project config keys
  • lxd/images: Allow virtual-machine and instance as source
  • lxd/images: Set right image type on publish
  • lxd/vm: Implement Export
  • lxd/instance: Fix expiry check
  • lxd/storage: Unpack unified VM images
  • lxd/migration: Rebuilds migrate.pb.go
  • lxd/migration: Adds BLOCK_AND_RSYNC migration transport type
  • lxd/instances/post: Adds VM support to createFromMigration
  • lxd/migrate/instance: Adds VM support to migrationSourceWs.Do
  • lxd/rsync: Adds support for passing arguments to rsync send command
  • lxd/storage/drivers/utils: Error quoting
  • lxd/storage/drivers/driver/common: Updates MigrationTypes to support block volumes for VMs
  • lxd/storage/drivers/driver/dir/volumes: Updates migration to support VMs
  • lxd/storage/drivers/driver/dir/utils: Skips initial quota for VM block migration
  • lxd/storage/drivers: Switches to ErrNotSupported for non-block volume paths
  • lxd/storage/drivers/generic/vfs: Adds VM migration support to genericVFSMigrateVolume
  • lxd/storage/drivers/generic: Adds VM migration support to genericCreateVolumeFromMigration
  • lxd/storage/drivers: Removes dupe checks using genericVFSMigrateVolume
  • lxd/storage/drivers/generic: Adds volume type specific migration transport type checks
  • lxd/storage/drivers/driver/lvm/volumes: Removes dupe check done in genericCreateVolumeFromMigration
  • lxd/migrate/storage/volumes: whitespace
  • lxd/storage/drivers/driver/btrfs: Adds block migration negotiation
  • lxd/storage/drivers/driver/btrfs/volumes: Adds VM migration support
  • lxd/storage/backend/lxd: Improve delete error messages
  • lxd/storage/utils: Adds FallbackMigrationType function
  • lxd: Replaces hardcoded instances of migration.MigrationFSType_RSYNC
  • lxd/storage/drivers/driver/zfs: Adds block migration negotiation
  • lxd/storage/drivers/driver/zfs/volumes: Adds VM migration support
  • lxd/storage/drivers/driver/ceph: Adds block migration negotiation support
  • lxd/storage/drivers/driver/ceph/volumes: Adds VM migration support
  • lxd/migrate/instance: Prevent live migrations for VMs
  • lxd: Add “instance” string where necessary
  • lxd/instances/snapshot: Fix expiration in profiles
  • lxd/images: Fix source type handling
  • lxc/export: Make API call more correct
  • lxd/storage/drivers/driver/btrfs/volumes: Dont activate quotas if not used
  • lxd/storage/drivers/driver/ceph/volumes: Adds VM block resize support
  • doc/security: Adds network security section
  • lxd: Unexport NewMigrationSource
  • lxd/storage: Fix crash on VM unpack
  • lxd: Unexport NewDaemon
  • lxd: Unexport RestServer
  • lxd: Unexport DefaultDaemonConfig and DefaultDaemon
  • lxd: Unexports AllowAuthenticated and AllowProjectPermission
  • lxd: Unexports DevLxdServer
  • lxd: Unexports daemon feature functions
  • lxd: Unexports migration setup functions
  • lxd: Unexports forwarded response helpers
  • client: Removes nullReadWriteCloser
  • client: Removes unused proxyInstanceMigration function
  • lxc-to-lxd: Removes unused vars
  • lxc-to-lxd: Removes unused connectTarget
  • lxc-to-lxd: Removes unused setupSource
  • lxd/cluster: Removes unused flagForce
  • lxc: Removes unused profile
  • lxc/console: Removes unused getStdout
  • lxd-agent: Removes unused rootUID and rootGID
  • lxc: Removes unused func showByDefault
  • lxd/cgroup: Removes unused cgCgroup2SuperMagic
  • lxd-agent: Unexports NewDaemon
  • memory_utils: align lxc + lxd
  • tree-wide: consistently initialize raw fds to -EBADF instead of -1 in cgo
  • lxd/storage/ceph: Fix ext4 shrinking
  • lxc/remote: Use helpers
  • lxc/remote: Validate remote name
  • i18n: Update translation templates
  • doc: Update requirements
  • lxd/init: Don’t offer dir as a remote backend
  • lxc/config: Fix behaviour of instance snapshot expiry
  • db/cluster: Bump the value of sqlite_sequence for storage_volumes
  • po: Update translations
  • shared/version/api: Add custom_volume_snapshot_expiry extension
  • doc: Add custom_volume_snapshot_expiry
  • lxd/db: Add expiry_date to storage_volumes_snapshots
  • shared/api: Add expiry fields to StorageVolumeSnapshot*
  • lxd/storage: Add expiry to volume snapshot pool functions
  • lxd: Add snapshots.expiry config key for storage volumes
  • lxd/db: Add custom volume snapshot functions
  • lxd: Handle volume snapshot expiry
  • lxd/storage: Add expiry date to VolumeDBCreate
  • lxd/storage: Update expiry date when updating volume snapshots
  • lxd/db: Add ProjectName to StorageVolumeArgs
  • lxd/db: Add new OperationCustomVolumeSnapshotsExpire
  • lxd/db: Add StorageVolumeSnapshotsGetExpired
  • lxd: Remove expired custom volume snapshots
  • *: Remove snapshot code from StoragePoolVolumeCreate
  • lxc: Add --no-expiry for volume snapshots
  • test: Add volume snapshot expiry test
  • doc: Add keys to volume config
  • po: Update translations
  • lxd/storage/drivers/driver/lvm/volumes: Fixes LVM VM snapshot list
  • lxd/cluster: Ignore CEPH custom volumes on removal
  • shared/version: Add volume_snapshot_scheduling API extension
  • lxd/storage: Add snapshots.* config keys
  • lxd/db: Extend StorageVolumeArgs
  • lxd: Support patterns in StorageVolumeNextSnapshot
  • lxd/db: Add StoragePoolVolumesGetAllByType
  • lxd: Add volume snapshot scheduling
  • doc: Add volume snapshot scheduling
  • lxd: Clean up logging for expired volume snapshots
  • doc/networks: describe how to notify systemd-resolved of lxd nameserver
  • lxd/storage/utils: Add missing comments
  • lxd/storage/utils: Add forceRemoveAll
  • lxd/storage/dir: Use forceRemoveAll
  • lxd/api/cluster/test: Removes unused DISABLED_TestCluster_Failover
  • lxd/api/cluster/test: Removes unused FLAKY_TestCluster_LeaveAndPromote
  • lxd/cluster/gateway: Removes unused cachedRaftNodes
  • lxd/cluster/heartbeat/test: Removes unused DISABLE_TestHeartbeat_MarkAsDown
  • lxd/cluster/membership/test: Removes unused FLAKY_TestPromote
  • lxd/db/containers: Removes unused snapshotIDsAndNames
  • lxd/db/db/internal/test: Removes unused dir var
  • lxd/db/testing: Removes unused var
  • lxd/device/device/utils/unix: Removes unused unixDeviceInstanceAttributes
  • lxd/device/nic/bridged: Removes unused dhcpAllocation
  • lxd/firewall: Removes unused constants
  • lxd/instance/drivers/driver/lxc: Removes unused cgroup2 var
  • lxd/main/forkproxy: Removes unused udpConn var
  • lxd/storage/drivers/driver/common: Removes unused load
  • shared/generate/file/buffer: Removes unused varDeclSliceToString
  • shared/generate/db/parse: Removes unused simpleTypeNames
  • shared/generated/file/path: Removes unused absPath
  • lxd/db/node: Tweaks LEFT JOIN to just JOIN in NodeIsEmpty()
  • lxd/sys: Don’t fail chmod on unresolvable symlinks
  • shared/containerwriter: Renames to instancewriter
  • lxd/instance/drivers: instancetarwriter usage
  • shared/instancewriter/instance/tar/writer: Modifies WriteFile to accept a file name arg
  • shared/instancewriter/instance/tar/writer: Adds ResetHardLinkMap function
  • lxd/instance/drivers: instancetarwriter.WriteFile name arg usage
  • lxd/db/containers: Renames ContainerBackupCreate and ContainerBackupRemove
  • i18n: Update translations from weblate
  • lxd/backup: Removes backupCreateTarball function
  • lxd/backup: Updates instance backup to use tar writer rather than tar cmd
  • lxd/backup: InstanceBackupRemove usage
  • lxd/storage/drivers/utils: Minor tweak to copyDevice error message
  • lxd/stroage/drivers/generic: Tweak error message of genericCreateVolumeFromMigration
  • lxd/storage/drivers/generic/vfs: Switches genericVFSBackupVolume to tar writer
  • lxd/images: Fixes unhandled error
  • lxd/storage/backend/lxd: Adds tarWriter to BackupInstance function
  • lxd/storage/backend/mock: Adds tarWriter to BackupInstance function
  • lxd/storage/drivers/driver/ceph/volumes: Adds tarWriter arg to BackupVolume
  • lxd/storage/drivers/driver/cephfs/volumes: Adds tarWriter arg to BackupVolume
  • lxd/storage/drivers/driver/dir/volumes: Adds tarWriter arg to BackupVolume
  • lxd/storage/drivers/driver/lvm/volumes: Adds tarWriter arg to BackupVolume
  • lxd/storage/drivers/drivers/mock: Adds tarWriter arg to BackupVolume
  • lxd/storage/drivers/interface: Adds tarWriter arg to BackupVolume
  • lxd/storage/pool/interace: Adds tarWriter arg to BackupInstance
  • lxd/storage/drivers/driver/btrfs/volumes: Adds tarWriter arg to BackupVolume
  • lxd/storage/drivers/driver/zfs/volumes: Adds tarWriter arg to BackupVolume
  • lxd/internal: Log some memory stats
  • shared: Drop Pipe function
  • lxd/containers: Add configfs and tracefs
  • btrfs quota to simulate total disk size

Try it for yourself

This new LXD release is already available for you to try on our demo service.

Downloads

The release tarballs can be found on our download page.

Binary builds are also available for:

  • Linux: snap install lxd
  • MacOS: brew install lxc
  • Windows: choco install lxc
4 Likes
3

YEAH!

3 Likes