Introduction
The LXD team is very excited to announce the release of LXD 5.2!
This isn’t a very busy release feature wise as we’ve been pretty focused on sorting out a variety of migration and storage issues but there are still quite a few welcome changes and improvements in here.
Enjoy!
New features and highlights
VPD information in resources API
To simplify the identification and tracking of complex PCI devices like SmartNICs, the LXD resources API now exposes PCI VPD (Vital Product Data) when available.
Here is an example entry for a built-in server network card:
{
"driver": "ixgbe",
"driver_version": "5.17.6+",
"iommu_group": 54,
"numa_node": 0,
"pci_address": "0000:01:00.1",
"product": "Ethernet Controller 10-Gigabit X540-AT2",
"product_id": "1528",
"vendor": "Intel Corporation",
"vendor_id": "8086",
"vpd": {
"entries": {
"PN": "AOC-UR-i4XT",
"SN": "OA14BS031362",
"V0": "0100",
"V1": "1.01",
"V2": "002590FB3BBD",
"V3": "002590FB3BBC",
"V4": "",
"V5": "",
"VA": "2"
},
"product_name": "Supermicro Network Adapter"
}
}
Cross-project profile copy
The command line tool now allows for copying of a profile from one project to another by using the --target-project parameter.
stgraber@dakara:~$ lxc profile list --project default
+---------+---------------------+---------+
| NAME | DESCRIPTION | USED BY |
+---------+---------------------+---------+
| bar | | 0 |
+---------+---------------------+---------+
| default | Default LXD profile | 13 |
+---------+---------------------+---------+
stgraber@dakara:~$ lxc profile list --project demo
+---------+--------------------------------------+---------+
| NAME | DESCRIPTION | USED BY |
+---------+--------------------------------------+---------+
| default | Default LXD profile for project demo | 0 |
+---------+--------------------------------------+---------+
stgraber@dakara:~$ lxc profile copy bar bar --project default --target-project demo
stgraber@dakara:~$ lxc profile list --project demo
+---------+--------------------------------------+---------+
| NAME | DESCRIPTION | USED BY |
+---------+--------------------------------------+---------+
| bar | | 0 |
+---------+--------------------------------------+---------+
| default | Default LXD profile for project demo | 0 |
+---------+--------------------------------------+---------+
stgraber@dakara:~$
HTTP streaming support in /dev/lxd API
The REST API exposed over /dev/lxd/sock inside of LXD instances has long had a /1.0/events endpoint to listen for events coming from LXD.
Until now, this endpoint required the use of websocket which turned out to be problematic for cloud-init and other tooling that don’t have robust websocket support.
To accomodate those tools, the endpoint now also supports HTTP streaming allowing for getting a stream of JSON structs through the HTTP endpoint.
root@u1:~# curl --unix-socket /dev/lxd/sock -N lxd/1.0/events
{"type":"config","timestamp":"2022-05-26T23:33:03.481505039-04:00","metadata":{"key":"user.foo","old_value":"","value":"bar"}}
Use of server-side filtering in CLI
Not something that’s directly visible other than perhaps by seeing a speed increase when dealing with large lxc list calls but a lot of the filtering is now using server-side filtering rather than client side.
Ceph librbd for virtual machines
Another one that’s not directly visible but can lead to a fair bit more stability and flexibility for virtual machines stored on Ceph. In previous releases LXD would rely on the kernel RBD driver to map the VM disk and expose it to QEMU.
Starting with this release, LXD now instructs QEMU to use librbd to directly interact with Ceph. Other than limiting the use of the kernel driver, this also allows for tweaking of the Ceph RBD cache through the system’s ceph.conf.
Complete changelog
Here is a complete list of all changes in this release:
Full commit list
- lxd/storage/drivers/btrfs: Modify received UUID
- lxd/storage/drivers/btrfs: Set Received UUID
- test: Extend migration refresh test
- lxd/secommp: Fix sysinfo syscall interception on 32 bit platforms
- client/util: Add timeouts to tlsHTTPClient
- client/util: Add timeouts to unixHTTPClient
- lxd/cluster/tls: Add timeouts to tlsTransport
- lxd/main/shutdown: Increase default timeout in cmdShutdown
- lxd/util/http: Adds timeouts in HTTPClient
- lxd/vsock/vsock: Adds timeouts in HTTPClient
- shared/cert: Adds timeouts in GetRemoteCertificate
- lxd/certificates: Simplify non-admin check
- lxd/certificates: Comment the logic a bit
- lxd/certificates: Move duplicate cert detection
- doc: Fix typos
- server.crt referred to as lxd.crt in the prometheus config file
- doc: headings and whitespace
- doc: check for inclusive language
- doc: change to “Machine in the middle”
- doc: add exceptions for inclusive language
- doc: update doc for OVN peer routing relationships
- github: Re-organize workflows
- github: Add DCO check
- github: Split static builds from tests
- github: Add branch target test
- github: Run commit tests on target
- github: Fix commit checks
- doc: update firewall configuration documentation
- doc: add a section about LXD and Docker
- shared/network: Always return 0 on error
- lxd/utils: Improve errors in instancesExist
- lxd/device/disk: Improve error in validateConfig
- lxd/utils: Check snapshots only once in instancesExist
- i18n: Update translation templates
- doc: Fix typo
- lxd: Prevent use of instance/snapshot name in instance specific routes
- doc: document how to handle SAN vs target name mismatch in Prometheus sample
- lxc/list: Don’t skip extra info when name filtering
- doc: rework doc for setting up OVN
- lxd/certificates: Fix token request validation
- doc: add a Discourse link about simple streams
- lxc/config_trust: revoke-token requires a name, not a token
- i18n: Update translation templates
- lxd/project: Fix bad error on invalid instance type
- lxc/config_trust: update help when client should be singular
- lxc/config_trust: clarify help for join token creation
- i18n: Update translation templates
- lxd/daemon: Fix LXD crash is failing to start due to already running LXD process
- lxd/request: Export CtxKey type and use for all keys.
- lxd/storage/drivers/driver/zfs/volumes: Simplifies UnmountVolume
- lxd/storage/drivers/driver/lvm/volumes: Simplifies UnmountVolume
- lxd/storage/drivers/driver/ceph/volumes: Simplifies UnmountVolume
- lxd/storage/drivers/volumes: v.driver.MountVolumeSnapshot usage in MountTask
- lxd/storage/drivers/interface: Removes ourMount indicator from MountVolumeSnapshot
- lxd/storage/backend/lxd: b.driver.MountVolumeSnapshot usage
- lxd/storage/drivers/driver/mock: Updates MountVolumeSnapshot
- lxd/storage/driver/driver/btrfs/volumes: Adds ref counting to MountVolumeSnapshot
- lxd/storage/drivers/driver/ceph/volumes: Adds ref counting to MountVolumeSnapshot
- lxd/storage/drivers/driver/cephfs/volumes: Adds ref counting to MountVolumeSnapshot
- lxd/storage/drivers/driver/dir/volumes: Adds ref counting to MountVolumeSnapshot
- lxd/storage/drivers/driver/lvm/volumes: Adds ref counting to MountVolumeSnapshot
- lxd/storage/drivers/driver/zfs/volumes: Adds ref counting to MountVolumeSnapshot
- doc: add a JSON file that defines the doc versions
- lxc/config: no name shown when editing the instance’s config
- i18n: Update translation templates
- api: resources_pci_vpd
- shared/api: Add VPD to PCI in resources
- lxd/resources: Fix comment
- lxd/resources: Add PCI VPD support
- doc/rest-api: Refresh swagger YAML
- lxd/response: Adds nolint directive for deadcode.
- lxd: Removes unused test function (deadcode).
- lxd/network/openvswitch/ovn: Pass --wait=sb when using ovn-nbctl
- lxd/network/driver/ovn: Fix cleanup of bridge mappings of physical OVS parent
- lxd/storage/backend/lxd: Restore instance snapshot volume config and description
- test: Adds tests for restoration of instance volume snapshot config
- test: Run snapshot naming tests for all storage pool types
- shared/instance: Adds volatile.apply_nvram config key
- lxd/instance/drivers/driver/qemu: Regenerate nvram on next start when security.secureboot profile config is changed
- doc/instances: Documents VM volatile.apply_nvram setting
- test/static_analysis: Adds golangci-lint and runs deadcode.
- lxd/resources: Handle negative CPU indexes
- lxc/config_device: Add extra example to device add
- i18n: Update translation templates
- lxc/profile: Add cross-project copy
- lxd: Increase client header timeout from 30s to 1 hour
- client: Increase header timeout from 30s to 1 hour
- shared/cert: Increase header timeout from 30s to 1 hour
- lxc/config_device: Fix small typo
- i18n: Update translation templates
- tests: Test project profile copy
- lxd/storage/utils: Updates DB functions to take Pool interface argument
- lxd/storage/utils: Updates VolumeDBSnapshotsGet to align with other DB function arguments
- lxd/migrate/storage/volumes: storagePools.VolumeDBSnapshotsGet usage
- lxd/storage/backend/lxd: Update DB function usage to pass Pool
- lxd/storage/backend/lxd: Updates UpdateInstanceBackupFile to use VolumeDBGet
- lxd/storage/backend/lxd: Preallocate snapshot slice in UpdateInstanceBackupFile
- lxd/db/storage/volumes: Populate content type and config in GetLocalStoragePoolVolumeSnapshotsWithType
- lxd/storage/backend: Populate instance storage volume snapshots in UpdateInstanceBackupFile
- lxd/backup: Switch to storagePools.VolumeDBGet in volumeBackupCreate
- lxd/backup: Updates volumeBackupWriteIndex to use storagePools.VolumeDBSnapshotsGet
- lxd/db/storage/volumes/snapshots: Removes unused function GetStorageVolumeSnapshotsNames
- lxd/storage/backend/lxd: Updates BackupCustomVolume to use VolumeDBSnapshotsGet
- lxd/images: Remove references to cluster node in user facing error messages
- lxd/db/query/transaction: Start errors with capital letter
- lxd/backup: Whitespace
- lxd/backup: Populate volume config in backupWriteIndex
- test: Add tests for checking that storage volume records are exported and restored from backups
- lxd/storage/utils: Adds removeUnknownKeys argument to VolumeDBCreate
- lxd/storage/backend/lxd: VolumeDBCreate updated usage
- lxd/storage/backend/lxd: Updates CreateInstanceFromMigration to strip unknown storage volume config keys
- lxd/storage/backend/lxd: Don’t strip unknown storage volume keys for same pool copies in CreateCustomVolumeFromCopy
- lxd/storage/backend/lxd: Updates CreateCustomVolumeFromMigration to strip unknown storage volume config keys
- lxd/storage/backend/lxd: VolumeDBCreate updated usage when stripping unknown storage volume config keys
- lxd/storage/backend/lxd: Updates CreateInstanceFromBackup to use storage volume config from backup
- lxd/storage/backend: Copy snapshot config for same-pool copies in CreateCustomVolumeFromCopy
- lxd/storage/backend/lxd: Use snapshot volume config when restoring custom volume in CreateCustomVolumeFromBackup
- test: Add tests to check snapshot volume config is restored from backup
- lxc/config_trust: Fix single argument trust remove
- i18n: Update translation templates
- lxd/filter: Support case insensitive and regular expresions
- client: Add support for server-side filtering for CLI list commands
- lxc/list: Use server-side filtering for list command
- lxc/image: Use server-side filtering for list command
- lxc/storage_volume: Use server-side filtering for list command
- lxc/utils: Add structHasField and getServerSupportedFilters
- i18n: Update translation templates
- Revert “lxc/config: no name shown when editing the instance’s config”
- shared/api: Mark ExpandedConfig and ExpandedDevices fields as omitempty
- lxc/config: Exclude expanded config from edit screen
- lxd/filter/value: Fix parsing of yaml tags in ValueOf
- i18n: Update translation templates
- test: Check instance name in edit screen
- lxd/resources: Handle unreadable VPD
- lxd/device: Add rbd format constants
- lxd-agent/state: Handle cgroup v2 cpu usage
- lxd-agent/state: Use /proc/meminfo
- lxd/device: Add DiskGetRBDFormat
- lxd/device: Add DiskParseRBDFormat
- lxd/storage/drivers/ceph: Add cluster and user name constants
- lxd/storage/drivers/ceph: Use constants
- lxd/device/disk: Use ceph constants
- lxd/instance/drivers/qmp: Add AddSecret
- lxd/storage/drivers/ceph: Add CephGetRBDImageName
- lxd/instance/drivers/qemu: Use QEMU rbd driver if possible
- lxd/device/disk: Use rbd syntax for rbd volumes
- lxd/storage/drivers/ceph: Fix typo
- lxd/instance/drivers/qemu: Handle rbd root devices
- lxd/instance: Fix auto-snapshot on restricted projects
- lxd/instance/qemu: Fix ceph username key
- lxd/instance/qemu: Fix ceph volname in projects
- test: Add additional tests for checking volume snapshot config is copied
- lxd/storage/volume: Use source snapshot volume config when copying to new volume in cmdStorageVolumeCopy
- lxd/storage/volume: Fix move of snapshot in cmdStorageVolumeCopy
- test: Adds check that source snapshot config is used when copying snapshot to new volume
- lxd/cluster/heartbeat: Improve heartbeat logging
- doc: add link to YouTube video for network ACLs
- lxd-agent/devlxd: Don’t expand format strings
- lxd/instance/drivers/qemu: Set conf option in rbd driver
- lxd/storage/pool/interface: Add GenerateInstanceBackupConfig definition
- lxd/storage/backend/mock: Adds GenerateInstanceBackupConfig definition
- lxd/storage/backend/lxd: Splits UpdateInstanceBackupFile into GenerateInstanceBackupConfig
- lxd/backup: Updates backupWriteIndex to use pool.GenerateInstanceBackupConfig
- lxd/storage/pool/interface: Adds GenerateCustomVolumeBackupConfig definition
- lxd/storage/backend/mock: Adds GenerateCustomVolumeBackupConfig function
- lxd/storage/backend/lxd: Adds GenerateCustomVolumeBackupConfig function
- lxd/backup: Updates volumeBackupWriteIndex to use pool.GenerateCustomVolumeBackupConfig
- lxd/backup: Move config struct into own package to use in non-cgo applications
- lxd/api/internal: backup.ConfigToInstanceDBArgs usage
- lxd/api/internal/recover: backupConfig usage
- lxd/api/internal/recover: backup.ConfigToInstanceDBArgs usage
- lxd/backup/backup/info: config.Config usage
- lxd/instance/instance/utils: backup.ConfigToInstanceDBArgs usage
- lxd/migrate/storage/volumes: Switch to using pool.GenerateCustomVolumeBackupConfig
- lxd/storage/backend/lxd: backupConfig.Config usage
- lxd/storage/backend/mock: backupConfig.Config usage
- lxd/storage/pool/interface: backupConfig.Config usage
- lxd/storage/backend/lxd: Fix incorrect revert in ImportCustomVolume
- lxd/instance/drivers/qemu: Fix ceph conf filename
- lxd/storage/drivers: Handle seek errors
- lxd/storage/backend/lxd: Handle error from vol.MountTask in detectUnknownCustomVolume
- lxd/storage/backend/lxd: Handle srcVol.MountTask errors
- lxd-generate: Explictly ignore rows.Close error in defer.
- lxd/db: Regenerate database methods.
- tests: Test stateless migration
- client: Checks or explicitly ignores errors.
- fuidshift: Checks or explicitly ignores errors.
- lxd/db/generate/db/stmt: Use join tag value instead of hardcoded ‘name’
- lxc: Checks or explicitly ignores errors.
- lxc-to-lxd: Checks or explicitly ignores errors.
- lxd-agent: Checks or explicitly ignores errors.
- lxd-benchmark: Checks or explicitly ignores errors.
- lxd-migrate: Checks or explicitly ignores errors.
- lxd-user: Checks or explicitly ignores errors.
- shared: Checks or explicitly ignores errors.
- test/macaroon-identity: Explicitly ignores errors.
- lxd/apparmor: Checks or explicitly ignores errors.
- lxd/archive: Checks or explicitly ignores errors.
- lxd/backup: Checks or explicitly ignores errors.
- lxd/bgp: Checks or explicitly ignores errors.
- lxd/cluster: Checks or explicitly ignores errors.
- lxd/config: Checks or explicitly ignores errors.
- lxd/db: Checks or explicitly ignores errors.
- lxd/device: Checks or explicitly ignores errors.
- lxd/dns: Checks or explicitly ignores errors.
- lxd/endpoints: Checks or explicitly ignores errors.
- lxd/events: Checks or explicitly ignores errors.
- lxd/firewall: Checks or explicitly ignores errors.
- lxd/fsmonitor: Checks or explicitly ignores errors.
- lxd/instance: Checks or explicitly ignores errors.
- lxd/maas: Checks or explicitly ignores errors.
- lxd/metrics: Checks or explicitly ignores errors.
- lxd/migration: Checks or explicitly ignores errors.
- lxd/network: Checks or explicitly ignores errors.
- lxd/operations: Checks or explicitly ignores errors.
- lxd/rbac: Checks or explicitly ignores errors.
- lxd/resources: Checks or explicitly ignores errors.
- lxd/response: Checks or explicitly ignores errors.
- lxd/rsync: Checks or explicitly ignores errors.
- lxd/storage: Checks or explicitly ignores errors.
- lxd/sys: Checks or explicitly ignores errors.
- lxd/task: Checks or explicitly ignores errors.
- lxd/util: Checks or explicitly ignores errors.
- lxd/seccomp: Checks or explicitly ignores errors.
- lxd/ip: Checks or explicitly ignores errors.
- lxd: Checks or explicitly ignores errors.
- test: Adds errcheck linter and list of excludes.
- i18n: Updates translations.
- lxd/storage/backend/lxd: Fix argument logging name in MigrateCustomVolume and CreateCustomVolumeFromMigration
- lxd/storage/backend/lxd: Fix refresh of non existent volume in CreateCustomVolumeFromMigration
- test: Add checks for custom storage volume refresh
- doc/storage: Add CEPHFS to table
- docs: Typo in ovn docs
- doc: add a version switcher to the docs
- lxd: Add allowInconsistent to migration fields.
- lxd: Default allowInconsistent to false for snapshots.
- lxd: Pass allowInconsistent to migration source on instance post.
- lxd/storage/drivers: Optionally ignore rsync error 24 for local copy.
- lxd/storage/drivers: Updates calls to genericVFSCopy.
- lxd/storage/drivers: Add allowInconsistent to CreateVolumeFromCopy on driver interface.
- lxd/storage: Pass allowInconsistent into driver.CreateVolumeFromCopy.
- lxd: Use allowInconsistent for pool and project migrations.
- lxd/instances/post: Handle unknown errors in createFromMigration
- lxd/migration: Updates proto buf version
- lxd/migration/utils: Adds IndexHeaderVersion to migration header
- lxd/migration/utils: Updates comments on constant to indicate index header may be sent first
- lxd/migration/utils: Adds IndexHeaderVersion constant
- test: Adds cross pool checking of storage volume snapshot config population
- lxd/migration/migration/volumes: Adds IndexHeaderVersion to VolumeSourceArgs and VolumeTargetArgs
- lxd/migration/migration/volumes: Adds InfoResponse struct
- lxd/storage/backend/lxd: Updates CreateCustomVolumeFromCopy to use srcPool.GenerateCustomVolumeBackupConfig
- lxd/storage/backend/lxd: Add support for migration index header and response in MigrateCustomVolume and CreateCustomVolumeFromCopy
- lxd/migrate/storage/volumes: Add support for migration index frame for custom volume migration
- lxd/storage/backend/lxd: Update RefreshCustomVolume to use srcPool.GenerateCustomVolumeBackupConfig
- lxd/storage/utils: Removes unused syncSnapshotsVolumeGet function
- test: Expand test_storage_local_volume_handling tests to check for custom volume refresh
- test: Add checks for custom storage volume config copy in migration
- lxd/storage/drivers: Adds allowInconsistent argument to RefreshVolume.
- lxd/storage: Pass allowInconsistent flag into volume refresh call.
- lxd/instance/qemu: Implement qemuBaseSections
- lxd/instance/qemu: Replace qemuBase template with qemuBaseSections
- lxd/bgp: Enable multihop
- doc: move content to different files
- doc: add a link to the YouTube video on BGP
- lxd/apparmor: Add mknod cap to archive
- lxd/instance/qemu: Fix some variable names to match code style
- doc: remove duplicate entries
- doc: spell out configuration option
- doc: fix capitalization of Ceph, CephFS, Btrfs, LVM and ZFS
- lxd/sys: Add ReleaseInfo
- lxd/api: Switch to OS struct for release info
- lxd/api: Reduce number of DB queries
- lxd/sys: Move KernelVersion from state
- lxd/sys: Add Uname
- lxd/cluster/config: Split to separate package
- lxd/state: Add global configuration
- lxd/cluster/config: Update for golint
- lxd: Update for config function names
- lxd/daemon: Setup globalConfig
- lxd/api_cluster: Reduce calls to State
- lxd/daemon_config: Use GlobalConfig
- lxd/api_1.0: Update daemon config
- lxd/state: Add ServerName
- lxd/daemon: Fill ServerName
- lxd/api_1.0: Use state variables
- lxd/api: Don’t access DB on every request
- lxd/storage_volumes_snapshot: Port to State config
- lxd/instance_post: Port to State config
- lxd/cluster: Port to State config
- lxd/instances_post: Port to State config
- lxd/images: Port to State config
- lxd/cluster/config: Add MetricsAuthentication
- lxd/api_metrics: Avoid needless DB calls
- lxd/network: Disable IPv6 on bridges with no IPv6 address
- lxd/db/snapshots: Add by-ID
- lxd/db/snapshots: Add GetLocalExpiredInstanceSnapshots
- tests: Don’t route IPv6 on network without IPv6
- lxd/instance/lxc: Reduce DB calls in Metrics
- lxd/cluster/config: Update test for new package
- lxd/api_cluster: Update serverName on bootstrap/join
- lxd/state: Initialize GlobalConfig in testing struct
- lxd/cluster: Update for State config
- lxd/daemon: Use Mutex around globalConfig and serverName
- lxd/api_cluster: Use Mutex around serverName
- lxd/instance: Use GetLocalExpiredInstanceSnapshots
- lxd/storage_volumes_snapshots: Don’t trigger expiry task on load
- lxd/db/storage_volumes_snapshots: Skip snapshots with NULL expiry
- lxd/events/connections: Add EventListenerConnection interface
- lxd/events: Implement websocketListenerConnection
- lxd/events: Implement streamListenerConnection
- lxd/events/common: Remove unused WriteMessage function
- lxd/events: Use EventListenerConnection in listenerCommon
- lxd/devlxd: Support http streaming
- lxd-agent/events: Support http streaming
- lxd: Use EventListenerConnecion
- lxd/events: s/heartbeat/start/
- test/deps/devlxd-client: Add stream monitor
- test: Add devlxd http stream
- gomod: Update dependencies
- i18n: Update translations from weblate
- lxd/db: Port GetStoragePoolVolumesWithType to ClusterTx
- lxd/storage_volumes_snapshots: Update to ClusterTx
- lxd/main_activateifneeded: Update to ClusterTx
- lxd/instance: Optimize instance snapshots
Try it for yourself
This new LXD release is already available for you to try on our demo service.
Downloads
The release tarballs can be found on our download page.
Binary builds are also available for:
- Linux: snap install lxd
- MacOS: brew install lxc
- Windows: choco install lxc