LXD 4.0 LTS has been released

stgraber · March 31, 2020, 11:45pm

Introduction

The LXD team is very excited to announce the release of LXD 4.0 LTS!

This is the 3rd LTS release for LXD and a very busy and exciting one!
The changelog below is split so that both users of LXD 3.23 and LXD 3.0 can see what we have in store for them.

As with all our other LTS releases, this one will be supported for 5 years (June 2025) and will receive a number of bugfix and security point releases over that time.

As for LXD 3.0, we’re hoping to release one last bugfix release as 3.0.5 in the near future before we enter security-only maintenance mode for its remaining 3 years.

Enjoy!

Breaking changes

Removal of `--container-only`, replaced by `--instance-only`

Our only CLI breaking changes with this release is the replacement of --container-only by --instance-only. Those following the feature releases will have had both supported for a few months now. With the 4.0 release, we’re removing the deprecated ones.

Highlights for 3.23 users

virtual machines: Support for backup (import/export)

It is now possible to use lxc export and lxc import with virtual machines.

A word of caution however. Virtual machines, unlike containers are only accessible as a large block device. This means that several GB of data will need to be read and compressed, no matter how much is actually used inside the VM.

This can lead to long export times and similarly long import times.

Doing so with --optimized on a backend like ZFS should considerably reduce the export time, assuming the backup is to be imported on a storage pool of the same type.

resources: PCI and USB devices in the resource API

The resources API (/1.0/resources) has been extended with a list of all PCI and USB devices on the system. This is of particular use when dealing with VFIO passthrough to virtual machines or passing through USB devices to containers.

Long example output

stgraber@castiana:~$ lxc query /1.0/resources | jq .pci
{
  "devices": [
    {
      "driver": "skl_uncore",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:00.0",
      "product": "Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers",
      "product_id": "5904",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "i915",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:02.0",
      "product": "HD Graphics 620",
      "product_id": "5916",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "",
      "driver_version": "",
      "numa_node": 0,
      "pci_address": "0000:00:08.0",
      "product": "Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model",
      "product_id": "1911",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "xhci_hcd",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:14.0",
      "product": "Sunrise Point-LP USB 3.0 xHCI Controller",
      "product_id": "9d2f",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "intel_pch_thermal",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:14.2",
      "product": "Sunrise Point-LP Thermal subsystem",
      "product_id": "9d31",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "mei_me",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:16.0",
      "product": "Sunrise Point-LP CSME HECI #1",
      "product_id": "9d3a",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1c.0",
      "product": "Sunrise Point-LP PCI Express Root Port #1",
      "product_id": "9d10",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1c.2",
      "product": "Sunrise Point-LP PCI Express Root Port #3",
      "product_id": "9d12",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1c.4",
      "product": "Sunrise Point-LP PCI Express Root Port #5",
      "product_id": "9d14",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1d.0",
      "product": "Sunrise Point-LP PCI Express Root Port #9",
      "product_id": "9d18",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "",
      "driver_version": "",
      "numa_node": 0,
      "pci_address": "0000:00:1f.0",
      "product": "Sunrise Point LPC Controller/eSPI Controller",
      "product_id": "9d4e",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "",
      "driver_version": "",
      "numa_node": 0,
      "pci_address": "0000:00:1f.2",
      "product": "Sunrise Point-LP PMC",
      "product_id": "9d21",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "snd_hda_intel",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1f.3",
      "product": "Sunrise Point-LP HD Audio",
      "product_id": "9d71",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "i801_smbus",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1f.4",
      "product": "Sunrise Point-LP SMBus",
      "product_id": "9d23",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "e1000e",
      "driver_version": "3.2.6-k",
      "numa_node": 0,
      "pci_address": "0000:00:1f.6",
      "product": "Ethernet Connection (4) I219-LM",
      "product_id": "15d7",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "rtsx_pci",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:02:00.0",
      "product": "RTS525A PCI Express Card Reader",
      "product_id": "525a",
      "vendor": "Realtek Semiconductor Co., Ltd.",
      "vendor_id": "10ec"
    },
    {
      "driver": "iwlwifi",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:04:00.0",
      "product": "Wireless 8265 / 8275",
      "product_id": "24fd",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "nvme",
      "driver_version": "1.0",
      "numa_node": 0,
      "pci_address": "0000:05:00.0",
      "product": "SSD 600P Series",
      "product_id": "f1a5",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:06:00.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:07:00.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:07:01.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:07:02.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:07:04.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "thunderbolt",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:08:00.0",
      "product": "JHL6540 Thunderbolt 3 NHI (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d2",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:09:00.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0a:00.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0a:01.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0a:02.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0a:04.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "ahci",
      "driver_version": "3.0",
      "numa_node": 0,
      "pci_address": "0000:0b:00.0",
      "product": "",
      "product_id": "0622",
      "vendor": "ASMedia Technology Inc.",
      "vendor_id": "1b21"
    },
    {
      "driver": "xhci_hcd",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0c:00.0",
      "product": "FL1100 USB 3.0 Host Controller",
      "product_id": "1100",
      "vendor": "Fresco Logic",
      "vendor_id": "1b73"
    },
    {
      "driver": "atlantic",
      "driver_version": "5.4.0-18-generic-kern",
      "numa_node": 0,
      "pci_address": "0000:0d:00.0",
      "product": "AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion]",
      "product_id": "87b1",
      "vendor": "Aquantia Corp.",
      "vendor_id": "1d6a"
    }
  ],
  "total": 32
}

stgraber@castiana:~$ lxc query /1.0/resources | jq .usb
{
  "devices": [
    {
      "bus_address": 1,
      "device_address": 4,
      "interfaces": [
        {
          "class": "Wireless",
          "class_id": 224,
          "driver": "btusb",
          "driver_version": "0.8",
          "number": 0,
          "subclass": "Radio Frequency",
          "subclass_id": 1
        },
        {
          "class": "Wireless",
          "class_id": 224,
          "driver": "btusb",
          "driver_version": "0.8",
          "number": 1,
          "subclass": "Radio Frequency",
          "subclass_id": 1
        }
      ],
      "product": "",
      "product_id": "0a2b",
      "speed": 12,
      "vendor": "Intel Corp.",
      "vendor_id": "8087"
    },
    {
      "bus_address": 1,
      "device_address": 3,
      "interfaces": [
        {
          "class": "Video",
          "class_id": 14,
          "driver": "uvcvideo",
          "driver_version": "1.1.1",
          "number": 0,
          "subclass": "Video Control",
          "subclass_id": 1
        },
        {
          "class": "Video",
          "class_id": 14,
          "driver": "uvcvideo",
          "driver_version": "1.1.1",
          "number": 1,
          "subclass": "Video Streaming",
          "subclass_id": 2
        }
      ],
      "product": "Integrated Camera",
      "product_id": "b5ce",
      "speed": 480,
      "vendor": "Chicony Electronics Co., Ltd",
      "vendor_id": "04f2"
    },
    {
      "bus_address": 3,
      "device_address": 2,
      "interfaces": [
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 0,
          "subclass": "Control Device",
          "subclass_id": 1
        },
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 1,
          "subclass": "Streaming",
          "subclass_id": 2
        },
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 2,
          "subclass": "Streaming",
          "subclass_id": 2
        },
        {
          "class": "Human Interface Device",
          "class_id": 3,
          "driver": "usbhid",
          "driver_version": "5.4.0-18-generic",
          "number": 3,
          "subclass": "",
          "subclass_id": 0
        }
      ],
      "product": "TX42C500",
      "product_id": "4933",
      "speed": 12,
      "vendor": "Realtek Semiconductor Corp.",
      "vendor_id": "0bda"
    },
    {
      "bus_address": 3,
      "device_address": 13,
      "interfaces": [
        {
          "class": "Video",
          "class_id": 14,
          "driver": "uvcvideo",
          "driver_version": "1.1.1",
          "number": 0,
          "subclass": "Video Control",
          "subclass_id": 1
        },
        {
          "class": "Video",
          "class_id": 14,
          "driver": "uvcvideo",
          "driver_version": "1.1.1",
          "number": 1,
          "subclass": "Video Streaming",
          "subclass_id": 2
        },
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 2,
          "subclass": "Control Device",
          "subclass_id": 1
        },
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 3,
          "subclass": "Streaming",
          "subclass_id": 2
        }
      ],
      "product": "HD Pro Webcam C920",
      "product_id": "082d",
      "speed": 480,
      "vendor": "Logitech, Inc.",
      "vendor_id": "046d"
    },
    {
      "bus_address": 3,
      "device_address": 16,
      "interfaces": [
        {
          "class": "Human Interface Device",
          "class_id": 3,
          "driver": "usbhid",
          "driver_version": "5.4.0-18-generic",
          "number": 0,
          "subclass": "",
          "subclass_id": 0
        },
        {
          "class": "Chip/SmartCard",
          "class_id": 11,
          "driver": "usbfs",
          "driver_version": "5.4.0-18-generic",
          "number": 1,
          "subclass": "",
          "subclass_id": 0
        }
      ],
      "product": "YubiKey FIDO+CCID",
      "product_id": "0406",
      "speed": 12,
      "vendor": "Yubico.com",
      "vendor_id": "1050"
    },
    {
      "bus_address": 3,
      "device_address": 17,
      "interfaces": [
        {
          "class": "Human Interface Device",
          "class_id": 3,
          "driver": "usbhid",
          "driver_version": "5.4.0-18-generic",
          "number": 0,
          "subclass": "Boot Interface Subclass",
          "subclass_id": 1
        },
        {
          "class": "Human Interface Device",
          "class_id": 3,
          "driver": "usbhid",
          "driver_version": "5.4.0-18-generic",
          "number": 1,
          "subclass": "Boot Interface Subclass",
          "subclass_id": 1
        }
      ],
      "product": "ThinkPad Compact USB Keyboard with TrackPoint",
      "product_id": "6047",
      "speed": 12,
      "vendor": "Lenovo",
      "vendor_id": "17ef"
    }
  ],
  "total": 6
}

network: Support for multiple ipvlan NIC devices

Multiple ipvlan devices can now be added to the same container provided that one of them has ipv4.gateway and/or ipv6.gateway set to none.

network: Support for host addresses on routed NIC

The host side address on routed nics can now be configured through the ipv4.host_address and ipv6.host_address properties.

clustering: Support for editing cluster roles

A new lxc cluster edit command allows for editing clustering roles.

It’s worth noting that there currently are no writable roles, but we expect to be adding some in the near future which will then be manageable through this API and command.

instances: Disk usage for custom volumes

Containers with custom storage volumes attached to them will now report those volume’s usage in the state API (and through lxc info):

stgraber@castiana:~$ lxc launch images:ubuntu/bionic c1
Creating c1
Starting c1

stgraber@castiana:~$ lxc storage volume create default vol1
Storage volume vol1 created
stgraber@castiana:~$ lxc storage volume create default vol2
Storage volume vol2 created

stgraber@castiana:~$ lxc storage volume attach default vol1 c1 vol1 /mnt/vol1
stgraber@castiana:~$ lxc storage volume attach default vol2 c1 vol2 /mnt/vol2

stgraber@castiana:~$ lxc info c1
Name: c1
Location: none
Remote: unix://
Architecture: x86_64
Created: 2020/04/01 00:00 UTC
Status: Running
Type: container
Profiles: default
Pid: 1439012
Ips:
  eth0:	inet	10.166.11.66	veth12c5ea18
  eth0:	inet6	fd42:4c81:5770:1eaf:216:3eff:fee2:43b6	veth12c5ea18
  eth0:	inet6	fe80::216:3eff:fee2:43b6	veth12c5ea18
  lo:	inet	127.0.0.1
  lo:	inet6	::1
Resources:
  Processes: 14
  Disk usage:
    root: 1.11MB
    vol1: 98.30kB
    vol2: 98.30kB
  CPU usage:
    CPU usage (in seconds): 0
  Memory usage:
    Memory (current): 46.94MB
  Network usage:
    eth0:
      Bytes received: 3.06kB
      Bytes sent: 2.93kB
      Packets received: 22
      Packets sent: 28
    lo:
      Bytes received: 0B
      Bytes sent: 0B
      Packets received: 0
      Packets sent: 0

instances: Disk usage for snapshots

The API now exposes the size of each individual snapshots.

stgraber@castiana:~$ lxc snapshot c1
stgraber@castiana:~$ lxc query /1.0/instances/c1/snapshots/snap0 | jq .size
61440

This will soon be displayed in lxc info once it’s gone through a redesign.

auth: Support for passwordless PKI mode

For those using LXD with a managed PKI, it is now possible to configure LXD to automatically trust any client certificate signed by the CA.

This is done with core.trust_ca_certificates.

To handle revocation, LXD also now accepts a CRL which should be placed alongside server.ca as server.crl.

Highlights for 3.0 users

In addition to the features and changes listed above, those who were using the LXD 3.0 LTS branch have the following “new” features to look forward to:

Virtual machines

LXD can now run both containers and virtual machines.

The experience and configuration works in much the same way though some device types and configuration options aren’t available for virtual machines yet.

Some operations are performed through an agent running in the virtual machine (lxc exec and lxc file). The agent comes pre-installed in the majority of our images.

To create a virtual machine rather than a container, simply pass --vm to lxc launch

VM images are now available for most commonly used Linux distributions with plans to add more in the future.

stgraber@castiana:~$ lxc launch images:centos/8 centos-8 --vm
Creating centos-8
Starting centos-8                             

stgraber@castiana:~$ lxc info centos-8
Name: centos-8
Location: none
Remote: unix://
Architecture: x86_64
Created: 2020/03/31 23:48 UTC
Status: Running
Type: virtual-machine
Profiles: default
Pid: 1426453
Ips:
  enp5s0:	inet	10.166.11.125
  enp5s0:	inet6	fd42:4c81:5770:1eaf:1c5b:d0a1:d892:5464
  enp5s0:	inet6	fe80::9bbf:7460:2ad0:6a9
  lo:	inet	127.0.0.1
  lo:	inet6	::1
Resources:
  Processes: 12
  Disk usage:
    root: 6.65MB
  CPU usage:
    CPU usage (in seconds): 5
  Memory usage:
    Memory (current): 123.94MB
    Memory (peak): 115.95MB
  Network usage:
    enp5s0:
      Bytes received: 2.55kB
      Bytes sent: 2.32kB
      Packets received: 21
      Packets sent: 20
    lo:
      Bytes received: 0B
      Bytes sent: 0B
      Packets received: 0
      Packets sent: 0

stgraber@castiana:~$ lxc exec centos-8 bash
[root@centos-8 ~]# cat /etc/redhat-release 
CentOS Linux release 8.1.1911 (Core) 
[root@centos-8 ~]# uname -a
Linux centos-8 4.18.0-147.5.1.el8_1.centos.plus.x86_64 #1 SMP Thu Feb 6 10:31:58 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@centos-8 ~]#

Projects

Projects are a way to segment your LXD server.
Each project can contain its own set of instances, images, profiles and storage volumes.

Those various features can be enabled/disabled on a per-project basis. If disable, the project inherit from the default project.

On top of this, there is support for both restrictions (disabling particular device types, privileged containers, …) and limits (limiting the amount of CPU, memory and instance count).

Instances

System call interception on containers
- Allows for limited mknod in containers
- Allows for limited setxattr in containers
- Can be used to allow mounting of privileged filesystems
- Can be used to redirect some filesystem mounts to FUSE
Addition of a backup/restore feature (lxc export and lxc import)
Copy/move instances between storage pools
Refresh of an instance copy (local or remote) with lxc copy --refresh
Protection against accidental deletion and shift (security.protection.delete and security.protection.shift)
shiftfs is now supported and used when available (replaces traditional shifting)
Automated snapshots and expiration
New unix-hotplug device type (similar to unix-char and unix-block)
usb device improvements:
- The add/remove uevent is now forwarded to the container
- It is possible to pass all USB devices
proxy device improvements:
- Privileged dropping options (security.uid and security.gid)
- Socket ownership options (uid, gid, mode)
- Support for HAProxy type header (proxy_protocol)
- Fast proxying using NAT when available (nat)
- Support for unix socket, udp and port ranges on udp and tcp
disk device improvements:
- Direct attach of Ceph rbd/fs disks to containers
- Custom mount options
- shift property to translate uid/gid into container-readable ones
nic device improvements:
- New ipvlan nictype
- New routed nictype
- ipv4.routes and ipv6.routes properties
- network property to easily connect to LXD managed networks
- Scurity filtering options
- VLAN & MAC filtering on SR-IOV devices

Network

Configurable NAT source address (ipv4.nat.address and ipv6.nat.address)
DHCP leases API and lxc network list-leases command
Network state API and lxc network info command
Configurable MAC address on managed networks (bridge.hwaddr)
Control on firewall rule application order (ipv4.nat.order and ipv6.nat.order)

Storage

New internal storage layer rewritten from scratch
New cephfs storage backend
Backups and images can now be stored inside a storage pool
Custom storage volume snapshots (including scheduling & expiry)
LVM striping support
Separate metadata and data pools for Ceph
Quotas on dir backend through ext4/xfs “project quotas”
security.shifted property on custom storage volumes

Images

API for nested LXD to fetch images from the host (security.devlxd.images)
squashfs compression support for new images
Profiles can now be tied to images
Image expiry can now be changed

Clustering improvements

Support for standby database nodes
Configurable number of database & standby nodes
Mixed architecture clustering
Clustering roles
New simplified cluster join API
Separate addresses for client and cluster traffic
Automatic image replication

CLI

New columns in lxc list and lxc image list
New lxc alias command
Consistent list commands including --format support
All set commands now accept multiple key=value
exec now accepts --uid, --gid and --cwd
Config overrides on lxc copy and lxc move
More commands now support the --target option for clustering

Future proofing

Support for nftables as an alternative to xtables
Support for limits through Cgroup2

API

Support for RBAC (Role Based Access Control) through Canonical RBAC
Default TLS key is now EC384
New /1.0/instances endpoint replacing /1.0/containers
Addition of server-side collection filtering on /1.0/instances and /1.0/images
Much more comprehensive resources API at /1.0/resources
Kernel features are now exposed in /1.0
LXC features are now exposed in /1.0
Built-in debug server (pprof) configurable through core.debug_address
Additional bulk-query (recursion) options for high demand endpoints
Events and Operations in a clustered environment now have a Location field

Complete changelog

Here is a complete list of all changes in this release:

Full commit list

shared/version/api: Add trust_ca_certificates
doc: Add core.trust_ca_certificates
lxd/cluster/config: Add core.trust_ca_certificates
*: Add parameters to CheckTrustState
shared/cert: Add CRL to CertInfo
lxd/util/http: Check CRL for revoked clients
test: Extend PKI test
lxd/etag: Quote generated etag values
lxd/apparmor: Apparently the order matters
shared/version/api: Add snapshot_disk_usage API extension
doc: Add snapshot_disk_usage
lxd/storage/drivers/btrfs: Fix quota
lxd/backup: Removes Privileged field from backup.Info struct
lxd/backup: Adds new fields in index.yaml
lxd/instances/post: bInfo.OptimizedStorage pointer usage
lxd/storage/backend/lxd: CreateInstanceFromBackup OptimizedStorage pointer usage
lxd/backup: Updates backupWriteIndex index.yaml fields
lxd/backup: Removes Project field from index.yaml
test/suites/storage: Add btrfs quota tests
shared/api: Add size to InstanceSnapshot
lxd/instance/drivers: Get snapshot usage
lxd/storage/drivers/btrfs: Don’t destroy qgroups
lxd/storage/drivers: Moves functions from generic.go to generic_vfs.go
lxd/storage/drivers: Generic VFS function usage after move &rename
lxd/instance/drivers: Add custom volumes to disk state
lxd/instance/drivers: Fix lxd-agent running order
lxc: Deprecate --container-only
i18n: Update translation templates
tests: Move away from container-only
lxc: Drop flagContainerOnly
lxd/storage/zfs: Fix deleted VM images restoration
lxc/storage/drivers/driver/btrfs/volumes: CreateVolumeFromBackup to use tar reader for optimized volume restore
lxc/storage/drivers/driver/zfs/volumes: CreateVolumeFromBackup to use tar reader for optimized volume restore
shared/archive: Adds CompressedTarReader function
lxd/backup/backup: shared.CompressedTarReader usage
test/suites/static/analysis: Reinstates checks for shared/instancewriter
lxd/instance/post: InstanceID usage
lxd/db/containers: Renames ContainerID to InstanceID
lxd/instances/post: Logging in createFromBackup
lxd/instances/post: Logging message change from container to instance
lxd/instances/post: Switches to revert package in createFromBackup
lxd: Merges instanceCreateFromBackup into createFromBackup
lxd/storage/drivers/utils: Adds blockDevSizeBytes function
lxd/storage/drivers/driver/ceph/volumes: Updates SetVolumeQuota to use blockDevSizeBytes
shared/instancewriter/instance/file/info: Adds FileInfo for os.FileInfo implementation
shared/instancewriter/instance/tar/writer: Adds WriteFileFromReader function
lxd/backup: Switches index.yaml file generation to use WriteFileFromReader in backupCreate
lxd/api/internal: d.cluster.InstanceID usage
lxd/storage/backend/lxd: Better error msg context in CreateInstanceFromBackup
lxd/backup: Removes volume type restriction in backupCreate
lxd/storage/drivers/generic/vfs: Adds VM support to genericVFSBackupVolume
lxd/storage/drivers: Uses sourcePath logging for consistency in BackupVolume
lxd/storage/drivers/driver/zfs/volumes: Adds optimised VM backup to BackupVolume
lxd/storage/drivers/driver/btrfs/volumes: Adds optimised VM backup to BackupVolume
lxd/storage/backend/lxd: Adds volume type logic for VMs to CreateInstanceFromBackup
lxd/api/internal: makes internalImport VM aware
lxd/storage/drivers/generic/vfs: Adds VM support to genericVFSBackupUnpack
lxd/storage/drivers/driver/zfs/volumes: MountVolume comment improvements
lxd/storage/drivers/driver/zfs/volumes: UnmountVolume improvements
lxd/storage/drivers/driver/zfs/volumes: Adds VM support to generic mode in MigrateVolume
lxd/storage/drivers/driver/zfs/volumes: Adds VM support to MountVolumeSnapshot
lxd/storage/drivers/driver/zfs/volumes: Adds VM support to UnmountVolumeSnapshot
lxd/storage/drivers/driver/zfs/volumes: Adds support for VM optimized backup restore
lxd/storage/drivers: Adds existing volume check to optimized backup restore
lxd/storage/drivers/driver/btrfs/volumes: Adds support for VM optimized backup restore
lxd/storage/backend/lxd: Updates CheckInstanceBackupFileSnapshots to be VM aware
lxd/storage/backend/lxd/patches: Ignores snapshots when retrieving list of custom volumes to be renamed
lxd/containers: Emit lifecycle event on user shutdown
lxd/storage/drivers: Adds OptimizedBackups driver Info flag
lxd/backup: Ignore requests for optimized backups when pool driver doesn’t support it
lxd/instances/post: Ensure optimized backup imports only import into same storage driver pools
lxd/instance/exec: Adds protection against clients reconnecting after exec has started
doc: Fix escaping
lxd/cluster: Tweak errors
api: clustering_edit_roles
shared/api: Add ClusterMemberPut
lxd/cluster: Make ClusterMember editable
client: Add UpdateClusterMember
lxc/cluster: Add edit sub-command
i18n: Update translation templates
lxd/firewall/drivers/drivers/consts: Adds FilterIPv6All constant
cgroup/init: close controllers file
doc/networks: Add missing maas.subnet.ipv4/maas.subnet.ipv6
scripts/bash: Add maas.subnet.ipv4/maas.subnet/ipv6 to network
client: Fix bad description for UpdateClusterMember
lxd/device/nic/bridged: Allow security.ipv6_filtering to be used on networks without IPv6
lxd/firewall/drivers/drivers/xtables: Adds FilterIPv6All support
lxd/firewall: Dont use compact function arg definitions
lxd/firewall/drivers/drivers/nftables: Adds FilterIPv6All support
lxd/network/network/utils: Adds support for bridged NIC network property when rebuilding dnsmasq static config
lxd/network/network/utils: Comment consistency
lxd/device/nic/bridged: Allow security.ipv4_filtering to be used on networks without IPv4
lxd/firewall/drivers/drivers/consts: Adds FilterIPv4All constant
lxd/firewall/drivers/drivers/xtables: Adds Adds FilterIPv4All support
lxd/firewall/drivers/drivers/nftables: Adds FilterIPv4All support
test: Adds bridged NIC tests for total protocol filtering
lxd/device/nic: Adds ipv4.host_address and ipv6.host_address keys
lxd/device/nic/routed: Adds ability to specify host-side veth interface IP address
api: Adds container_nic_routed_host_address API extension
doc/instances: Updates routed nic doc with ipv4.host_address and ipv6.host_address keys
scripts/bash/lxd-client: Updates bash device keys for routed NIC
lxd/device/nic/ipvlan: Adds ipv4.gateway and ipv6.gateway support
api: Adds container_nic_ipvlan_gateway API extension
doc/instances: Adds ipvlan ipv4.gateway and ipv6.gateway docs
lxd/device/nic/routed: Sets accept_ra=0 on host interface
lxc: Fix for current cobra
lxd/device/nic_routed: Don’t fail on missing IPv6
lxd/device/nic_routed: Set rp_filter=1
forkexec: rework
forkexec: tweak
lxd/firewall/firewall/interface: Adds InstanceSetupRPFilter and InstanceClearRPFilter
lxd/firewall/drivers/drivers/xtables: Improves proxy NAT rule removal errors
lxd/firewall/drivers/drivers/xtables: Renames iptablesConfig to iptablesAdd
lxd/firewall/drivers/drivers/xtables: Implements reverse path filters
lxd/device/nic/routed: Applies firewall based reverse path filter for IPv4 and IPv6
lxd/storage/drivers/ceph: Re-create image snapshot
lxd/storage/drivers: Update comment on readonly snapshot
lxd/firewall/drivers/drivers/nftables: Implements reverse path filters
shared/instancewriter/instance/tar/writer: Adds ignoreGrowth arg to WriteFile
lxd/storage/drivers/generic/vfs: Sets ignoreGrowth arg true in WriteFile usage
lxd: Existing WriteFile usage updated to set ignoreGrowth to false
lxd/device/nic/bridged: Disables IPv6 on bridged host side interface
lxd/exec: Fix forwarding for VMs
lxd: Rename forwarding functions
i18n: Update translations from weblate
lxd/networks: Fix network leases list for instances using “network” option
lxd/instance/drivers/driver/qemu: Restart on failure
shared/idmap: Better root fallback
lxd/instance/drivers/driver/qemu: Fixes dependencies for lxd-agent
lxd-agent/main/agent: Better logging
shared/version/api: Add resources_usb_pci API extension
doc: Add resources_usb_pci
shared/api: Add USB and PCI resources
shared/usbid: Add USB vendor and devices
lxd/resources: Add USB resource
lxd/resources: Add PCI resource
test/suites/static_analysis: Skip shared/usbid/load_data.go

Try it for yourself

This new LXD release is already available for you to try on our demo service.

Downloads

The release tarballs can be found on our download page.

Binary builds are also available for:

Linux: snap install lxd
MacOS: brew install lxc
Windows: choco install lxc

stgraber · April 1, 2020, 12:15am

LXD 4.0 is available in the candidate channel currently and will be promoted to stable on Thursday. At that time, the 4.0 LTS channel will also appear and then be kept around for the next 5 years.

Enjoy!

cyphar · April 1, 2020, 11:39pm

Hurray! Packaged and pushed to openSUSE’s repos (already merged for the next Tumbleweed snapshot).

stgraber · April 2, 2020, 12:31am

Thanks!

stgraber · April 2, 2020, 4:39pm

4.0 is rolling out to stable now.

Yosu_Cadilla · April 5, 2020, 5:25pm

Fantastic!

michacassola · April 5, 2020, 8:49pm

@stgraber Do the virtual machines that per the docs use QEMU utilize the KVM kernel module? If not, will they eventually?

@Yosu_Cadilla Now I know what you use to backup Mautic.

stgraber · April 6, 2020, 1:50am

They do, yes.

Nick_Knutov · April 6, 2020, 12:00pm

Is it possible to set user-level disk quotes with zfs backed now?

stgraber · April 6, 2020, 2:02pm

I don’t believe so. Traditional user/group quotas isn’t something we have much experience with, so we’d need someone who understands how those are supposed to work on normal ZFS to at least give us some detailed pointers.

The fact that zfs allow and per-uid/per-gid zfs settings seem to be involved is a bit frightening as ZFS has close to no understanding of namespaces so may very easily get confused as to what user is actually allowed to do this.

Nick_Knutov · April 6, 2020, 6:03pm

I found I can set quota from node side, using actual user id:

zfs set userquota@101000=5G lxd/containers/test-image

Is there a way to communicate from container side to node and send some commands? I suppose I can write the daemon to set and get disk quotas.

Nick_Knutov · April 24, 2020, 12:33pm

I suppose I can impliment management of user-level quotas as contaner-side agent for example, but I need the way to communicate with host.

derekmahar · May 8, 2020, 7:04pm

Is it really possible to install LXC on Windows?

derekmahar · May 8, 2020, 7:06pm

PS C:\Windows\System32> choco info lxc
Chocolatey v0.10.15
lxc 4.0.1 [Approved]
 Title: LXD client | Published: 2020-04-22
 Package approved as a trusted package on Apr 21 2020 22:56:35.
 Package testing status: Passing on Apr 21 2020 22:52:32.
 Number of Downloads: 608 | Downloads for this version: 56
 Package url
 Chocolatey Package Source: https://github.com/lxc/lxd-pkg-chocolatey
 Package Checksum: 'vFKKgvzq0g0uGiWlUyg/OeHLTs9eJe053CIiAcn3uO+guUcu+qVEyTN+4aLPz38L3V7AZ3m1eIdl/eJhhSiWIw==' (SHA512)
 Tags: lxc lxd
 Software Site: https://linuxcontainers.org/lxd
 Software License: https://github.com/lxc/lxd/blob/master/COPYING
 Software Source: https://github.com/lxc/lxd
 Documentation: https://lxd.readthedocs.io/
 Issues: https://github.com/lxc/lxd/issues
 Summary: System container manager - client
 Description: Client for the LXD container manager
 Release Notes: https://linuxcontainers.org/lxd/news

I see that it’s just the LXD client.