LXD 4.0 LTS has been released

Introduction

The LXD team is very excited to announce the release of LXD 4.0 LTS!

This is the 3rd LTS release for LXD and a very busy and exciting one!
The changelog below is split so that both users of LXD 3.23 and LXD 3.0 can see what we have in store for them.

As with all our other LTS releases, this one will be supported for 5 years (June 2025) and will receive a number of bugfix and security point releases over that time.

As for LXD 3.0, we’re hoping to release one last bugfix release as 3.0.5 in the near future before we enter security-only maintenance mode for its remaining 3 years.

Enjoy!

Breaking changes

Removal of --container-only, replaced by --instance-only

Our only CLI breaking changes with this release is the replacement of --container-only by --instance-only. Those following the feature releases will have had both supported for a few months now. With the 4.0 release, we’re removing the deprecated ones.

Highlights for 3.23 users

virtual machines: Support for backup (import/export)

It is now possible to use lxc export and lxc import with virtual machines.

A word of caution however. Virtual machines, unlike containers are only accessible as a large block device. This means that several GB of data will need to be read and compressed, no matter how much is actually used inside the VM.

This can lead to long export times and similarly long import times.

Doing so with --optimized on a backend like ZFS should considerably reduce the export time, assuming the backup is to be imported on a storage pool of the same type.

resources: PCI and USB devices in the resource API

The resources API (/1.0/resources) has been extended with a list of all PCI and USB devices on the system. This is of particular use when dealing with VFIO passthrough to virtual machines or passing through USB devices to containers.

Long example output
stgraber@castiana:~$ lxc query /1.0/resources | jq .pci
{
  "devices": [
    {
      "driver": "skl_uncore",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:00.0",
      "product": "Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers",
      "product_id": "5904",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "i915",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:02.0",
      "product": "HD Graphics 620",
      "product_id": "5916",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "",
      "driver_version": "",
      "numa_node": 0,
      "pci_address": "0000:00:08.0",
      "product": "Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model",
      "product_id": "1911",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "xhci_hcd",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:14.0",
      "product": "Sunrise Point-LP USB 3.0 xHCI Controller",
      "product_id": "9d2f",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "intel_pch_thermal",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:14.2",
      "product": "Sunrise Point-LP Thermal subsystem",
      "product_id": "9d31",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "mei_me",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:16.0",
      "product": "Sunrise Point-LP CSME HECI #1",
      "product_id": "9d3a",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1c.0",
      "product": "Sunrise Point-LP PCI Express Root Port #1",
      "product_id": "9d10",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1c.2",
      "product": "Sunrise Point-LP PCI Express Root Port #3",
      "product_id": "9d12",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1c.4",
      "product": "Sunrise Point-LP PCI Express Root Port #5",
      "product_id": "9d14",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1d.0",
      "product": "Sunrise Point-LP PCI Express Root Port #9",
      "product_id": "9d18",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "",
      "driver_version": "",
      "numa_node": 0,
      "pci_address": "0000:00:1f.0",
      "product": "Sunrise Point LPC Controller/eSPI Controller",
      "product_id": "9d4e",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "",
      "driver_version": "",
      "numa_node": 0,
      "pci_address": "0000:00:1f.2",
      "product": "Sunrise Point-LP PMC",
      "product_id": "9d21",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "snd_hda_intel",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1f.3",
      "product": "Sunrise Point-LP HD Audio",
      "product_id": "9d71",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "i801_smbus",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:00:1f.4",
      "product": "Sunrise Point-LP SMBus",
      "product_id": "9d23",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "e1000e",
      "driver_version": "3.2.6-k",
      "numa_node": 0,
      "pci_address": "0000:00:1f.6",
      "product": "Ethernet Connection (4) I219-LM",
      "product_id": "15d7",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "rtsx_pci",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:02:00.0",
      "product": "RTS525A PCI Express Card Reader",
      "product_id": "525a",
      "vendor": "Realtek Semiconductor Co., Ltd.",
      "vendor_id": "10ec"
    },
    {
      "driver": "iwlwifi",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:04:00.0",
      "product": "Wireless 8265 / 8275",
      "product_id": "24fd",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "nvme",
      "driver_version": "1.0",
      "numa_node": 0,
      "pci_address": "0000:05:00.0",
      "product": "SSD 600P Series",
      "product_id": "f1a5",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:06:00.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:07:00.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:07:01.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:07:02.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:07:04.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "thunderbolt",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:08:00.0",
      "product": "JHL6540 Thunderbolt 3 NHI (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d2",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:09:00.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0a:00.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0a:01.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0a:02.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "pcieport",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0a:04.0",
      "product": "JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]",
      "product_id": "15d3",
      "vendor": "Intel Corporation",
      "vendor_id": "8086"
    },
    {
      "driver": "ahci",
      "driver_version": "3.0",
      "numa_node": 0,
      "pci_address": "0000:0b:00.0",
      "product": "",
      "product_id": "0622",
      "vendor": "ASMedia Technology Inc.",
      "vendor_id": "1b21"
    },
    {
      "driver": "xhci_hcd",
      "driver_version": "5.4.0-18-generic",
      "numa_node": 0,
      "pci_address": "0000:0c:00.0",
      "product": "FL1100 USB 3.0 Host Controller",
      "product_id": "1100",
      "vendor": "Fresco Logic",
      "vendor_id": "1b73"
    },
    {
      "driver": "atlantic",
      "driver_version": "5.4.0-18-generic-kern",
      "numa_node": 0,
      "pci_address": "0000:0d:00.0",
      "product": "AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion]",
      "product_id": "87b1",
      "vendor": "Aquantia Corp.",
      "vendor_id": "1d6a"
    }
  ],
  "total": 32
}

stgraber@castiana:~$ lxc query /1.0/resources | jq .usb
{
  "devices": [
    {
      "bus_address": 1,
      "device_address": 4,
      "interfaces": [
        {
          "class": "Wireless",
          "class_id": 224,
          "driver": "btusb",
          "driver_version": "0.8",
          "number": 0,
          "subclass": "Radio Frequency",
          "subclass_id": 1
        },
        {
          "class": "Wireless",
          "class_id": 224,
          "driver": "btusb",
          "driver_version": "0.8",
          "number": 1,
          "subclass": "Radio Frequency",
          "subclass_id": 1
        }
      ],
      "product": "",
      "product_id": "0a2b",
      "speed": 12,
      "vendor": "Intel Corp.",
      "vendor_id": "8087"
    },
    {
      "bus_address": 1,
      "device_address": 3,
      "interfaces": [
        {
          "class": "Video",
          "class_id": 14,
          "driver": "uvcvideo",
          "driver_version": "1.1.1",
          "number": 0,
          "subclass": "Video Control",
          "subclass_id": 1
        },
        {
          "class": "Video",
          "class_id": 14,
          "driver": "uvcvideo",
          "driver_version": "1.1.1",
          "number": 1,
          "subclass": "Video Streaming",
          "subclass_id": 2
        }
      ],
      "product": "Integrated Camera",
      "product_id": "b5ce",
      "speed": 480,
      "vendor": "Chicony Electronics Co., Ltd",
      "vendor_id": "04f2"
    },
    {
      "bus_address": 3,
      "device_address": 2,
      "interfaces": [
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 0,
          "subclass": "Control Device",
          "subclass_id": 1
        },
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 1,
          "subclass": "Streaming",
          "subclass_id": 2
        },
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 2,
          "subclass": "Streaming",
          "subclass_id": 2
        },
        {
          "class": "Human Interface Device",
          "class_id": 3,
          "driver": "usbhid",
          "driver_version": "5.4.0-18-generic",
          "number": 3,
          "subclass": "",
          "subclass_id": 0
        }
      ],
      "product": "TX42C500",
      "product_id": "4933",
      "speed": 12,
      "vendor": "Realtek Semiconductor Corp.",
      "vendor_id": "0bda"
    },
    {
      "bus_address": 3,
      "device_address": 13,
      "interfaces": [
        {
          "class": "Video",
          "class_id": 14,
          "driver": "uvcvideo",
          "driver_version": "1.1.1",
          "number": 0,
          "subclass": "Video Control",
          "subclass_id": 1
        },
        {
          "class": "Video",
          "class_id": 14,
          "driver": "uvcvideo",
          "driver_version": "1.1.1",
          "number": 1,
          "subclass": "Video Streaming",
          "subclass_id": 2
        },
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 2,
          "subclass": "Control Device",
          "subclass_id": 1
        },
        {
          "class": "Audio",
          "class_id": 1,
          "driver": "snd-usb-audio",
          "driver_version": "5.4.0-18-generic",
          "number": 3,
          "subclass": "Streaming",
          "subclass_id": 2
        }
      ],
      "product": "HD Pro Webcam C920",
      "product_id": "082d",
      "speed": 480,
      "vendor": "Logitech, Inc.",
      "vendor_id": "046d"
    },
    {
      "bus_address": 3,
      "device_address": 16,
      "interfaces": [
        {
          "class": "Human Interface Device",
          "class_id": 3,
          "driver": "usbhid",
          "driver_version": "5.4.0-18-generic",
          "number": 0,
          "subclass": "",
          "subclass_id": 0
        },
        {
          "class": "Chip/SmartCard",
          "class_id": 11,
          "driver": "usbfs",
          "driver_version": "5.4.0-18-generic",
          "number": 1,
          "subclass": "",
          "subclass_id": 0
        }
      ],
      "product": "YubiKey FIDO+CCID",
      "product_id": "0406",
      "speed": 12,
      "vendor": "Yubico.com",
      "vendor_id": "1050"
    },
    {
      "bus_address": 3,
      "device_address": 17,
      "interfaces": [
        {
          "class": "Human Interface Device",
          "class_id": 3,
          "driver": "usbhid",
          "driver_version": "5.4.0-18-generic",
          "number": 0,
          "subclass": "Boot Interface Subclass",
          "subclass_id": 1
        },
        {
          "class": "Human Interface Device",
          "class_id": 3,
          "driver": "usbhid",
          "driver_version": "5.4.0-18-generic",
          "number": 1,
          "subclass": "Boot Interface Subclass",
          "subclass_id": 1
        }
      ],
      "product": "ThinkPad Compact USB Keyboard with TrackPoint",
      "product_id": "6047",
      "speed": 12,
      "vendor": "Lenovo",
      "vendor_id": "17ef"
    }
  ],
  "total": 6
}

network: Support for multiple ipvlan NIC devices

Multiple ipvlan devices can now be added to the same container provided that one of them has ipv4.gateway and/or ipv6.gateway set to none.

network: Support for host addresses on routed NIC

The host side address on routed nics can now be configured through the ipv4.host_address and ipv6.host_address properties.

clustering: Support for editing cluster roles

A new lxc cluster edit command allows for editing clustering roles.

It’s worth noting that there currently are no writable roles, but we expect to be adding some in the near future which will then be manageable through this API and command.

instances: Disk usage for custom volumes

Containers with custom storage volumes attached to them will now report those volume’s usage in the state API (and through lxc info):

stgraber@castiana:~$ lxc launch images:ubuntu/bionic c1
Creating c1
Starting c1

stgraber@castiana:~$ lxc storage volume create default vol1
Storage volume vol1 created
stgraber@castiana:~$ lxc storage volume create default vol2
Storage volume vol2 created

stgraber@castiana:~$ lxc storage volume attach default vol1 c1 vol1 /mnt/vol1
stgraber@castiana:~$ lxc storage volume attach default vol2 c1 vol2 /mnt/vol2

stgraber@castiana:~$ lxc info c1
Name: c1
Location: none
Remote: unix://
Architecture: x86_64
Created: 2020/04/01 00:00 UTC
Status: Running
Type: container
Profiles: default
Pid: 1439012
Ips:
  eth0:	inet	10.166.11.66	veth12c5ea18
  eth0:	inet6	fd42:4c81:5770:1eaf:216:3eff:fee2:43b6	veth12c5ea18
  eth0:	inet6	fe80::216:3eff:fee2:43b6	veth12c5ea18
  lo:	inet	127.0.0.1
  lo:	inet6	::1
Resources:
  Processes: 14
  Disk usage:
    root: 1.11MB
    vol1: 98.30kB
    vol2: 98.30kB
  CPU usage:
    CPU usage (in seconds): 0
  Memory usage:
    Memory (current): 46.94MB
  Network usage:
    eth0:
      Bytes received: 3.06kB
      Bytes sent: 2.93kB
      Packets received: 22
      Packets sent: 28
    lo:
      Bytes received: 0B
      Bytes sent: 0B
      Packets received: 0
      Packets sent: 0

instances: Disk usage for snapshots

The API now exposes the size of each individual snapshots.

stgraber@castiana:~$ lxc snapshot c1
stgraber@castiana:~$ lxc query /1.0/instances/c1/snapshots/snap0 | jq .size
61440

This will soon be displayed in lxc info once it’s gone through a redesign.

auth: Support for passwordless PKI mode

For those using LXD with a managed PKI, it is now possible to configure LXD to automatically trust any client certificate signed by the CA.

This is done with core.trust_ca_certificates.

To handle revocation, LXD also now accepts a CRL which should be placed alongside server.ca as server.crl.

Highlights for 3.0 users

In addition to the features and changes listed above, those who were using the LXD 3.0 LTS branch have the following “new” features to look forward to:

Virtual machines

LXD can now run both containers and virtual machines.

The experience and configuration works in much the same way though some device types and configuration options aren’t available for virtual machines yet.

Some operations are performed through an agent running in the virtual machine (lxc exec and lxc file). The agent comes pre-installed in the majority of our images.

To create a virtual machine rather than a container, simply pass --vm to lxc launch

VM images are now available for most commonly used Linux distributions with plans to add more in the future.

stgraber@castiana:~$ lxc launch images:centos/8 centos-8 --vm
Creating centos-8
Starting centos-8                             

stgraber@castiana:~$ lxc info centos-8
Name: centos-8
Location: none
Remote: unix://
Architecture: x86_64
Created: 2020/03/31 23:48 UTC
Status: Running
Type: virtual-machine
Profiles: default
Pid: 1426453
Ips:
  enp5s0:	inet	10.166.11.125
  enp5s0:	inet6	fd42:4c81:5770:1eaf:1c5b:d0a1:d892:5464
  enp5s0:	inet6	fe80::9bbf:7460:2ad0:6a9
  lo:	inet	127.0.0.1
  lo:	inet6	::1
Resources:
  Processes: 12
  Disk usage:
    root: 6.65MB
  CPU usage:
    CPU usage (in seconds): 5
  Memory usage:
    Memory (current): 123.94MB
    Memory (peak): 115.95MB
  Network usage:
    enp5s0:
      Bytes received: 2.55kB
      Bytes sent: 2.32kB
      Packets received: 21
      Packets sent: 20
    lo:
      Bytes received: 0B
      Bytes sent: 0B
      Packets received: 0
      Packets sent: 0

stgraber@castiana:~$ lxc exec centos-8 bash
[root@centos-8 ~]# cat /etc/redhat-release 
CentOS Linux release 8.1.1911 (Core) 
[root@centos-8 ~]# uname -a
Linux centos-8 4.18.0-147.5.1.el8_1.centos.plus.x86_64 #1 SMP Thu Feb 6 10:31:58 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@centos-8 ~]# 

Projects

Projects are a way to segment your LXD server.
Each project can contain its own set of instances, images, profiles and storage volumes.

Those various features can be enabled/disabled on a per-project basis. If disable, the project inherit from the default project.

On top of this, there is support for both restrictions (disabling particular device types, privileged containers, …) and limits (limiting the amount of CPU, memory and instance count).

Instances

  • System call interception on containers
    • Allows for limited mknod in containers
    • Allows for limited setxattr in containers
    • Can be used to allow mounting of privileged filesystems
    • Can be used to redirect some filesystem mounts to FUSE
  • Addition of a backup/restore feature (lxc export and lxc import)
  • Copy/move instances between storage pools
  • Refresh of an instance copy (local or remote) with lxc copy --refresh
  • Protection against accidental deletion and shift (security.protection.delete and security.protection.shift)
  • shiftfs is now supported and used when available (replaces traditional shifting)
  • Automated snapshots and expiration
  • New unix-hotplug device type (similar to unix-char and unix-block)
  • usb device improvements:
    • The add/remove uevent is now forwarded to the container
    • It is possible to pass all USB devices
  • proxy device improvements:
    • Privileged dropping options (security.uid and security.gid)
    • Socket ownership options (uid, gid, mode)
    • Support for HAProxy type header (proxy_protocol)
    • Fast proxying using NAT when available (nat)
    • Support for unix socket, udp and port ranges on udp and tcp
  • disk device improvements:
    • Direct attach of Ceph rbd/fs disks to containers
    • Custom mount options
    • shift property to translate uid/gid into container-readable ones
  • nic device improvements:
    • New ipvlan nictype
    • New routed nictype
    • ipv4.routes and ipv6.routes properties
    • network property to easily connect to LXD managed networks
    • Scurity filtering options
    • VLAN & MAC filtering on SR-IOV devices

Network

  • Configurable NAT source address (ipv4.nat.address and ipv6.nat.address)
  • DHCP leases API and lxc network list-leases command
  • Network state API and lxc network info command
  • Configurable MAC address on managed networks (bridge.hwaddr)
  • Control on firewall rule application order (ipv4.nat.order and ipv6.nat.order)

Storage

  • New internal storage layer rewritten from scratch
  • New cephfs storage backend
  • Backups and images can now be stored inside a storage pool
  • Custom storage volume snapshots (including scheduling & expiry)
  • LVM striping support
  • Separate metadata and data pools for Ceph
  • Quotas on dir backend through ext4/xfs “project quotas”
  • security.shifted property on custom storage volumes

Images

  • API for nested LXD to fetch images from the host (security.devlxd.images)
  • squashfs compression support for new images
  • Profiles can now be tied to images
  • Image expiry can now be changed

Clustering improvements

  • Support for standby database nodes
  • Configurable number of database & standby nodes
  • Mixed architecture clustering
  • Clustering roles
  • New simplified cluster join API
  • Separate addresses for client and cluster traffic
  • Automatic image replication

CLI

  • New columns in lxc list and lxc image list
  • New lxc alias command
  • Consistent list commands including --format support
  • All set commands now accept multiple key=value
  • exec now accepts --uid, --gid and --cwd
  • Config overrides on lxc copy and lxc move
  • More commands now support the --target option for clustering

Future proofing

  • Support for nftables as an alternative to xtables
  • Support for limits through Cgroup2

API

  • Support for RBAC (Role Based Access Control) through Canonical RBAC
  • Default TLS key is now EC384
  • New /1.0/instances endpoint replacing /1.0/containers
  • Addition of server-side collection filtering on /1.0/instances and /1.0/images
  • Much more comprehensive resources API at /1.0/resources
  • Kernel features are now exposed in /1.0
  • LXC features are now exposed in /1.0
  • Built-in debug server (pprof) configurable through core.debug_address
  • Additional bulk-query (recursion) options for high demand endpoints
  • Events and Operations in a clustered environment now have a Location field

Complete changelog

Here is a complete list of all changes in this release:

Full commit list
  • shared/version/api: Add trust_ca_certificates
  • doc: Add core.trust_ca_certificates
  • lxd/cluster/config: Add core.trust_ca_certificates
  • *: Add parameters to CheckTrustState
  • shared/cert: Add CRL to CertInfo
  • lxd/util/http: Check CRL for revoked clients
  • test: Extend PKI test
  • lxd/etag: Quote generated etag values
  • lxd/apparmor: Apparently the order matters
  • shared/version/api: Add snapshot_disk_usage API extension
  • doc: Add snapshot_disk_usage
  • lxd/storage/drivers/btrfs: Fix quota
  • lxd/backup: Removes Privileged field from backup.Info struct
  • lxd/backup: Adds new fields in index.yaml
  • lxd/instances/post: bInfo.OptimizedStorage pointer usage
  • lxd/storage/backend/lxd: CreateInstanceFromBackup OptimizedStorage pointer usage
  • lxd/backup: Updates backupWriteIndex index.yaml fields
  • lxd/backup: Removes Project field from index.yaml
  • test/suites/storage: Add btrfs quota tests
  • shared/api: Add size to InstanceSnapshot
  • lxd/instance/drivers: Get snapshot usage
  • lxd/storage/drivers/btrfs: Don’t destroy qgroups
  • lxd/storage/drivers: Moves functions from generic.go to generic_vfs.go
  • lxd/storage/drivers: Generic VFS function usage after move &rename
  • lxd/instance/drivers: Add custom volumes to disk state
  • lxd/instance/drivers: Fix lxd-agent running order
  • lxc: Deprecate --container-only
  • i18n: Update translation templates
  • tests: Move away from container-only
  • lxc: Drop flagContainerOnly
  • lxd/storage/zfs: Fix deleted VM images restoration
  • lxc/storage/drivers/driver/btrfs/volumes: CreateVolumeFromBackup to use tar reader for optimized volume restore
  • lxc/storage/drivers/driver/zfs/volumes: CreateVolumeFromBackup to use tar reader for optimized volume restore
  • shared/archive: Adds CompressedTarReader function
  • lxd/backup/backup: shared.CompressedTarReader usage
  • test/suites/static/analysis: Reinstates checks for shared/instancewriter
  • lxd/instance/post: InstanceID usage
  • lxd/db/containers: Renames ContainerID to InstanceID
  • lxd/instances/post: Logging in createFromBackup
  • lxd/instances/post: Logging message change from container to instance
  • lxd/instances/post: Switches to revert package in createFromBackup
  • lxd: Merges instanceCreateFromBackup into createFromBackup
  • lxd/storage/drivers/utils: Adds blockDevSizeBytes function
  • lxd/storage/drivers/driver/ceph/volumes: Updates SetVolumeQuota to use blockDevSizeBytes
  • shared/instancewriter/instance/file/info: Adds FileInfo for os.FileInfo implementation
  • shared/instancewriter/instance/tar/writer: Adds WriteFileFromReader function
  • lxd/backup: Switches index.yaml file generation to use WriteFileFromReader in backupCreate
  • lxd/api/internal: d.cluster.InstanceID usage
  • lxd/storage/backend/lxd: Better error msg context in CreateInstanceFromBackup
  • lxd/backup: Removes volume type restriction in backupCreate
  • lxd/storage/drivers/generic/vfs: Adds VM support to genericVFSBackupVolume
  • lxd/storage/drivers: Uses sourcePath logging for consistency in BackupVolume
  • lxd/storage/drivers/driver/zfs/volumes: Adds optimised VM backup to BackupVolume
  • lxd/storage/drivers/driver/btrfs/volumes: Adds optimised VM backup to BackupVolume
  • lxd/storage/backend/lxd: Adds volume type logic for VMs to CreateInstanceFromBackup
  • lxd/api/internal: makes internalImport VM aware
  • lxd/storage/drivers/generic/vfs: Adds VM support to genericVFSBackupUnpack
  • lxd/storage/drivers/driver/zfs/volumes: MountVolume comment improvements
  • lxd/storage/drivers/driver/zfs/volumes: UnmountVolume improvements
  • lxd/storage/drivers/driver/zfs/volumes: Adds VM support to generic mode in MigrateVolume
  • lxd/storage/drivers/driver/zfs/volumes: Adds VM support to MountVolumeSnapshot
  • lxd/storage/drivers/driver/zfs/volumes: Adds VM support to UnmountVolumeSnapshot
  • lxd/storage/drivers/driver/zfs/volumes: Adds support for VM optimized backup restore
  • lxd/storage/drivers: Adds existing volume check to optimized backup restore
  • lxd/storage/drivers/driver/btrfs/volumes: Adds support for VM optimized backup restore
  • lxd/storage/backend/lxd: Updates CheckInstanceBackupFileSnapshots to be VM aware
  • lxd/storage/backend/lxd/patches: Ignores snapshots when retrieving list of custom volumes to be renamed
  • lxd/containers: Emit lifecycle event on user shutdown
  • lxd/storage/drivers: Adds OptimizedBackups driver Info flag
  • lxd/backup: Ignore requests for optimized backups when pool driver doesn’t support it
  • lxd/instances/post: Ensure optimized backup imports only import into same storage driver pools
  • lxd/instance/exec: Adds protection against clients reconnecting after exec has started
  • doc: Fix escaping
  • lxd/cluster: Tweak errors
  • api: clustering_edit_roles
  • shared/api: Add ClusterMemberPut
  • lxd/cluster: Make ClusterMember editable
  • client: Add UpdateClusterMember
  • lxc/cluster: Add edit sub-command
  • i18n: Update translation templates
  • lxd/firewall/drivers/drivers/consts: Adds FilterIPv6All constant
  • cgroup/init: close controllers file
  • doc/networks: Add missing maas.subnet.ipv4/maas.subnet.ipv6
  • scripts/bash: Add maas.subnet.ipv4/maas.subnet/ipv6 to network
  • client: Fix bad description for UpdateClusterMember
  • lxd/device/nic/bridged: Allow security.ipv6_filtering to be used on networks without IPv6
  • lxd/firewall/drivers/drivers/xtables: Adds FilterIPv6All support
  • lxd/firewall: Dont use compact function arg definitions
  • lxd/firewall/drivers/drivers/nftables: Adds FilterIPv6All support
  • lxd/network/network/utils: Adds support for bridged NIC network property when rebuilding dnsmasq static config
  • lxd/network/network/utils: Comment consistency
  • lxd/device/nic/bridged: Allow security.ipv4_filtering to be used on networks without IPv4
  • lxd/firewall/drivers/drivers/consts: Adds FilterIPv4All constant
  • lxd/firewall/drivers/drivers/xtables: Adds Adds FilterIPv4All support
  • lxd/firewall/drivers/drivers/nftables: Adds FilterIPv4All support
  • test: Adds bridged NIC tests for total protocol filtering
  • lxd/device/nic: Adds ipv4.host_address and ipv6.host_address keys
  • lxd/device/nic/routed: Adds ability to specify host-side veth interface IP address
  • api: Adds container_nic_routed_host_address API extension
  • doc/instances: Updates routed nic doc with ipv4.host_address and ipv6.host_address keys
  • scripts/bash/lxd-client: Updates bash device keys for routed NIC
  • lxd/device/nic/ipvlan: Adds ipv4.gateway and ipv6.gateway support
  • api: Adds container_nic_ipvlan_gateway API extension
  • doc/instances: Adds ipvlan ipv4.gateway and ipv6.gateway docs
  • lxd/device/nic/routed: Sets accept_ra=0 on host interface
  • lxc: Fix for current cobra
  • lxd/device/nic_routed: Don’t fail on missing IPv6
  • lxd/device/nic_routed: Set rp_filter=1
  • forkexec: rework
  • forkexec: tweak
  • lxd/firewall/firewall/interface: Adds InstanceSetupRPFilter and InstanceClearRPFilter
  • lxd/firewall/drivers/drivers/xtables: Improves proxy NAT rule removal errors
  • lxd/firewall/drivers/drivers/xtables: Renames iptablesConfig to iptablesAdd
  • lxd/firewall/drivers/drivers/xtables: Implements reverse path filters
  • lxd/device/nic/routed: Applies firewall based reverse path filter for IPv4 and IPv6
  • lxd/storage/drivers/ceph: Re-create image snapshot
  • lxd/storage/drivers: Update comment on readonly snapshot
  • lxd/firewall/drivers/drivers/nftables: Implements reverse path filters
  • shared/instancewriter/instance/tar/writer: Adds ignoreGrowth arg to WriteFile
  • lxd/storage/drivers/generic/vfs: Sets ignoreGrowth arg true in WriteFile usage
  • lxd: Existing WriteFile usage updated to set ignoreGrowth to false
  • lxd/device/nic/bridged: Disables IPv6 on bridged host side interface
  • lxd/exec: Fix forwarding for VMs
  • lxd: Rename forwarding functions
  • i18n: Update translations from weblate
  • lxd/networks: Fix network leases list for instances using “network” option
  • lxd/instance/drivers/driver/qemu: Restart on failure
  • shared/idmap: Better root fallback
  • lxd/instance/drivers/driver/qemu: Fixes dependencies for lxd-agent
  • lxd-agent/main/agent: Better logging
  • shared/version/api: Add resources_usb_pci API extension
  • doc: Add resources_usb_pci
  • shared/api: Add USB and PCI resources
  • shared/usbid: Add USB vendor and devices
  • lxd/resources: Add USB resource
  • lxd/resources: Add PCI resource
  • test/suites/static_analysis: Skip shared/usbid/load_data.go

Try it for yourself

This new LXD release is already available for you to try on our demo service.

Downloads

The release tarballs can be found on our download page.

Binary builds are also available for:

  • Linux: snap install lxd
  • MacOS: brew install lxc
  • Windows: choco install lxc
11 Likes

LXD 4.0 is available in the candidate channel currently and will be promoted to stable on Thursday. At that time, the 4.0 LTS channel will also appear and then be kept around for the next 5 years.

Enjoy!

2 Likes

Hurray! Packaged and pushed to openSUSE’s repos (already merged for the next Tumbleweed snapshot).

1 Like

Thanks!

4.0 is rolling out to stable now.

1 Like

:heart_eyes: Fantastic!

1 Like

@stgraber Do the virtual machines that per the docs use QEMU utilize the KVM kernel module? If not, will they eventually?

@Yosu_Cadilla Now I know what you use to backup Mautic. :smiley: :smiley:

1 Like

They do, yes.

1 Like

Is it possible to set user-level disk quotes with zfs backed now?

I don’t believe so. Traditional user/group quotas isn’t something we have much experience with, so we’d need someone who understands how those are supposed to work on normal ZFS to at least give us some detailed pointers.

The fact that zfs allow and per-uid/per-gid zfs settings seem to be involved is a bit frightening as ZFS has close to no understanding of namespaces so may very easily get confused as to what user is actually allowed to do this.

I found I can set quota from node side, using actual user id:

zfs set userquota@101000=5G lxd/containers/test-image

Is there a way to communicate from container side to node and send some commands? I suppose I can write the daemon to set and get disk quotas.

I suppose I can impliment management of user-level quotas as contaner-side agent for example, but I need the way to communicate with host.

Is it really possible to install LXC on Windows?

PS C:\Windows\System32> choco info lxc
Chocolatey v0.10.15
lxc 4.0.1 [Approved]
 Title: LXD client | Published: 2020-04-22
 Package approved as a trusted package on Apr 21 2020 22:56:35.
 Package testing status: Passing on Apr 21 2020 22:52:32.
 Number of Downloads: 608 | Downloads for this version: 56
 Package url
 Chocolatey Package Source: https://github.com/lxc/lxd-pkg-chocolatey
 Package Checksum: 'vFKKgvzq0g0uGiWlUyg/OeHLTs9eJe053CIiAcn3uO+guUcu+qVEyTN+4aLPz38L3V7AZ3m1eIdl/eJhhSiWIw==' (SHA512)
 Tags: lxc lxd
 Software Site: https://linuxcontainers.org/lxd
 Software License: https://github.com/lxc/lxd/blob/master/COPYING
 Software Source: https://github.com/lxc/lxd
 Documentation: https://lxd.readthedocs.io/
 Issues: https://github.com/lxc/lxd/issues
 Summary: System container manager - client
 Description: Client for the LXD container manager
 Release Notes: https://linuxcontainers.org/lxd/news

I see that it’s just the LXD client.