Weekly status #185

[tomp (Thomas Parrott)]

Weekly status for the weeks of the 8th of February to the 14th of February.

Introduction

Now that @mbordere is maintaining the Dqlite, Go-dqlite and Raft repositories, we have added sections tracking changes to those repositories below.

LXD

The new features that have landed this past week are:

• VM stateful stop, start and snapshots using the the --stateful flag. As part of this we have also re-worked the lxd-agent systemd units to not require an active mount back to the host. This is so we can avoid having a virtiofs mount inside all VMs (which currently prevents stateful operations due to qemu limitations). Instead it copies the agent to a tmpfs mount and then unmounts the virtiofs mount at start up.
• Added Ceph storage pool setting ceph.rbd.features to enable access to additional Ceph volume creation settings which the underlying ceph tools and kernel modules expose (this varies based on the version of ceph being used).
• Added support for setting images.* and backups.* server config keys on a per-project basis.
• Initial Network ACL API endpoints and associated lxc network acl commands (although at this time none of the network drivers support ACLs so it cannot be used yet).

The following improvements and bugs have also been fixed:

• Worked around kernel IPVLAN cleanup issues by getting LXD to move the IPVLAN interface back to the host and deleting it on container stop.
• Worked around an issue that occurred on some SR-IOV NICs (broadcom in this case) that required the VF interfaces to have an explicit MAC set, now where the user has not provided one we set it to a random one at instance start.
• Added support for live updating NIC fields when using network property. This has not been possible up until this point due to an issue that considered the config keys inherited from the network as NIC changes which triggered a full device remove and re-add rather than a live update.
• Improved cluster heart-beating in LXD to not run concurrent heartbeats and adds additional logging to aid debugging of heartbeat issues.

LXC

• The lxc.net.[i].script.down hooks are now passed the FDs of the container namespaces (via environment variables, like the existing .stop hook) to allow hooks to access the container’s namespaces before the network is taken down.
• If the iw command cannot be found an error is now returned.
• Continued hardening of cgroups and fd-only codepaths.

Raft

• An issue with memory spikes on nodes where IO to persistent storage is consistently slow has been fixed.

Youtube channel

We’ve started a Youtube channel with live streams covering LXD releases and its use in the wider ecosystem.

You may want to give it a watch and/or subscribe for more content in the coming weeks.

LXD

Contribute to LXD

Ever wanted to contribute to LXD but not sure where to start?
We’ve recently gone through some effort to properly tag issues suitable for new contributors on Github: Easy issues for new contributors

Upcoming events

• Nothing to report this week

Ongoing projects

The list below is feature or refactoring work which will span several weeks/months and can’t be tied directly to a single Github issue or pull request.

• Distrobuilder Windows support
• Virtual networks in LXD
• Various kernel work
• Stable release work for LXC, LXCFS and LXD

Upstream changes

The items listed below are highlights of the work which happened upstream over the past week and which will be included in the next release.

LXD

• Network: ACL API, schema and CLI
• lxd/storage: Only delete volumes from non-pending pools
• Network: Don’t depend on kernel for ipvlan NIC cleanup
• Network: Use volatile VF MAC if VF has no automatic MAC set
• lxd/instance/qemu: Rework lxd-agent startup
• Implement VM stateful stop/start
• Network: Adds support for live updating NIC fields when using network property
• Add ceph.rbd.features
• Switch back to upstream go-qemu
• Add images and backups server config keys per project
• LXD: Misc tweaks
• Network: OVN ACL precursor changes
• Network: ACL usage tweaks to allow for reuse of finding instance NICs that use an ACL in the future
• utils: trim whitespace from block device UUID
• Fix VM quotas on btrfs/dir following new size.state
• Move the snapshot creation logic to the instance drivers
• Network: Misc changes to used by logic
• Add stateful snapshots for VMs
• Fix UsedBy on projects for remote volumes
• lxd/cluster: Guarantee single hearbeat loop
• lxd/cluster: Improve heartbeat logging
• doc/rest-api: Fix and clarify backup API
• Trivial bugfixes
• doc/rest-api: More fixes for backups
• Modernize certificates
• Projects remote cache expiry
• doc/backup: Mention subuid/subgid

LXC

• network: Add error message if iw couldn’t be found
• conf: expand fd-only setup codepaths
• criu: fixes
• cgroups: fixes
• tree-wide: fixes
• Improved mount api support checking & console setup hardening
• tree-wide: convert to strequal() and strnequal()
• hooks: rework exposing namespace information
• start: small fixes

LXCFS

• Nothing to report this week

Distrobuilder

• generators: Refresh lxd-agent systemd units

Dqlite (RAFT library)

• recv_append_entries: Always free the batch

Dqlite (database)

• Nothing to report this week

Dqlite (Go bindings)

• Nothing to report this week

Distribution work

This section is used to track the work done in downstream Linux distributions to ship the latest LXC, LXD and LXCFS as well as work to get various software to work properly inside containers.

Ubuntu

• LXCFS 4.0.7 was uploaded to Ubuntu 21.04
• LXC 4.0.6 was uploaded to Ubuntu 21.04

Snap

• lxc: Cherry-picked upstream bugfixes
• lxd: Cherry-picked upstream bugfixes