Weekly status #265

[tomp (Thomas Parrott)]

Weekly status for the week of the 5th September to the 11th September.

Introduction

This past week has seen two of our LXD roadmap items merged; local object storage support using MinIO and restricted.network.access feature in restricted projects. There were also performance improvements for macvlan NICs in VMs and some more improvements to DB query efficiency.

In addition @stgraber has continued his storage themed videos this week by looking at custom storage volumes in LXD:

Custom storage volumes in LXD

LXD

New features:

• Local S3 object storage support using MinIO: Following on from the recently added S3 object storage support using Ceph Object Gateway we have now added support for local S3 object storage by leveraging MinIO. For more detail please see Linux Containers - LXD - Has been moved to Canonical and [LXD] Object storage (S3 API).
• Restricted networks access for projects: A new setting has been added for restricted projects (those that have restricted=true set) called restricted.networks.access. This new setting is a comma-delimited list of network names from the default project that are allowed to be referenced by LXD entities belonging to the restricted project. This setting also controls which networks can been seen in the LXD API when accessed using a restricted project’s context. Additionally the existing restricted.devices.nic setting is now also used to control which networks can be accessed from within the restricted project. If set to block then no networks can be accessed. If set to managed (which is the default if not specified) then only managed networks can be accessed. If set to allow then all networks can be accessed. For managed and allow the list of networks that can be accessed is then filtered if restricted.networks.access is set.

Improvements:

• Added multi-queue and vhost-net offloading support to macvlan NICs for virtual machines.
• Added profile info to backup index file.
• Removes some functions that loaded storage volume record info inefficiently and replaced with more efficient function usage.
• DB generator improvements to make it more flexible and clearer to use. Also added context support.

Bug fixes:

• Fixed an issue to handle linux32 and linux64 arch personalities in lxc-to-lxd command.
• Updated AppArmor profile to allow read/write access to /sys/fs/bpf.
• Fixed an issue with extended attribute access when overlay does not filter response.
• Made dnsmasq start up failures for bridge networks be considered a network start failure. This will cause a warning to be logged in lxc warning list as well as enable automatic retrying to start the network.
• Fixed a panic in lxc alias add when no existing client config file exists.
• Updated nftables firewall driver to pass rule config via STDIN to the nft command rather than via command arguments so that we don’t hit the argument length limit when using larger rulesets.
• Fixed an issue that was causing the Cached property of an image record to be lost when initializing an image from an existing image in a different project.

Distrobuilder

Improvements:

• Added support for OpenWrt 22.03.

Dqlite (database)

Bug fixes:

• Fixed an issue that caused the Handover method of App, to continue looping over other servers after successfully transferring leadership to another server.
• Dqlite shell fixes.

YouTube videos

The LXD team is running a YouTube channel with live streams covering LXD releases and weekly videos on different aspects of LXD. You may want to give it a watch and/or subscribe for more content in the coming weeks.

https://www.youtube.com/lxd-videos

Contribute to LXD

Ever wanted to contribute to LXD but not sure where to start?
We’ve recently gone through some effort to properly tag issues suitable for new contributors on Github: Easy issues for new contributors

Upcoming events

• Nothing planned currently.

Ongoing projects

The list below is feature or refactoring work which will span several weeks/months and can’t be tied directly to a single Github issue or pull request.

• Stable release work for LXC, LXCFS and LXD
• [LXD] Stream lifecycle and log events to Loki
• Infiniband support for VMs
• Dqlite disk-only operation mode

Upstream changes

The items listed below are highlights of the work which happened upstream over the past week and which will be included in the next release.

LXD

• Storage: Object storage with MinIO
• Networks: Adds restricted.networks.access setting
• Instance: Load api.Project into instance common struct
• lxd-generate: Create filtered statements from pre-existing “objects” statement.
• Instance: Remove duplicate profiles argument from instance.Load()
• lxc-to-lxd: Handle linux32 and linux64 arch personalities
• VM: Adds support for multi-queue macvlan NICs in VMs
• doc/network/bgp: add some clarifications to the BGP docs
• doc/metrics: add a list of provided metrics
• Add profiles to backup data
• lxd-generate: Refactor stmt generation with more manual intervention
• doc/migration: Document lxc-to-lxd tool
• doc/storage/dir: add link to the YouTube video
• lxd/apparmor: allow rw access to /sys/fs/bpf
• GetAllXattr: handle overlay filtering
• Network: Make dnsmasq start failure a network start failure
• Migration cleanup
• lxc: Fix panic when doing lxc alias add without existing config.yaml
• doc/migration: clarify how to enable live migration for VMs
• DB: Removes inefficient storagePoolVolumeGetType and associated functions
• lxd/db/generate/db/mapping: Use ‘join’ tag to deduce column name
• gomod: Updates dependencies
• Storage: Use a sub-directory for minio on storage volume
• LXD: Add context support to subprocess package
• AppArmor: Use non-deferenced outputPath to derive profile name in archiveProfile
• Shared: Don’t modify the global logrus standard logger
• Shared: Update GetAllXattr implementation using github.com/pkg/xattr
• lxd: Fix calls to inst.Project().Name when used in fmt.Sprintf
• lxd-generate: Use QueryRowContext for Get<entity>ID
• Database: Use QueryContext in query.Scan and query.SelectObjects
• Database: Replace tx.Query with tx.QueryContext and tx.QueryRow with tx.QueryRowContext
• Firewall: Pass nftables rules via stdin rather than as command arguments
• gomod: go mod tidy
• Image: Maintain Cached property when creating an image in a project from an existing project
• Test: Adds LXD_SKIP_TESTS option
• Lxd-generate: Remove superfluous GetID error check

LXC

• Nothing to report this week

LXCFS

• Nothing to report this week

Distrobuilder

• sources/openwrt: Support OpenWrt 22.03
• Minor docs suggestions

Dqlite (RAFT library)

• Nothing to report this week

Dqlite (database)

• dqlite.h: Lineup version macros with release.
• test: Make sure tracing is enabled

Dqlite (Go bindings)

• app: Don’t transfer leadership more than once
• github: Run go test in CI
• Revert “github: Run go test in CI”
• Dqlite demo fixes
• Shell fixes

LXD Charm

• Nothing to report this week

Distribution work

This section is used to track the work done in downstream Linux distributions to ship the latest LXC, LXD and LXCFS as well as work to get various software to work properly inside containers.

Ubuntu

• Nothing to report this week

Snap

• Nothing to report this week

[Nick_Knutov]

Is there any examples of how to use restricted.networks.access?

I have container with routed network with public ipv4 and rootless docker inside. I have created only one network interface (eth0) during container creation, but docker inside creates multiple networks inside.

lxc list shows all network ineterfaces in table, which is definitly unwanted in this case. Does restricted.networks.access help to hide interfaces which is not managed by LXD in this case?

[tomp (Thomas Parrott)]

The restricted.networks.access setting controls which LXD host side networks are visible/usable in lxc network list and as parent networks for instance NIC devices.

It doesn’t control what network interfaces can be created inside the instances, and lxc list will show all of those.