Linux user quotas from inside LXD container / guest

LXD
,
1

I have been looking all around. Here is what I ended with. I hope someone would be able to confirm and/or correct my findings.

The filesystem supported features are in the doc

So, the only way to have quotas support from inside the container is using BTRFS, which I personally don’t wanna to use. And those would be BTRFS quotas, not standard linux quotas.

On ZFS you can only set quotas from the host, not from the guest, because ZFS support in LXD doesn’t check the “Storage driver usable inside a container” box. There an issue about it. Regarding users and groups quotas to be set from the host. I haven’t tested it and wonder how it is supported and if it needs UID/GID mapping from guest to host. No sure about that though.

On ZFS, ZVOLs do exist and can be formatted as ext4, they are seen as a standard block device from the guest side, and thus can be used for standard linux quotas. But those cannot be used for rootfs, see the issue on GitHub. You can still mount it in areas that have users data to check against quotas (eg. /home, /var).

Then there is libvirt virtualization over ZFS ZVOLs, which is slower but just works as expected regarding linux quotas.

As an advice, always make your ZVOLs at minimum needed size as it is much easier to expand than to shrink, the later requiring downtime.

2

So there are two things being mixed up here:

  • Container quotas
  • User quotas

The former refers to being able to set the size of a particular dataset/subvolume which can then be assigned to a container. On the host, unless you’re using the directory backend, we have support for this on all storage backends.

Then there is using those type of quotas INSIDE a container, typically for use by nested containers. That requires support by the filesystem for unprivileged users to be able to do that. Btrfs allows some amount of that though its qgroup quotas aren’t all that useful for security (easy to bypass). Everything else effectively doesn’t have this capability so nested containers cannot be size restricted.

Then you have user/group quotas. Those are handled entirely separately with most filesystems supporting them. My understanding of those is pretty limited though the fact that they are based on kuid/kgid is a bit of a problem unless you’re running containers to have non-overlapping uid/gid allocation. That’s because even if the filesystem allows for root in the user namespace to set the quotas (which may be blocked), it may allow for root in one container to see and alter the quotas of users in another container. That’s particularly an issue if shared filesystems are involved.

There is an open issue for supporting zvols as rootfs for LXD though so far nobody really committed the time to implement and maintain this as a storage driver.
It would also come with the usual set of downsides of block based storage backends, difficult snapshots, slow network transfers, limited ability to live resize, …

I suspect the first thing that’d be worth figuring out here is whether user/group quotas are configurable as root inside an unprivileged container at all, if not, then a lot of this is kinda moot until this changes. If it’s possible, then we’d need to consider the security implications and then see what needs to happen on a per storage backend basis (and specifically track this in the storage documentation too).

1 Like
3

Well as it can be seen here it is not possible to enable standard linux users quota inside a container even using an ext4 formatted ZVOL. The container would need to do the mount itself I guess.

My use cases are not related to nested container. I have two:

  • my hosting panel uses quotas on / for websites user’s quota management
  • I wanted to build a multi user ssh backup container with quotas for each local user
4

Is rootfs mounted by the container itself ? If so ZVOL rootfs support would be the solution, with the drawbacks of block based storage backends, indeed.

5

@stgraber I have seen progress, but I am not sure. Is it now possible to attach an ext4 formatted ZVOL to an LXD instance ? So that the instance sees it as an ext4 FS and gets able to manage linux quotas on it ?
Or, can a ZFS ZVOL be attached to an instance as a block device, so the instance can format it in the FS choosen ?

6

You can create a custom volume on a zfs pool that has zfs.block_mode=true, e.g.

lxc storage volume create zfs vol1 \
    size=1GiB \
    zfs.block_mode=true \
    block.filesystem=ext4

lxc config device add <instance> mydisk disk \
    pool=zfs \
    source=vol1 \
    path=/mnt

This will create a ZVOL in the zfs pool, format it with ext4 and then mount that into the instance.

This will then present as an ext4 volume inside the instance.

You can also do this for the root disk, but currently it must be enabled at the pool level and only takes effect for new instances and custom volumes created on that pool, like so:

lxc storage set zfs \
    volume.zfs.block_mode=true \
    volume.block.filesystem=ext4

lxc init images:ubuntu/jammy c1 -s zfs

We do plan on adding support for specifying pool driver config options for instances, rather than having to set them on the pool itself, so that only specific instances use specific settings.

1 Like
7

Awesome, how can I customize the mount options to add usrquota,grpquota ?

# lxc config show isp
devices:
  mydisk:
    path: /data
    pool: default
    raw.mount.options: usrquota,grpquota
    required: "true"
    source: isp-data
    type: disk

» lxc exec isp -- mount | grep data
/dev/zvol/lpool/lxd/custom/default_isp-data on /data type ext4 (rw,relatime,discard,stripe=2)
8

Should be block.mount_options on the storage volume.

https://linuxcontainers.org/lxd/docs/latest/reference/storage_zfs/#storage-volume-configuration