Linux user quotas from inside LXD container / guest

lxd
lxcfs

#1

I have been looking all around. Here is what I ended with. I hope someone would be able to confirm and/or correct my findings.

The filesystem supported features are in the doc

So, the only way to have quotas support from inside the container is using BTRFS, which I personally don’t wanna to use. And those would be BTRFS quotas, not standard linux quotas.

On ZFS you can only set quotas from the host, not from the guest, because ZFS support in LXD doesn’t check the “Storage driver usable inside a container” box. There an issue about it. Regarding users and groups quotas to be set from the host. I haven’t tested it and wonder how it is supported and if it needs UID/GID mapping from guest to host. No sure about that though.

On ZFS, ZVOLs do exist and can be formatted as ext4, they are seen as a standard block device from the guest side, and thus can be used for standard linux quotas. But those cannot be used for rootfs, see the issue on GitHub. You can still mount it in areas that have users data to check against quotas (eg. /home, /var).

Then there is libvirt virtualization over ZFS ZVOLs, which is slower but just works as expected regarding linux quotas.

As an advice, always make your ZVOLs at minimum needed size as it is much easier to expand than to shrink, the later requiring downtime.


How to set secondary volume mount options?
(Stéphane Graber) #2

So there are two things being mixed up here:

  • Container quotas
  • User quotas

The former refers to being able to set the size of a particular dataset/subvolume which can then be assigned to a container. On the host, unless you’re using the directory backend, we have support for this on all storage backends.

Then there is using those type of quotas INSIDE a container, typically for use by nested containers. That requires support by the filesystem for unprivileged users to be able to do that. Btrfs allows some amount of that though its qgroup quotas aren’t all that useful for security (easy to bypass). Everything else effectively doesn’t have this capability so nested containers cannot be size restricted.

Then you have user/group quotas. Those are handled entirely separately with most filesystems supporting them. My understanding of those is pretty limited though the fact that they are based on kuid/kgid is a bit of a problem unless you’re running containers to have non-overlapping uid/gid allocation. That’s because even if the filesystem allows for root in the user namespace to set the quotas (which may be blocked), it may allow for root in one container to see and alter the quotas of users in another container. That’s particularly an issue if shared filesystems are involved.

There is an open issue for supporting zvols as rootfs for LXD though so far nobody really committed the time to implement and maintain this as a storage driver.
It would also come with the usual set of downsides of block based storage backends, difficult snapshots, slow network transfers, limited ability to live resize, …

I suspect the first thing that’d be worth figuring out here is whether user/group quotas are configurable as root inside an unprivileged container at all, if not, then a lot of this is kinda moot until this changes. If it’s possible, then we’d need to consider the security implications and then see what needs to happen on a per storage backend basis (and specifically track this in the storage documentation too).


#3

Well as it can be seen here it is not possible to enable standard linux users quota inside a container even using an ext4 formatted ZVOL. The container would need to do the mount itself I guess.

My use cases are not related to nested container. I have two:

  • my hosting panel uses quotas on / for websites user’s quota management
  • I wanted to build a multi user ssh backup container with quotas for each local user

#4

Is rootfs mounted by the container itself ? If so ZVOL rootfs support would be the solution, with the drawbacks of block based storage backends, indeed.