Unable to set a disk size for / using btrfs

DjoDjo · August 10, 2017, 9:49pm

Hello,

I probably misconfigured something …
I want to set a disk size for the root (/) device.
But in the CT, I get the full host device size, /dev/md3, instead of a 100GB device :
-> lxc exec test – df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/md3 407G 298M 405G 1% /

Here is the default profile:
-> lxc profile show default
config: {}
description: Default LXD profile
devices:
root:
path: /
pool: default
size: 100GB
type: disk
name: default
used_by:

/1.0/containers/test

-> mount | grep btrfs
/dev/md3 on /btrfs type btrfs (rw,relatime,ssd,space_cache,subvolid=5,subvol=/)

Any help ?

Thank you !

stgraber · August 10, 2017, 10:02pm

Unlike ZFS, btrfs doesn’t render its quotas in the “df” output.

So your quota is applied to your container and btrfs should prevent you from exceeding it, but you won’t see it in the df output.

Also note that a user in the container can bypass btrfs quotas due to the unfortunate way qgroups work…

github.com

lxc/lxd/blob/master/doc/storage.md#btrfs

# Storage configuration
LXD supports creating and managing storage pools and storage volumes.
General keys are top-level. Driver specific keys are namespaced by driver name.
Volume keys apply to any volume created in the pool unless the value is
overridden on a per-volume basis.

## Storage pool configuration
Key                             | Type      | Condition                         | Default                    | API Extension                      | Description
:--                             | :---      | :--------                         | :------                    | :------------                      | :----------
size                            | string    | appropriate driver and source     | 0                          | storage                            | Size of the storage pool in bytes (suffixes supported). (Currently valid for loop based pools and zfs.)
source                          | string    | -                                 | -                          | storage                            | Path to block device or loop file or filesystem entry
btrfs.mount\_options            | string    | btrfs driver                      | user\_subvol\_rm\_allowed  | storage\_btrfs\_mount\_options     | Mount options for block devices
ceph.cluster\_name              | string    | ceph driver                       | ceph                       | storage\_driver\_ceph              | Name of the ceph cluster in which to create new storage pools.
ceph.osd.force\_reuse           | bool      | ceph driver                       | false                      | storage\_ceph\_force\_osd\_reuse   | Force using an osd storage pool that is already in use by another LXD instance.
ceph.osd.pg\_num                | string    | ceph driver                       | 32                         | storage\_driver\_ceph              | Number of placement groups for the osd storage pool.
ceph.osd.pool\_name             | string    | ceph driver                       | name of the pool           | storage\_driver\_ceph              | Name of the osd storage pool.
ceph.rbd.clone\_copy            | string    | ceph driver                       | true                       | storage\_driver\_ceph              | Whether to use RBD lightweight clones rather than full dataset copies.
ceph.user.name                  | string    | ceph driver                       | admin                      | storage\_ceph\_user\_name          | The ceph user to use when creating storage pools and volumes.
lvm.thinpool\_name              | string    | lvm driver                        | LXDThinPool                | storage                            | Thin pool where images and containers are created.
lvm.use\_thinpool               | bool      | lvm driver                        | true                       | storage\_lvm\_use\_thinpool        | Whether the storage pool uses a thinpool for logical volumes.

This file has been truncated. show original

DjoDjo · August 10, 2017, 10:16pm

Ohhhh ok, sorry, I remember I read that already now…

Can we see the disk usage percent of the subvolume from the host ? (not inside the CT)

Thank you Stéphane !

stgraber · August 10, 2017, 10:26pm

root@lxd-demo01:~# btrfs qgroup show -reF /var/lib/lxd/storage-pools/default/containers/tryit-willing/
qgroupid         rfer         excl     max_rfer     max_excl 
--------         ----         ----     --------     -------- 
0/517       675.55MiB     40.55MiB         none     10.00GiB

Not exactly the most user friendly command, but it does give you numbers

DjoDjo · August 10, 2017, 10:33pm

Ok excellent

Sorry, one more question:
I would like to add a second subvolume on the CT, so I have a “/data” mount point with a different disk space.

But:
-> lxc profile device add default data disk path=/data pool=default size=100GB source=/btrfs
error: Only the root disk may have a size quota.

Am I doing something wrong ? Or it is not possible ?

DjoDjo · August 11, 2017, 4:14pm

I found that a solution can be creating a btrfs subvolume manually, applying a quota on it, and then adding the device to the CT.

For those we are interested:
btrfs subvolume create /btrfs/data
btrfs qgroup limit 42G /btrfs/data
lxc config device add test default disk source=/btrfs/data path=/data

stgraber · August 11, 2017, 4:38pm

With LXD 2.16 you should also be able to use “lxc storage volume create POOL-NAME VOLUME-NAME size=42GB” and then use “lxc storage volume attach POOL-NAME VOLUME-NAME test /data” to attach it to your container.

DjoDjo · August 11, 2017, 5:31pm

Oh yes it looks better like this

Thank you again