Can't expand container storage

Hello, I am new to lxc /lxd and I would appreciate your help

I have configured some containers on my linux server, and would like to increase the space they are allocated.

i have read the other similar posts, but I don’t think i have the same configuration, i don’t use snap as far as i can see, and i can’t find the file /var/snap/lxd/common/lxd/disks/default.img

i saw a solution somewhere that i need to set the entire pool storage to 100GB (or GiB?), but i wanted to ask here first in case I should do something before hand (backup up the files, configuration etc…)

Some of the containers are running,( DATA 1 to 4, Network and VPN) the others are stopped.

$df -h
Filesystem                                      Size  Used Avail Use% Mounted on
udev                                             32G     0   32G   0% /dev
tmpfs                                           6,3G  1,6M  6,3G   1% /run
rpool/ROOT/ubuntu                               1,5G 1008M  519M  67% /
tmpfs                                            32G     0   32G   0% /dev/shm
tmpfs                                           5,0M     0  5,0M   0% /run/lock
tmpfs                                            32G     0   32G   0% /sys/fs/cgroup
/dev/sda8                                       511M  3,3M  508M   1% /boot/efi
rpool/home                                       46G   46G  519M  99% /home
rpool/home/root                                  77G   77G  519M 100% /root
rpool/var/cache                                 522M  3,4M  519M   1% /var/cache
rpool/var/log                                   848M  330M  519M  39% /var/log
rpool/var/spool                                 520M  896K  519M   1% /var/spool
rpool/var/tmp                                   519M  128K  519M   1% /var/tmp
lxd_data/lxd                                     14T  838G   13T   6% /var/lib/lxd
tmpfs                                           100K     0  100K   0% /var/lib/lxd/shmounts
tmpfs                                           100K     0  100K   0% /var/lib/lxd/devlxd
lxd_data/eholink/containers/Network-fl11-tw1     94G  540M   94G   1% /var/lib/lxd/storage-pools/default/containers/Network-fl11-tw1
lxd_data/eholink/containers/VPN-fl11-tw1         94G  527M   93G   1% /var/lib/lxd/storage-pools/default/containers/VPN-fl11-tw1
lxd_data/eholink/containers/DATA-LVL3-fl11-tw1   93G  2,4G   90G   3% /var/lib/lxd/storage-pools/default/containers/DATA-LVL3-fl11-tw1
lxd_data/eholink/containers/DATA-LVL2-fl11-tw1   93G   61G   32G  66% /var/lib/lxd/storage-pools/default/containers/DATA-LVL2-fl11-tw1
lxd_data/eholink/containers/DATA-LVL4-fl11-tw1   93G   80G   13G  87% /var/lib/lxd/storage-pools/default/containers/DATA-LVL4-fl11-tw1
lxd_data/eholink/containers/DATA-LVL1-fl11-tw1   92G   21G   71G  23% /var/lib/lxd/storage-pools/default/containers/DATA-LVL1-fl11-tw1
tmpfs                                           6,3G     0  6,3G   0% /run/user/1006
$ zpool status -v
  pool: lxd_data
 state: ONLINE
status: One or more devices could not be used because the label is missing or
	invalid.  Sufficient replicas exist for the pool to continue
	functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
  scan: scrub repaired 0B in 0h59m with 0 errors on Sun May 12 01:23:47 2024
config:

	NAME         STATE     READ WRITE CKSUM
	lxd_data     ONLINE       0     0     0
	  drbd_data  ONLINE       0     0     0
	cache
	  sda2       UNAVAIL      0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
  scan: scrub repaired 0B in 0h11m with 0 errors on Sun May 12 00:35:56 2024
config:

	NAME                                            STATE     READ WRITE CKSUM
	rpool                                           ONLINE       0     0     0
	  scsi-3600062b204fa7a402711ee28ac3ee230-part1  ONLINE       0     0     0

errors: No known data errors

$ lsblk
NAME         MAJ:MIN  RM   SIZE RO TYPE MOUNTPOINT
sda            8:0     0  14,6T  0 disk 
├─sda1         8:1     0   128G  0 part 
├─sda2         8:2     0  14,4T  0 part 
│ └─drbd1032 147:1032  0  14,4T  0 disk 
├─sda8         8:8     0   512M  0 part /boot/efi
└─sda9         8:9     0    18M  0 part 
sdb            8:16    0 447,1G  0 disk 
└─sdb2         8:18    0 447,1G  0 part 

I also noticed that $lxc storage list shows that the pool default is used by 24 , but $lxc storage volume list default shows 23 entries,i guess because the profiles aren’t counted ?

$ lxc storage list
+---------+-------------+--------+------------------+---------+
|  NAME   | DESCRIPTION | DRIVER |      SOURCE      | USED BY |
+---------+-------------+--------+------------------+---------+
| default |             | zfs    | lxd_data/eholink | 24      |
+---------+-------------+--------+------------------+---------+

$ lxc storage info default
info:
  description: ""
  driver: zfs
  name: default
  space used: 185.15GB
  total space: 14.23TB
used by:
  containers:
  - DATA-LVL1-fl11-tw1
  - DATA-LVL1-fl11-tw1
  - DATA-LVL1-fl11-tw1
  - DATA-LVL2-fl11-tw1
  - DATA-LVL2-fl11-tw1
  - DATA-LVL3-fl11-tw1
  - DATA-LVL3-fl11-tw1
  - DATA-LVL3-fl11-tw1
  - DATA-LVL4-fl11-tw1
  - DATA-LVL4-fl11-tw1
  - DATA-LVL4-fl11-tw1
  - DATA-LVL5-fl11-tw1
  - DATA-LVL5-fl11-tw1
  - Network-fl11-tw1
  - VPN-fl11-tw1
  - VPN-fl11-tw1
  - data2restored
  - image-backup-DATA-LVL1-fl11-tw1-novembre-06-2020-11-15-01
  - image-backup-DATA-LVL2-fl11-tw1-novembre-06-2020-11-15-01
  - image-backup-DATA-LVL3-fl11-tw1-novembre-06-2020-11-15-01
  - image-backup-DATA-LVL4-fl11-tw1-novembre-06-2020-11-15-01
  images:
  - 0e81f7497421e595c48ec00e99a55ec5786a0174026ff4e8aa2c7b81484e6a53
  - b902849785eb2e2c0221cec5f3b971c50aeee9eb94b652281506a037bf8de193
  profiles:
  - default
$ lxc storage volume list default
+----------------------+------------------------------------------------------------------+-------------+---------+
|         TYPE         |                               NAME                               | DESCRIPTION | USED BY |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | DATA-LVL1-fl11-tw1                                               |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | DATA-LVL2-fl11-tw1                                               |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | DATA-LVL3-fl11-tw1                                               |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | DATA-LVL4-fl11-tw1                                               |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | DATA-LVL5-fl11-tw1                                               |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | Network-fl11-tw1                                                 |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | VPN-fl11-tw1                                                     |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | data2restored                                                    |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | image-backup-DATA-LVL1-fl11-tw1-novembre-06-2020-11-15-01        |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | image-backup-DATA-LVL2-fl11-tw1-novembre-06-2020-11-15-01        |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | image-backup-DATA-LVL3-fl11-tw1-novembre-06-2020-11-15-01        |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container            | image-backup-DATA-LVL4-fl11-tw1-novembre-06-2020-11-15-01        |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container (snapshot) | DATA-LVL1-fl11-tw1/DATA-LVL1-fl11-tw1-snapshot                   |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container (snapshot) | DATA-LVL1-fl11-tw1/DATA-LVL1-fl11-tw1-snapshot2                  |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container (snapshot) | DATA-LVL2-fl11-tw1/DATA-LVL2-fl11-tw1-snapshot                   |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container (snapshot) | DATA-LVL3-fl11-tw1/DATA-LVL3-fl11-tw1-snapshot                   |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container (snapshot) | DATA-LVL3-fl11-tw1/snap0                                         |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container (snapshot) | DATA-LVL4-fl11-tw1/DATA-LVL4-fl11-tw1-snapshot2                  |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container (snapshot) | DATA-LVL4-fl11-tw1/DATA-LVL4-fl11-tw1-snapshot3                  |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container (snapshot) | DATA-LVL5-fl11-tw1/DATA-LVL4-fl11-tw1-snapshot                   |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| container (snapshot) | VPN-fl11-tw1/VPN-fl11-tw1-snapshot                               |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| image                | 0e81f7497421e595c48ec00e99a55ec5786a0174026ff4e8aa2c7b81484e6a53 |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+
| image                | b902849785eb2e2c0221cec5f3b971c50aeee9eb94b652281506a037bf8de193 |             | 1       |
+----------------------+------------------------------------------------------------------+-------------+---------+

I usually tinker with the container DATA-LVL4-fl11-tw1 before modifying anything on the other DATA containers
I tried:( i kept changing GB and GiB , size=value and size value but no change)

$ lxc config device override DATA-LVL4-fl11-tw1 root size=100GB
Device root overridden for DATA-LVL4-fl11-tw1

$ lxc storage volume set default DATA-LVL4-fl11-tw1 size 100GiB
Error: not found

$ lxc config show DATA-LVL4-fl11-tw1 
--truncated--
devices:
  root:
    path: /
    pool: default
    size: 100GB
    type: disk
--truncated--

the other DATA containers (don’t) have the root subsection in the devices section.
i just rechecked and it was added to all of the DATA containers even DATA 5, but not Network-fl11-tw1 and the others.

I also tried to restart lxd and restart my container , the output of df -h hasn’t changed.

I figured it out1 it was a quota issue:

$ zfs get all lxd_data/eholink/containers/DATA-LVL2-fl11-tw1 
NAME                                            PROPERTY              VALUE                                                                                     SOURCE
lxd_data/eholink/containers/DATA-LVL3-fl11-tw1  quota                 93,1G                                                                                     local
lxd_data/eholink/containers/DATA-LVL4-fl11-tw1  quota                 93,1G                                                                                     local

after executing

$ lxc config device unset DATA-LVL4-fl11-tw1 root size
$ zfs get quota lxd_data/eholink/containers/DATA-LVL4-fl11-tw1 

lxd_data/eholink/containers/DATA-LVL4-fl11-tw1  quota                 none                                                                                      local
$ lxc config show DATA-LVL4-fl11-tw1 
devices:
  root:
    path: /
    pool: default
    size: ""
    type: disk

the root device was removed from the other containers though.

and now i just use sudo zfs set quota=1TB full-container-name #as shown in the output of $ df -h for example