Resizing zfs filesystem in LXD container

I have a single lxd container:

$ lxc list
+--------------+---------+---------------------+-----------------+-----------+-----------+
|     NAME     |  STATE  |        IPV4         |      IPV6       |   TYPE    | SNAPSHOTS |
+--------------+---------+---------------------+-----------------+-----------+-----------+
| cbuild       | RUNNING | 10.190.6.155 (eth0) | fd42:etc (eth0) | CONTAINER | 0         |

It has a full disk:

$ lxc exec cbuild -- df -h -x squashfs | grep default
default/containers/cbuild   27G   27G  256K 100% /

I’d like to make that disk bigger.

$ lxc config device override cbuild root size=40GB
# as per https://discuss.linuxcontainers.org/t/resize-containers-volume/4853 among others
Device root overridden for cbuild

but that doesn’t affect the size of the disk inside the container. The container is running Ubuntu 20.04 (according to lsb_release inside the container) and I’m fairly sure that I created it from the stock ubuntu images rather than anything weird. Seemingly this image doesn’t auto-resize its disks? Or perhaps it does and I’m doing this wrong. Anyway, I don’t understand what I need to do here to give the container more disk space. Do I need to change the size of the underlying storage? (Is this a “zpool”, whatever that is?) Or do I need to resize the partition from inside the image and then expand the zfs filesystem in it, as if this were a real disk that I’d just put more space on? I’m terribly confused.

This is most likely because your zpool is full and there isn’t 40G to give that container.

zfs list -t all would show you the total size and free space on zpool at the top.

If you need to grow your zpool and it’s using a loop file, we have instructions here:
https://linuxcontainers.org/lxd/docs/master/storage#growing-a-loop-backed-zfs-pool

Aha, ok. I assume I run this on the host? That zfs command seems to be in both zfsutils-linux and zfs-fuse; which one should I be using?

$ zfs list -t all 
Command 'zfs' not found, but can be installed with:
sudo apt install zfsutils-linux  # version 0.8.3-1ubuntu12.1, or
sudo apt install zfs-fuse        # version 0.7.0-20

Normally I’d choose the most recent version, but it seems weird to me that they’re different packages, so I don’t want to be using the wrong thing.

zfsutils-linux is the right one, the fuse version is horrible :slight_smile:

Great stuff!

I have followed the instructions given and it does indeed seem to have worked; my container’s space is now larger! Thank you.

To confirm, this is what I did:

sudo apt install zfsutils-linux # to get the zfs command

zfs list -t all
# this lists loads of stuff, but "default" is at the top
# and shows the amount of space I had, which was 27GB

sudo ls /var/snap/lxd/common/lxd/disks/default.img
# this file exists, confirming that I have ZFS, it's using a loop file,
# it's called "default", and I'm using the snapped LXD

lxc stop cbuild # stop my LXD instance; unsure if this is required
sudo snap stop lxd # for non-snap this might be sudo service stop lxd?
# unsure if stopping lxd is required before fiddling with the ZFS stuff

sudo truncate -s +10G /var/lib/lxd/disks/default.img
# I wanted to add 10G of space because, as above, I told
# the container it had 40GB rather than the 30GB it had already

# for the below the docs say "lxd", not "default", as the pool name
# if you put the wrong name you get "cannot open 'lxd': no such pool"
sudo zpool set autoexpand=on default
sudo zpool online -e default /var/lib/lxd/disks/default.img
sudo zpool set autoexpand=off default

sudo snap start lxd # and restart lxd
lxc start cbuild # and my instance

and everything worked. So thank you for that!

5 Likes
sudo snap stop lxd # for non-snap this might be sudo service stop lxd?
# unsure if stopping lxd is required before fiddling with the ZFS stuff

sudo truncate -s +10G /var/lib/lxd/disks/default.img
# I wanted to add 10G of space because, as above, I told
# the container it had 40GB rather than the 30GB it had already
sudo snap start lxd # and restart lxd # <<<   start the lxd again if not he will tell u no such pool too
# for the below the docs say "lxd", not "default", as the pool name
# if you put the wrong name you get "cannot open 'lxd': no such pool"
sudo zpool set autoexpand=on default
sudo zpool online -e default /var/lib/lxd/disks/default.img
sudo zpool set autoexpand=off default

# sudo snap start lxd # and restart lxd

LXD now supports growing a loop backed storage pool using:

lxc storage set size=<new size>

See https://linuxcontainers.org/lxd/docs/master/howto/storage_pools/#resize-a-storage-pool