I deleted all containers and all images from the BTRFS server, when I run this command this what I am left with, not sure if this is relevant. Maybe problem is I am trying to set a quota less than what is being used in total, which its not handling well.
$ sudo btrfs fi show
Label: none uuid: 916f6d6a-c243-477c-b43b-20d12eb15518
Total devices 1 FS bytes used 656.00KiB
devid 1 size 53.50GiB used 2.01GiB path /dev/sda3
Looks a bit confused, it says you have 656KB used on the filesystem which sounds reasonable, but then the device shows 2.01GB used.
I have Ubuntu installed inside a Parallels virtual machine on the M1 preview, not sure if that or ARM64 compiled version is causing problem.
$ sudo btrfs fi df /btrfs
Data, single: total=1.00GiB, used=512.00KiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=1.01GiB, used=128.00KiB
GlobalReserve, single: total=3.25MiB, used=0.00B
sudo btrfs fi usage /btrfs/
Overall:
Device size: 53.50GiB
Device allocated: 2.01GiB
Device unallocated: 51.49GiB
Device missing: 0.00B
Used: 656.00KiB
Free (estimated): 52.49GiB (min: 52.49GiB)
Data ratio: 1.00
Metadata ratio: 1.00
Global reserve: 3.25MiB (used: 0.00B)
Data,single: Size:1.00GiB, Used:512.00KiB (0.05%)
/dev/sda3 1.00GiB
Metadata,single: Size:1.01GiB, Used:128.00KiB (0.01%)
/dev/sda3 1.01GiB
System,single: Size:4.00MiB, Used:16.00KiB (0.39%)
/dev/sda3 4.00MiB
Unallocated:
/dev/sda3 51.49GiB
$ sudo btrfs balance start /btrfs
WARNING:
Full balance without filters requested. This operation is very
intense and takes potentially very long. It is recommended to
use the balance filters to narrow down the scope of balance.
Use 'btrfs balance start --full-balance' option to skip this
warning. The operation will start in 10 seconds.
Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting balance without any filters.
$ sudo btrfs fi show
Label: none uuid: 916f6d6a-c243-477c-b43b-20d12eb15518
Total devices 1 FS bytes used 656.00KiB
devid 1 size 53.50GiB used 2.03GiB path /dev/sda3
Sorry yesterday it was a long day, I went in with a fresh approach today, and it appears the problem is that LXD is reporting the process as complete, but BTRFS is still doing things behind the scenes, going back to the container and starting it some time afterwards, it starts up without errors.
The same problem occurs when restoring from a backup, trying to start the container immediately after this causes errors, until BTRFS finishes what it is doing.
So just to confirm, create a container, and then patch it to set the quota, and then trying to start through the API causes these problems, you might not get this from the command line unless you paste in the commands, as it there a period of 20 seconds or so that BTRFS needs. If not BTRFS reports the quota as exceeding. Also tested with Alpine container and 1GB limit, no problem, But Ubuntu this fails.
It needs approx 20 seconds after being reported as complete before it can be booted without any errors. I don’t remember this problem existing when using BTRFS loopback device, I am only experiencing this now that I am using custom BTRFS partition of 50GB.