yes
root@esx:/home/downloads# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 63.3M 1 loop /snap/core20/1778
loop1 7:1 0 49.6M 1 loop /snap/snapd/17883
loop2 7:2 0 136.7M 1 loop /snap/lxd/24164
loop3 7:3 0 63.2M 1 loop /snap/core20/1738
loop4 7:4 0 136.7M 1 loop /snap/lxd/24155
sda 8:0 0 1.8T 0 disk
├─sda1 8:1 0 1.8T 0 part
└─sda9 8:9 0 8M 0 part
sdb 8:16 0 931.5G 0 disk
├─sdb1 8:17 0 931.5G 0 part
└─sdb9 8:25 0 8M 0 part
sdc 8:32 0 931.5G 0 disk
├─sdc1 8:33 0 931.5G 0 part
└─sdc9 8:41 0 8M 0 part
sdd 8:48 0 931.5G 0 disk
├─sdd1 8:49 0 931.5G 0 part
└─sdd9 8:57 0 8M 0 part
sde 8:64 0 931.5G 0 disk
├─sde1 8:65 0 931.5G 0 part
└─sde9 8:73 0 8M 0 part
sdf 8:80 0 931.5G 0 disk
├─sdf1 8:81 0 931.5G 0 part
└─sdf9 8:89 0 8M 0 part
zd0 230:0 0 46.6G 0 disk
zd16 230:16 0 50G 0 disk
zd32 230:32 0 326G 0 disk
zd48 230:48 0 350G 0 disk
nvme0n1 259:0 0 465.8G 0 disk
├─nvme0n1p1 259:2 0 512M 0 part /boot/efi
├─nvme0n1p2 259:3 0 464.3G 0 part /
└─nvme0n1p3 259:4 0 976M 0 part [SWAP]
nvme1n1 259:1 0 465.8G 0 disk
Could it be that running priviliged container is the problem?
Then you need not worry about separating the zpool by use type or slog. It’s a nice to have feature if you are running a lot of disks. Btrfs runs slower than ZFS, but use what you know. I switched from btrfs to ZFS due issues associated with lack of maturity. Once I learned that LXD could use ZFS, I scrapped proxmox as well.
After looking at your lsblk printout. Personally, for a SOHO setup, I’d definitely stick with ZFS over BTRFS. I’d slice off a part of that NVME and create a partition for a SLOG, but honestly SLOG is overkill for SOHO. However, you do get massive nerd-cred and bragging rights, lol.
Your system should have configured itself correctly. By chance did you create your zpool from an already formatted drive (i.e. ext4) and then attach LXD to it?
Yes,
The 4 drives where configured as btrfs raid10. So i deleted the btrfs part. migrated the vms to a external hdd.
created a new zpool with the 4 drives and migrated the machines back. I never did a shred on the disk. only with fdisk write away the partition table.
Did you create the zpool through zfs or lxd?
with zfs.
Because i wanted a raid10
will it help.
if i migrate all vm/containers again. shred all 4 ssd drives and then rebuild the pool with zfs?
First, let’s see if you have layered zfs on top of another filesystem. These commands should limit the output to the drives in question.
-
lsblk -f | grep sd -
df -Th | grep "^/dev/sd"
If that is the case, then after backing up like you did before, we can destroy/wipe/shred the disks for the pool. Then use LXD to make a ZFS pool on a single disk intended for the zpool.
root@esx:/home/downloads# lsblk -f | grep sd
sda btrfs backupdisk fdc464e0-8143-46d1-89c9-eae805e47408
├─sda1 zfs_member 5000 usb_backup 2822313664365506659
└─sda9
sdb btrfs lxd 23dfa8bd-b1f5-4854-af10-1497433fe7b4
├─sdb1 zfs_member 5000 lxdpool 7306657253134884339
└─sdb9
sdc btrfs lxd 23dfa8bd-b1f5-4854-af10-1497433fe7b4
├─sdc1 zfs_member 5000 lxdpool 7306657253134884339
└─sdc9
sdd btrfs lxd 23dfa8bd-b1f5-4854-af10-1497433fe7b4
├─sdd1 zfs_member 5000 lxdpool 7306657253134884339
└─sdd9
sde btrfs lxd 23dfa8bd-b1f5-4854-af10-1497433fe7b4
├─sde1 zfs_member 5000 lxdpool 7306657253134884339
└─sde9
sdf btrfs downloads 6c8295e3-6084-42fe-a9ac-486f05944db1
├─sdf1 zfs_member 5000 downloads 17223595357488254896
The second command returns empty
Run it without the grep, does it reference those drives at all?
root@esx:/home/downloads# df -Th | “^/dev/sd”
-bash: ^/dev/sd: No such file or directory
Could you post the zpool status -v command output?
Regards.
root@esx:/home/downloads# zpool status -v
pool: downloads
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
downloads ONLINE 0 0 0
ata-Samsung_SSD_870_QVO_1TB_S5RRNF0R642514M ONLINE 0 0 0
errors: No known data errors
pool: lxdpool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
lxdpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-Samsung_SSD_870_QVO_1TB_S5RRNF0R642530P ONLINE 0 0 0
ata-Samsung_SSD_870_QVO_1TB_S5RRNF0R644421A ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
ata-Samsung_SSD_870_QVO_1TB_S5RRNF0R642539W ONLINE 0 0 0
ata-Samsung_SSD_870_QVO_1TB_S5RRNF0R642467A ONLINE 0 0 0
errors: No known data errors
pool: usb_backup
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
usb_backup ONLINE 0 0 0
ata-Samsung_SSD_870_QVO_2TB_S5RPNF0R603981X ONLINE 0 0 0
errors: No known data errors
if i use this command:
root@esx:~# df -Th /dev/sd*
Filesystem Type Size Used Avail Use% Mounted on
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
udev devtmpfs 63G 0 63G 0% /dev
Try just: df -Th
I suspect that you have ZFS on top of another filesystem; which is causing your system to run sloooow; and it means backing up like you did before; then wiping/cussing-out/shredding the drives intended for the zpool.
Then you should be able to build the raid 10 array using ZFS and then attach it to LXD as storage. However, I speculate that the best way to go about this is to initialize the zpool from within LXD on a single drive and then add drives to the pool via ZFS.
root@esx:~# df -Th
Filesystem Type Size Used Avail Use% Mounted on
udev devtmpfs 63G 0 63G 0% /dev
tmpfs tmpfs 13G 5.4M 13G 1% /run
/dev/nvme0n1p2 btrfs 466G 4.3G 460G 1% /
tmpfs tmpfs 63G 0 63G 0% /dev/shm
tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/nvme0n1p1 vfat 511M 5.8M 506M 2% /boot/efi
downloads/downloads zfs 899G 648G 252G 73% /home/downloads
usb_backup/backup zfs 1.8T 375G 1.4T 21% /home/usb_backup
usb_backup zfs 1.4T 128K 1.4T 1% /usb_backup
downloads zfs 252G 256K 252G 1% /downloads
/dev/loop0 squashfs 64M 64M 0 100% /snap/core20/1738
/dev/loop1 squashfs 64M 64M 0 100% /snap/core20/1778
/dev/loop3 squashfs 137M 137M 0 100% /snap/lxd/24164
/dev/loop2 squashfs 137M 137M 0 100% /snap/lxd/24155
/dev/loop4 squashfs 50M 50M 0 100% /snap/snapd/17883
tmpfs tmpfs 1.0M 0 1.0M 0% /var/snap/lxd/common/ns
tmpfs tmpfs 13G 4.0K 13G 1% /run/user/0
ok,
i will do this. one last question, when i run lxc init and it asks me for storage i yust can give the full drive to lxd?
and when i shred the disk do i need to partition it for zfs? or is a shred enough and the just give it to lxd?
Shred, mount the drive if you have too, and then give it to LXD.
Feed a single drive to LXD; such as “/dev/sda”. It should do the rest.
Then use ZFS commands to configure the raid 10 after the fact. Later, I can play around with this idea in a test machine; but not at the moment. Again this is just a guess. But, please report back if you move forward with this.
I will try this.
However if i do a lxd init will it remove the second storage pool also? what i want to do is create a second lxd storage and move all machines to this storage pool. The lxc init and move all back