Summary: Getting error when I try to replicate gluster in lxd vms. The vms are utilizing mounted
Error:
root@gluster1 ~# gluster volume create vol1 replica 3 gluster1:/glustervolume1/vol1 gluster2:/glustervolume1/vol1 gluster3:/glustervolume1/vol1
volume create: vol1: failed: Glusterfs is not supported on brick: gluster1:/glustervolume1/vol1.
Setting extended attributes failed, reason: Operation not supported.
Synopsis:
I have been able to set up a glusterfs nodes in lxd vms.
The gluster pool seem ok from the test below (test is run on the first node gluster1):
root@gluster1 ~# gluster pool list
UUID Hostname State
42f06d41-492e-4cbe-8397-b163a8940244 gluster2 Connected
4151aa87-6de6-4ae7-bb94-fca0f8558de7 gluster3 Connected
1e33df9c-bc40-4721-b4f8-f195166e0947 localhost Connected
On the host I have created dedicated partitions, /dev/sda3/ and /dev/sd4. Which are then mounted on the host as /glustervolume and /gulstervolume2. For confirmation and reference, see below:
emp-06@emp06 ~> df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 2.4G 4.1M 2.4G 1% /run
/dev/sda2 171G 147G 16G 91% /
tmpfs 12G 100K 12G 1% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 12G 0 12G 0% /run/qemu
/dev/sda1 511M 6.1M 505M 2% /boot/efi
tmpfs 1.0M 0 1.0M 0% /var/snap/lxd/common/ns
tmpfs 2.4G 120K 2.4G 1% /run/user/1000
/dev/sda4 32G 24K 30G 1% /glustervolume2
/dev/sda3 31G 24K 30G 1% /glustervolume
I have then mounted the partitions on all the lxd glusterfs nodes(example for the 3rd node):
emp-06@emp06 ~> lxc exec gluster3 -- mkdir -p /glustervolume1/vol1
lxc stop gluster3
lxc config device add gluster3 sharedisk1 disk source=/glustervolume3/ path=/glustervolume1/
lxc start gluster3
Device sharedisk1 added to gluster3
To confirm that the partition is mounted on the lxd, I do:
root@gluster1 ~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 9.6G 1.7G 7.9G 17% /
tmpfs 483M 0 483M 0% /dev/shm
tmpfs 194M 640K 193M 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 50M 14M 37M 27% /run/lxd_agent
lxd_sharedisk1 31G 28K 30G 1% /glustervolume1
/dev/sda15 105M 6.1M 99M 6% /boot/efi
tmpfs 1.0M 0 1.0M 0% /var/snap/lxd/common/ns
tmpfs 97M 4.0K 97M 1% /run/user/0
However when I try to setup replica for glusterfs, I get error:
root@gluster1 ~# gluster volume create vol1 replica 3 gluster1:/glustervolume1/vol1 gluster2:/glustervolume1/vol1 gluster3:/glustervolume1/vol1
volume create: vol1: failed: Glusterfs is not supported on brick: gluster1:/glustervolume1/vol1.
Setting extended attributes failed, reason: Operation not supported.