Even with root user Im receiving 'Operation not permitted' when try creating gluster volume between Ubuntu 14.04 LXC containers

Even with root user Im receiving ‘Operation not permitted’ when try creating gluster volume between Ubuntu 14.04 LXC containers.
Even adding the ppa:gluster/glusterfs-4.1 the update failed to get the last version of GlusterFs as can see on code below.
Need test the solution locally before install it on prodution and even locally i cant install.
Just come here now after all this time because wanted try hard before bother you again with it, thanks for the help.
Im trying solve the problem since your last comment without sucess, the current status is the following:

xfce4-terminal -T LocalTerm
exit

At LocalTerm:

PS1='[\u@\h]-[\D{%T}]-[\W] => '  

lxc profile copy default default-bkp  
lxc profile set default raw.lxc lxc.apparmor.profile=unconfined  

clear   
lxc stop ubt1404X64C001  
lxc stop ubt1404X64C002  

lxc delete ubt1404X64C001  
lxc delete ubt1404X64C002  

lxc launch ubuntu:14.04 ubt1404X64C001  
lxc launch ubuntu:14.04 ubt1404X64C002  

clear  
sleep 10  

lxc list --format csv  

lxc config get ubt1404X64C001 security.privileged  
lxc config set ubt1404X64C001 security.privileged true  
lxc config get ubt1404X64C002 security.privileged  
lxc config set ubt1404X64C002 security.privileged true  
lxc restart ubt1404X64C001  
lxc restart ubt1404X64C002    

xfce4-terminal -T ubt1404X64C001Term -e "bash -c 'lxc exec ubt1404X64C001 -- /bin/bash; exec bash; exit 0;'"   
xfce4-terminal -T ubt1404X64C002Term -e "bash -c 'lxc exec ubt1404X64C002 -- /bin/bash; exec bash; exit 0;'"   

At ubt1404X64C001Term and ubt1404X64C002Term:

PS1='[\u@\h]-[\D{%T}]-[\W] => '  
echo '' >> /etc/hosts  
apt-get update   

apt-get upgrade  
apt-get dist-upgrade  
apt-get autoremove  
apt-get install netcat net-tools  
lsb_release -a  
	No LSB modules are available.  
	Distributor ID:	Ubuntu  
	Description:	Ubuntu 14.04.5 LTS  
	Release:	14.04  
	Codename:	trusty  
uname -a  
	Linux ubt1404X64C002 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux  
cat /etc/fstab  
	LABEL=cloudimg-rootfs	/	 ext4	defaults	0 0		  
apt-get install attr fuse libaio1 liburcu-dev libxml2 python2.7 python-pip rpcbind   
echo '' > ~/testAttr  
setfattr -n user.name -v "UserNameValue" ~/testAttr  
setfattr -n trusted.glusterfs.test -v "working" ~/testAttr  
getfattr -d -m - ~/testAttr  
	trusted.glusterfs.test="working"  
	user.name="UserNameValue"  
add-apt-repository ppa:gluster/glusterfs-4.1  
apt-get update  
	W: Failed to fetch   
	http://ppa.launchpad.net/gluster/glusterfs-4.1/ubuntu/dists  
	/trusty/main/binary-amd64/Packages  404  Not Found  
	E: Some index files failed to download. They have been ignored, or old ones used instead.  
	but available to: artful,bionic,cosmic,devel and xenial  
apt-get install glusterfs-server glusterfs-client open-iscsi watchdog    
glusterfs --version | grep built  
	glusterfs 3.4.2 built on Jan 14 2014 18:05:35  
	Not installed 4.1 version because apt-get update failed with error above to trusty version  

At ubt1404X64C002Term:

ifconfig eth0 | grep 'inet '  
	20.30.40.50  
echo '10.20.30.40 ubt1404X64C001' >> /etc/hosts	  
cat /etc/hostname  
ping ubt1404X64C001  
iptables -I INPUT -p all -s ubt1404X64C001 -j ACCEPT  

At ubt1404X64C001Term:

ifconfig eth0 | grep 'inet '  
	10.20.30.40  
echo '20.30.40.50 ubt1404X64C002' >> /etc/hosts  
cat /etc/hostname  
ping ubt1404X64C002  
iptables -I INPUT -p all -s ubt1404X64C002 -j ACCEPT  

mkdir -p /root/tomirrorwith/glusterfs  
cd /root/tomirrorwith/glusterfs	  
gluster peer probe ubt1404X64C002  
	peer probe: success  
gluster peer status   
	Hostname: ubt1404X64C002    
	Port: 24007    
	State: Peer in Cluster (Connected)    
gluster volume create gv0 replica 2 ubt1404X64C001:/root/tomirrorwith/glusterfs ubt1404X64C002:/root/tomirrorwith/glusterfs force  
	volume create: gv0: failed  
	------  
	/var/log/glusterfs/cli.log at ubt1404X64C001  
		W [rpc-transport.c:175:rpc_transport_load] 0-rpc-transport: missing 'option transport-type'. defaulting to "socket"  
		I [socket.c:3480:socket_init] 0-glusterfs: SSL support is NOT enabled  
		I [socket.c:3495:socket_init] 0-glusterfs: using system polling thread  
		I [cli-cmd-volume.c:392:cli_cmd_volume_create_cbk] 0-cli: Replicate cluster type found. Checking brick order.  
		I [cli-cmd-volume.c:304:cli_cmd_check_brick_order] 0-cli: Brick order okay  
		I [cli-rpc-ops.c:805:gf_cli_create_volume_cbk] 0-cli: Received resp to create volume  
		I [input.c:36:cli_batch] 0-: Exiting with: -1   
	------  
	/var/log/glusterfs/etc-glusterfs-glusterd.vol.log at ubt1404X64C002  
		E [glusterd-op-sm.c:3719:glusterd_op_ac_stage_op] 0-management: Stage failed on operation 'Volume Create', Status : -1  
	------  

TRY FIX At C001TermUbt1404X64:

    cd /sys/module/fuse/parameters/  
    echo Y > userns_mounts  
        ERROR:  
            bash: userns_mounts: Permission denied  

The same question on ServerFault

It’s attempting to set some extended attributes, if those are file capabilities, then this may work on newer kernels, if it’s another type of attribute, this will not work unless you make the container privileged (security.privileged=true).

1 Like

I updated the question body above with your suggestions and was trying hard on your sugestion all this time without solution, thanks for the help.

I cant change the ubuntu version on my cheap cloud provider but did locally on ubuntu 18.04 to test.
On ubuntu 18.04 is possible install the version 4.1 of gluster and it solved the error above but the gluster dont mirror the content of my mirrored folder as you can see below:

xfce4-terminal -T LocalTerm
exit

At LocalTerm:

PS1='[\u@\h]-[\D{%T}]-[\W] => '  

lxc profile copy default default-bkp  
lxc profile set default raw.lxc lxc.apparmor.profile=unconfined  

clear   
lxc stop ubt1804X64C001  
lxc stop ubt1804X64C002  

lxc delete ubt1804X64C001  
lxc delete ubt1804X64C002  

lxc launch ubuntu:18.04 ubt1804X64C001  
lxc launch ubuntu:18.04 ubt1804X64C002  

clear  
sleep 10  

lxc list --format csv  

lxc config get ubt1804X64C001 security.privileged  
lxc config set ubt1804X64C001 security.privileged true  
lxc config get ubt1804X64C002 security.privileged  
lxc config set ubt1804X64C002 security.privileged true  
lxc restart ubt1804X64C001  
lxc restart ubt1804X64C002    

xfce4-terminal -T ubt1804X64C001Term -e "bash -c 'lxc exec ubt1804X64C001 -- /bin/bash; exec bash; exit 0;'"   
xfce4-terminal -T ubt1804X64C002Term -e "bash -c 'lxc exec ubt1804X64C002 -- /bin/bash; exec bash; exit 0;'"   

At ubt1804X64C001Term and ubt1804X64C002Term:

PS1='[\u@\h]-[\D{%T}]-[\W] => '  
echo '' >> /etc/hosts  
apt-get update   

apt-get upgrade  
apt-get dist-upgrade  
apt-get autoremove  
apt-get install netcat net-tools  
lsb_release -a  
	No LSB modules are available.  
	Distributor ID:	Ubuntu  
	Description:	Ubuntu 18.04.1 LTS  
	Release:	18.04  
	Codename:	bionic  
uname -a  
	Linux ubt1804X64C001 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux   
cat /etc/fstab  
	LABEL=cloudimg-rootfs	/	 ext4	defaults	0 0  
apt-get install attr fuse libaio1 liburcu-dev libxml2 python2.7 python-pip rpcbind   
echo '' > ~/testAttr  
setfattr -n user.name -v "UserNameValue" ~/testAttr  
setfattr -n trusted.glusterfs.test -v "working" ~/testAttr  
getfattr -d -m - ~/testAttr  
	trusted.glusterfs.test="working"  
	user.name="UserNameValue"  
add-apt-repository ppa:gluster/glusterfs-4.1  
apt-get update  
	OK   
apt-get install glusterfs-server glusterfs-client open-iscsi watchdog 
glusterfs --version | grep 'glusterfs '  
	glusterfs 4.1.5    
mkdir -p /root/tomirrorwith/glusterfs  
cd /root/tomirrorwith/glusterfs	  
ls

At ubt1804X64C002Term:

ifconfig eth0 | grep 'inet '  
	20.30.40.50  
echo '10.20.30.40 ubt1804X64C001' >> /etc/hosts	  
cat /etc/hostname  
ping ubt1804X64C001  
iptables -I INPUT -p all -s ubt1804X64C001 -j ACCEPT  

At ubt1804X64C001Term:

ifconfig eth0 | grep 'inet '  
	10.20.30.40  
echo '20.30.40.50 ubt1804X64C002' >> /etc/hosts  
cat /etc/hostname  
ping ubt1804X64C002  
iptables -I INPUT -p all -s ubt1804X64C002 -j ACCEPT  

gluster peer probe ubt1804X64C002  
	peer probe: success  
gluster peer status   
	Number of Peers: 1  
	Hostname: ubt1804X64C002      
	Port: 24007  
	State: Peer in Cluster (Connected)  

gluster volume create gv0 replica 2 ubt1804X64C001:/root/tomirrorwith/glusterfs ubt1804X64C002:/root/tomirrorwith/glusterfs force  
	volume create: gv0: success: please start the volume to access data  
gluster volume start gv0  
	volume start: gv0: success 
echo 'TestValue' > TestFile

At ubt1804X64C002Term:

cd /root/tomirrorwith/glusterfs	  
ls
# There is no TestFile created here on mirrored machine