My first basic container.. mkdir: ‘/home/user’: Permission denied

LXC
#1

For all of the config files, only those in locations that required root permissions, were made with sudo. I have read both official and downstream debian documentation at least twice and also read all the manpages. Ran on Ubuntu based OS with up to date kernel, mint 20.3 cinnamon.

export DOWNLOAD_KEYSERVER="hkp://keyserver.ubuntu.com"
systemd-run --unit=test --scope -p “Delegate=yes” – lxc-create -n test -t download
debian
bullseye
amd64

Downloading the image index
Downloading the rootfs
Downloading the metadata
mkdir: cannot create directory ‘/home/user’: Permission denied
lxc-create: test: lxccontainer.c: create_run_template: 1616 Failed to create container from template
lxc-create: test: utils.c: lxc_rmdir_onedev: 191 Permission denied - Failed to stat “/media/user/x/test/rootfs”
lxc-create: test: storage/dir.c: dir_destroy: 104 Permission denied - Failed to delete “/media/user/x/test/rootfs”
lxc-create: test: storage/storage_utils.c: storage_destroy_wrapper: 477 Failed to destroy storage
lxc-create: test: lxccontainer.c: container_destroy: 3008 Error destroying rootfs for test
lxc-create: test: tools/lxc_create.c: main: 319 Failed to create container test

cat /etc/subuid
user:100000:65536
cat /etc/subgid
user:100000:65536

default.conf
lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:xx:xx:xx
lxc.mount.auto = proc:mixed sys:mixed cgroup:rw:force
lxc.ephemeral = 0
lxc.apparmor.profile = unconfined
lxc.arch = amd64

# this config has these 2 lines appended
~/.config/lxc/default.conf
lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

# both files have the same content
#this path is owned by the same user.
/etc/lxc/lxc.conf
~/.config/lxc/lxc.conf
lxc.lxcpath = /media/user/x

/etc/lxc/lxc-usernet
user veth lxcbr0 255

/etc/default/lxc-net
USE_LXC_BRIDGE=“true”
LXC_BRIDGE=“lxcbr0”
LXC_ADDR=“10.0.3.1”
LXC_NETMASK=“255.255.255.0”
LXC_NETWORK=“10.0.3.0/24”
LXC_DHCP_RANGE=“10.0.3.2,10.0.3.254”
LXC_DHCP_MAX=“253”
LXC_DHCP_CONFILE=""
LXC_DOMAIN=""

#installed with
sudo apt-get install apparmor libpam-cgfs uidmap bridge-utils libvirt0 lxc

lxc-checkconfig
#See below…

#2

lxc-checkconfig
LXC version 4.0.6
— Namespaces —
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled

— Control groups —
Cgroups: enabled

Cgroup v1 mount points:
/sys/fs/cgroup/systemd
/sys/fs/cgroup/rdma
/sys/fs/cgroup/cpuset
/sys/fs/cgroup/freezer
/sys/fs/cgroup/devices
/sys/fs/cgroup/pids
/sys/fs/cgroup/misc
/sys/fs/cgroup/hugetlb
/sys/fs/cgroup/net_cls,net_prio
/sys/fs/cgroup/blkio
/sys/fs/cgroup/perf_event
/sys/fs/cgroup/cpu,cpuacct
/sys/fs/cgroup/memory

Cgroup v2 mount points:
/sys/fs/cgroup/unified

Cgroup v1 clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled

— Misc —
Veth pair device: enabled, not loaded
Macvlan: enabled, not loaded
Vlan: enabled, not loaded
Bridges: enabled, loaded
Advanced netfilter: enabled, loaded
CONFIG_NF_NAT_IPV4: missing
CONFIG_NF_NAT_IPV6: missing
CONFIG_IP_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_IP6_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, loaded
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, loaded
FUSE (for use with lxcfs): enabled, not loaded

— Checkpoint/Restore —
checkpoint restore: enabled
CONFIG_FHANDLE: enabled
CONFIG_EVENTFD: enabled
CONFIG_EPOLL: enabled
CONFIG_UNIX_DIAG: enabled
CONFIG_INET_DIAG: enabled
CONFIG_PACKET_DIAG: enabled
CONFIG_NETLINK_DIAG: enabled
File capabilities:

#That was all.
As you can see there are 2 missing, but seems to be unified in newer kernels and not my problem?
I bet not loaded is also no problem.
What is my problem guys?? :frowning:

#3

Update: (Still not fixed)
I now know for sure that the error:
mkdir: cannot create directory ‘/home/user’: Permission denied
Is not relative to the root of the container specified in lxc.conf with lxc.lxcpath

That is because I changed my $HOME environment variable to the actual home directory @/etc/passwd. I had a softlink@/home/user to the actual directory @/media before.

The new error is: mkdir: cannot create directory ‘/media/user’: Permission denied
By the way this directory obviously already exists.

Full error message:
export DOWNLOAD_KEYSERVER=“hkp://keyserver.ubuntu.com”
systemd-run --unit=test --scope -p “Delegate=yes” – lxc-create -n test -t download – -d debian -r bullseye -a amd64

Running scope as unit: test.scope
Setting up the GPG keyring
Downloading the image index
Downloading the rootfs
Downloading the metadata
mkdir: cannot create directory ‘/media/user’: Permission denied
lxc-create: test: lxccontainer.c: create_run_template: 1616 Failed to create container from template
lxc-create: test: utils.c: lxc_rmdir_onedev: 191 Permission denied - Failed to stat “/media/user/x/Containers/test/rootfs”
lxc-create: test: storage/dir.c: dir_destroy: 104 Permission denied - Failed to delete “/media/user/x/Containers/test/rootfs”
lxc-create: test: storage/storage_utils.c: storage_destroy_wrapper: 477 Failed to destroy storage
lxc-create: test: lxccontainer.c: container_destroy: 3008 Error destroying rootfs for test
lxc-create: test: tools/lxc_create.c: main: 319 Failed to create container test

I also changed the permissions on the dir of lxc.lxcpath to allow anyone to read and write and that didn’t do a thing, so that is also not it.

Research about failed to stat error:
"but-in the case of stat() and lstat() - execute (search) permission is required on all of the directories in path that lead to the file. " I will try this, won’t fix the errors following it though… I think

#4

I can’t believe I fixed it.
First of all I put --user in the systemd-run lxc-create command… I don’t understand why I didn’t put that there in the first place.

export DOWNLOAD_KEYSERVER=“hkp://keyserver.ubuntu.com”
systemd-run --unit=test --user --scope -p “Delegate=yes” – lxc-create -n test -t download – -d debian -r bullseye -a amd64

The second thing is I checked /etc for subuid and subgid and I saw 2 backup files called subuid- and subgid-
I saw inside the backup file that the wrong uidmap is applied by the lxc package:
test:165536:65536
Instead of 100000 65536. Also don’t know if it’s really supposed to put in the name of the container instead of the user.
Nowhere in my config script I have put 165536, so I don’t know what magic is being done by the lxc package.

Before I put these 2 lines only inside ~/.config/lxc/default.conf
,since I am not using lxc-create as root.
Also putting this line in /etc/lxc/default.conf magically fixed my problem and the container suddenly installed properly.
lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

I will try removing it and testing again to see if I am crazy or not
Ok tested, apparently/probably the only thing I did wrong was not putting –user in, so I also can’t find any fault in the quick start guide. I believe I removed it by mistake instead of –unit=, which I assume is there only for convenience and I won’t be using cgroups for now. I do vaguely remember it also not working before I did that but whatever.

Oh and also put
enable lxc-net
restart lxc-net
after the /etc/lxc/lxc-usernet edit, had it before that in my script for some reason
Dunno if that mattered