Unable to start LXC unprivileged container: ERROR lxc_cgfs - cgroups Permission denied

I am trying to start LXC container with Debian.
I follow this documentation :
https://wiki.debian.org/LXC
https://linuxcontainers.org/fr/lxc/getting-started/

I was able to create an unprivileged container but when I try to start the container, I always failed on Permission denied. I tried several possible fixes, but the problem remains. It looks as a permissions problem. Can someone help me find a solution without compromising security ?

Some system informations :

lxc-start -n centos7 -d --logfile /tmp/lxc.log --logpriority DEBUG
cat /tmp/lxc.log
lxc-start 20170712084453.979 INFO lxc_start_ui - tools/lxc_start.c:main:275 - using rcfile /home/olivier/.local/share/lxc/centos7/config
lxc-start 20170712084453.979 WARN lxc_confile - confile.c:config_pivotdir:1916 - lxc.pivotdir is ignored. It will soon become an error.
lxc-start 20170712084453.979 INFO lxc_confile - confile.c:config_idmap:1537 - read uid map: type u nsid 0 hostid 1214112 range 65536
lxc-start 20170712084453.979 INFO lxc_confile - confile.c:config_idmap:1537 - read uid map: type g nsid 0 hostid 1214112 range 65536
lxc-start 20170712084453.980 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:330 - Going to wait for pid 5165.
lxc-start 20170712084453.980 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:349 - Trying to sync with child process.
lxc-start 20170712084453.981 INFO lxc_start - start.c:lxc_check_inherited:235 - Closed inherited fd: 4.
lxc-start 20170712084453.981 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:387 - Using pipe file descriptor 5 for monitord.
lxc-start 20170712084453.987 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:364 - Successfully synced with child process.
lxc-start 20170712084453.988 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:333 - Finished waiting on pid 5165.
lxc-start 20170712084453.988 INFO lxc_container - lxccontainer.c:do_lxcapi_start:802 - Attempting to set proc title to [lxc monitor] /home/olivier/.local/share/lxc centos7
lxc-start 20170712084453.989 INFO lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver nop
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:418 - processing: .reject_force_umount # comment this to allow umount -f; not recommended.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:590 - Adding native rule for reject_force_umount action 0.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:do_resolve_add_rule:259 - Setting Seccomp rule to reject force umounts.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:593 - Adding compat rule for reject_force_umount action 0.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:do_resolve_add_rule:259 - Setting Seccomp rule to reject force umounts.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:do_resolve_add_rule:259 - Setting Seccomp rule to reject force umounts.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:418 - processing: .[all].
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:418 - processing: .kexec_load errno 1.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:590 - Adding native rule for kexec_load action 327681.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:593 - Adding compat rule for kexec_load action 327681.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:418 - processing: .open_by_handle_at errno 1.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:590 - Adding native rule for open_by_handle_at action 327681.
lxc-start 20170712084453.989 INFO lxc_seccomp - seccomp.c:parse_config_v2:593 - Adding compat rule for open_by_handle_at action 327681.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:418 - processing: .init_module errno 1.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:590 - Adding native rule for init_module action 327681.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:593 - Adding compat rule for init_module action 327681.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:418 - processing: .finit_module errno 1.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:590 - Adding native rule for finit_module action 327681.
lxc-start 20170712084453.990 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:330 - Going to wait for pid 5169.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:593 - Adding compat rule for finit_module action 327681.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:418 - processing: .delete_module errno 1.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:590 - Adding native rule for delete_module action 327681.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:593 - Adding compat rule for delete_module action 327681.
lxc-start 20170712084453.990 INFO lxc_seccomp - seccomp.c:parse_config_v2:603 - Merging in the compat Seccomp ctx into the main one.
lxc-start 20170712084453.990 DEBUG lxc_start - start.c:setup_signal_fd:273 - Set SIGCHLD handler with file descriptor: 5.
lxc-start 20170712084453.990 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:349 - Trying to sync with child process.
lxc-start 20170712084453.990 INFO lxc_start - start.c:lxc_check_inherited:235 - Closed inherited fd: 4.
lxc-start 20170712084453.990 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:387 - Using pipe file descriptor 5 for monitord.
lxc-start 20170712084453.990 DEBUG console - console.c:lxc_console_peer_default:438 - process does not have a controlling terminal
lxc-start 20170712084453.996 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:364 - Successfully synced with child process.
lxc-start 20170712084453.997 DEBUG lxc_monitor - monitor.c:lxc_monitord_spawn:333 - Finished waiting on pid 5169.
lxc-start 20170712084453.997 INFO lxc_monitor - monitor.c:lxc_monitor_sock_name:201 - using monitor socket name “lxc/a1179fb1c6067142//home/olivier/.local/share/lxc” (length of socket name 51 must be <= 105)
lxc-start 20170712084453.997 DEBUG lxc_monitor - monitor.c:lxc_monitor_open:225 - opening monitor socket lxc/a1179fb1c6067142//home/olivier/.local/share/lxc with len 51
lxc-start 20170712084454.694 INFO lxc_start - start.c:lxc_init:475 - Container “centos7” is initialized.
lxc-start 20170712084454.701 DEBUG lxc_start - start.c:__lxc_start:1325 - Not dropping CAP_SYS_BOOT or watching utmp.
lxc-start 20170712084454.701 INFO lxc_cgroup - cgroups/cgroup.c:cgroup_init:68 - cgroup driver cgroupfs initing for centos7
lxc-start 20170712084454.706 ERROR lxc_cgfs - cgroups/cgfs.c:lxc_cgroupfs_create:909 - Could not set clone_children to 1 for cpuset hierarchy in parent cgroup.
lxc-start 20170712084454.707 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset/
lxc-start 20170712084454.708 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/pids/user.slice/user-1000.slice/session-2.scope
lxc-start 20170712084454.709 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/memory/user.slice
lxc-start 20170712084454.710 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/net_cls,net_prio/
lxc-start 20170712084454.710 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/freezer/
lxc-start 20170712084454.712 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/blkio/user.slice
lxc-start 20170712084454.712 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/perf_event/
lxc-start 20170712084454.713 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpu,cpuacct/user.slice
lxc-start 20170712084454.714 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/devices/user.slice
lxc-start 20170712084454.714 ERROR lxc_cgfs - cgroups/cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/systemd/user.slice/user-1000.slice/session-2.scope
lxc-start 20170712084454.715 ERROR lxc_start - start.c:lxc_spawn:1119 - Failed creating cgroups.
lxc-start 20170712084454.715 ERROR lxc_start - start.c:__lxc_start:1354 - Failed to spawn container “centos7”.
lxc-start 20170712084454.716 INFO lxc_conf - conf.c:run_script_argv:427 - Executing script “/usr/share/lxcfs/lxc.reboot.hook” for container “centos7”, config section “lxc”.
lxc-start 20170712084454.575 WARN lxc_commands - commands.c:lxc_cmd_rsp_recv:177 - Command get_cgroup failed to receive response: Connection reset by peer.
lxc-start 20170712084459.580 ERROR lxc_start_ui - tools/lxc_start.c:main:366 - The container failed to start.
lxc-start 20170712084459.580 ERROR lxc_start_ui - tools/lxc_start.c:main:368 - To get more details, run the container in foreground mode.
lxc-start 20170712084459.580 ERROR lxc_start_ui - tools/lxc_start.c:main:370 - Additional information can be obtained by setting the --logfile and --logpriority options.

cat /proc/self/cgroup
10:cpuset:/
9:pids:/user.slice/user-1000.slice/session-2.scope
8:memory:/user.slice
7:net_cls,net_prio:/
6:freezer:/
5:blkio:/user.slice
4:perf_event:/
3:cpu,cpuacct:/user.slice
2:devices:/user.slice
1:name=systemd:/user.slice/user-1000.slice/session-2.scope
0::/user.slice/user-1000.slice/session-2.scope

cat ~/.local/share/lxc/centos7/config
Distribution configuration
lxc.include = /usr/share/lxc/config/centos.common.conf
lxc.include = /usr/share/lxc/config/centos.userns.conf
lxc.arch = x86_64

Container specific configuration
lxc.id_map = u 0 1214112 65536
lxc.id_map = g 0 1214112 65536
lxc.rootfs = /home/olivier/.local/share/lxc/centos7/rootfs
lxc.rootfs.backend = dir
lxc.utsname = centos7

Network configuration
lxc.network.type = veth
lxc.network.link = lxcbr0

cat /etc/subuid
systemd-timesync:100000:65536
systemd-network:165536:65536
systemd-resolve:231072:65536
systemd-bus-proxy:296608:65536
messagebus:362144:65536
avahi:427680:65536
Debian-exim:493216:65536
statd:558752:65536
colord:624288:65536
dnsmasq:689824:65536
geoclue:755360:65536
pulse:820896:65536
rtkit:886432:65536
saned:951968:65536
usbmux:1017504:65536
lightdm:1083040:65536
hplip:1148576:65536
olivier:1214112:65536
_apt:1279648:65536
libvirt-qemu:1345184:65536
olivier:1214112:65537

cat /etc/subgid
systemd-timesync:100000:65536
systemd-network:165536:65536
systemd-resolve:231072:65536
systemd-bus-proxy:296608:65536
messagebus:362144:65536
avahi:427680:65536
Debian-exim:493216:65536
statd:558752:65536
colord:624288:65536
dnsmasq:689824:65536
geoclue:755360:65536
pulse:820896:65536
rtkit:886432:65536
saned:951968:65536
usbmux:1017504:65536
lightdm:1083040:65536
hplip:1148576:65536
olivier:1214112:65536
_apt:1279648:65536
libvirt-qemu:1345184:65536
olivier:1214112:65537

lxc-ls -f
NAME STATE AUTOSTART GROUPS IPV4 IPV6
centos7 STOPPED 0 - - -

lxc-checkconfig
Kernel configuration not found at /proc/config.gz; searching…
Kernel configuration found at /boot/config-4.11.0-1-amd64
— Namespaces —
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Warning: newuidmap is not setuid-root
Warning: newgidmap is not setuid-root
Network namespace: enabled

— Control groups —
Cgroup: enabled
Cgroup clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled

— Misc —
Veth pair device: enabled
Macvlan: enabled
Vlan: enabled
Bridges: enabled
Advanced netfilter: enabled
CONFIG_NF_NAT_IPV4: enabled
CONFIG_NF_NAT_IPV6: enabled
CONFIG_IP_NF_TARGET_MASQUERADE: enabled
CONFIG_IP6_NF_TARGET_MASQUERADE: enabled
CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled
FUSE (for use with lxcfs): enabled

— Checkpoint/Restore —
checkpoint restore: enabled
CONFIG_FHANDLE: enabled
CONFIG_EVENTFD: enabled
CONFIG_EPOLL: enabled
CONFIG_UNIX_DIAG: enabled
CONFIG_INET_DIAG: enabled
CONFIG_PACKET_DIAG: enabled
CONFIG_NETLINK_DIAG: enabled
File capabilities: enabled

Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig

This looks identical to this other thread: Failed creating cgroups

Thanks that works with ubuntu package.