File exists - Failed to create directory "/sys/fs/cgroup/systemd//lxc.payload/test001"

I was successful in creating this container once, but after destroying it and trying again, it fails to start. I’ve tried this on a different system, and it seems to be working fine, so I’m not sure what’s caused this one to go badly, because both systems are running the lxd snap: version 3.15 rev 11270

Here’s what I’m doing:
I’ve created/published a local image that I want to base things on, called tf-bionic. This is just bionic with a few other things installed.
$ lxc init tf-bionic test001
$ lxc config set $1 raw.idmap “both $UID 1000”
$ lxc config device add test001 tfdir disk source=$PWD path=/home/ubuntu
$ lxc config device add test001 sshconf disk source=$HOME/.ssh path=/home/ubuntu/.ssh readonly=true
$ lxc config device add test001 deviceconf disk source=/path/to/config/default.conf path=/home/ubuntu/default.conf readonly=true
$ lxc start test001

When I try to start it, I get:

Error: Failed to run: /snap/lxd/current/bin/lxd forkstart test001 /var/snap/lxd/common/lxd/containers /var/snap/lxd/common/lxd/logs/test001/lxc.conf:
Try lxc info --show-log test001 for more info

Here’s the output from lxc info --show-log:

Name: test001
Location: none
Remote: unix://
Architecture: x86_64
Created: 2019/07/19 21:27 UTC
Status: Stopped
Type: persistent
Profiles: default

Log:

lxc test001 20190719212806.860 ERROR cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1295 - File exists - Failed to create directory “/sys/fs/cgroup/systemd//lxc.payload/test001”
lxc test001 20190719212806.860 ERROR cgfsng - cgroups/cgfsng.c:container_create_path_for_hierarchy:1338 - Failed to create cgroup “/sys/fs/cgroup/systemd//lxc.payload/test001”
lxc test001 20190719212806.860 ERROR cgfsng - cgroups/cgfsng.c:cgfsng_payload_create:1486 - Failed to create cgroup “/sys/fs/cgroup/systemd//lxc.payload/test001”
lxc test001 20190719212806.874 WARN conf - conf.c:lxc_map_ids:2970 - newuidmap binary is missing
lxc test001 20190719212806.874 WARN conf - conf.c:lxc_map_ids:2976 - newgidmap binary is missing
lxc test001 20190719212806.898 WARN conf - conf.c:lxc_map_ids:2970 - newuidmap binary is missing
lxc test001 20190719212806.898 WARN conf - conf.c:lxc_map_ids:2976 - newgidmap binary is missing
lxc test001 20190719212806.979 ERROR conf - conf.c:mount_entry_create_dir_file:2179 - Permission denied - Failed to create directory “/var/snap/lxd/common/lxc//home/ubuntu/.ssh”
lxc test001 20190719212806.979 ERROR conf - conf.c:lxc_setup:3664 - Failed to setup mount entries
lxc test001 20190719212806.979 ERROR start - start.c:do_start:1282 - Failed to setup container “test001”
lxc test001 20190719212806.979 ERROR sync - sync.c:__sync_wait:62 - An error occurred in another process (expected sequence number 5)
lxc test001 20190719212807.328 ERROR start - start.c:__lxc_start:1975 - Failed to spawn container “test001”
lxc test001 20190719212807.328 ERROR lxccontainer - lxccontainer.c:wait_on_daemonized_start:864 - Received container state “ABORTING” instead of “RUNNING”
lxc test001 20190719212807.776 WARN conf - conf.c:lxc_map_ids:2970 - newuidmap binary is missing
lxc test001 20190719212807.776 WARN conf - conf.c:lxc_map_ids:2976 - newgidmap binary is missing
lxc 20190719212807.958 WARN commands - commands.c:lxc_cmd_rsp_recv:132 - Connection reset by peer - Failed to receive response for command “get_state”

Any suggestions for how to fix this and get it back to where it can provision again?

I’m still not sure how it got into this state, but in case anyone ever does hit it, I did manage to work through it by simply removing all directories under /sys/fs/cgroup/systemd/lxc.payload/test001

Looks like there has been stricter AppArmor policy enabled. For LXC it’s possible to disable AppArmor using:

lxc.aa_profile = unconfined

Probably defaults in you distribution has been changed. The config is inherited from one defined in lxc.include directive.