What must a container image provide?

I am expermenting with a container that is very minimal, essentially busybox and one other process. Files cherry-picked from Alpine into an empty rootfs. Not using alpine or any other base images.
Rootfs built by hand.

When I attempt to start a container from this image it fails with the output shown below. However, if I try to start it a second time then it works. And it works thereafter. It’s only the first time it doesn’t work. If I delete the container and recreate from the same image then it fails again, like before, for the first attempt to start and works thereafter.

I suspect I am missing something from the rootfs but I don’t know what. I am using LXD 4.12 on an ArchLinux host running with systemd 248.

lxc testbox 20210427105658.657 ERROR    cgfsng - cgroups/cgfsng.c:__cg_mount_direct:1786 - No such file or directory - Failed to mount "/usr/lib/lxc/rootfs/sys/fs/cgroup" with cgroup filesystem type cgroup2
lxc testbox 20210427105658.660 ERROR    conf - conf.c:lxc_mount_auto_mounts:725 - No such file or directory - Failed to mount "/sys/fs/cgroup"
lxc testbox 20210427105658.661 ERROR    conf - conf.c:lxc_setup:3371 - Failed to setup remaining automatic mounts
lxc testbox 20210427105658.661 ERROR    start - start.c:do_start:1218 - Failed to setup container "testbox"
lxc testbox 20210427105658.662 ERROR    sync - sync.c:__sync_wait:36 - An error occurred in another process (expected sequence number 5)
lxc testbox 20210427105658.664 ERROR    lxccontainer - lxccontainer.c:wait_on_daemonized_start:859 - Received container state "ABORTING" instead of "RUNNING"
lxc testbox 20210427105658.672 ERROR    start - start.c:__lxc_start:1999 - Failed to spawn container "testbox"
lxc testbox 20210427105658.672 WARN     start - start.c:lxc_abort:1012 - No such process - Failed to send SIGKILL via pidfd 18 for process 79723
lxc 20210427105658.140 WARN     commands - commands.c:lxc_cmd_rsp_recv:124 - Connection reset by peer - Failed to receive response for command "get_state"

Here’s a dump of the rootfs image contents:

crw------- root/root       5,1 rootfs/dev/console
crw--w---- root/root       4,1 rootfs/dev/tty1
drwxr-xr-x root/root         0 rootfs/lib/
drwxr-xr-x root/root         0 rootfs/
drwxr-xr-x root/root         0 rootfs/bin/
drwxr-xr-x root/root         0 rootfs/dev/
drwxr-xr-x root/root         0 rootfs/etc/
drwxr-xr-x root/root         0 rootfs/etc/network/
drwxr-xr-x root/root         0 rootfs/etc/network/if-pre-up.d/
drwxr-xr-x root/root         0 rootfs/etc/network/if-up.d/
drwxr-xr-x root/root         0 rootfs/proc/
drwxr-xr-x root/root         0 rootfs/sbin/
drwxr-xr-x root/root         0 rootfs/usr/
drwxr-xr-x root/root         0 rootfs/usr/share/
drwxr-xr-x root/root         0 rootfs/usr/share/udhcpc/
drwxr-xr-x root/root         0 rootfs/var/
drwxr-xr-x root/root         0 rootfs/var/lib/
drwxr-xr-x root/root         0 rootfs/var/lib/misc/
drwxr-xr-x root/root         0 rootfs/var/run/
drwxr-xr-x root/root         0 rootfs/var/tmp/
lrwxrwxrwx root/root         0 rootfs/lib/libc.musl-x86_64.so.1 -> ld-musl-x86_64.so.1
lrwxrwxrwx root/root         0 rootfs/bin/run-parts -> /bin/busybox
lrwxrwxrwx root/root         0 rootfs/bin/sh -> /bin/busybox
lrwxrwxrwx root/root         0 rootfs/sbin/ifup -> /bin/busybox
lrwxrwxrwx root/root         0 rootfs/sbin/init -> /bin/busybox
lrwxrwxrwx root/root         0 rootfs/sbin/ip -> /bin/busybox
lrwxrwxrwx root/root         0 rootfs/sbin/udhcpc -> /bin/busybox
-rw-r--r-- root/root       136 metadata.yaml
-rw-r--r-- root/root        27 rootfs/etc/inittab
-rw-r--r-- root/root        36 rootfs/etc/group
-rw-r--r-- root/root        68 rootfs/etc/passwd
-rwxr-xr-x root/root      3121 rootfs/usr/share/udhcpc/default.script
-rwxr-xr-x root/root    596528 rootfs/lib/ld-musl-x86_64.so.1
-rwxr-xr-x root/root    841288 rootfs/bin/busybox

Any pointers apprecaited, thanks!

My bet would be on you missing rootfs/sys in there.

Looks like your horse came in :wink:
Thanks for the pointer.

@starfry Would you be willing to share your image building process?