[Solved] Docker disk quota exceeded

Hello,

I have an Ubuntu Jammy container running with Archlinux as the host. I have 64GB of ram but docker is refusing to start any new containers. Before migrating from LXD to Incus i remember setting something to limit memory usage, however i have looked around but cant find anything obvious.

$ docker compose up -d
[+] Running 0/1
 ⠹ Container unifi-network-application-mongodb  Starting                                                                                                                                                     0.2s 
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: unable to join session keyring: unable to create session key: disk quota exceeded: unknown

Free inside container shows the full 64GB

$ free -m
               total        used        free      shared  buff/cache   available
Mem:           64109        4154       50141         606        9813       59349
Swap:          32767           0       32767

# ulimit -a
real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) unlimited
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 0
file size                   (blocks, -f) unlimited
pending signals                     (-i) 256361
max locked memory           (kbytes, -l) 8192
max memory size             (kbytes, -m) unlimited
open files                          (-n) 1048576
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) 8192
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) unlimited
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited


There is no limit in the incus instance config,

$ incus config show docker1|grep -i limit

There is enough space in /var/lib/docker

$ df -h /var/lib/docker
Filesystem      Size  Used Avail Use% Mounted on
/dev/zd48       118G  8.9G  103G   8% /var/lib/docker

How do i find out why i cant run a docker container?


I setup a second new ubuntu (noble) instance and setup docker in it. It also gave the same error,

unable to start container process: error during container init: unable to join session keyring: unable to create session key: disk quota exceeded

It seems that incus is limiting memory somehow.

Solved by running this on archlinux host,

echo 200000 | sudo tee /proc/sys/kernel/keys/maxkeys
2 Likes

I am curious, does a smaller value also work? The docs recommend 2000, but I bet it is not taking the docker use case into account.

1 Like