Incus 6.0 Container Privilege issue

[Scott_T]

@stgraber I have discovered a major issue in the last 48 hours in 6.0. I am upgraded to the latest iteration. I had to rebuild an existing server five times to pin it down.

Apparently certain containers that have not had security.privileged=true have thrown this error.

error mounting "mqueue" to rootfs at "/dev/mqueue": ... permission denied

These are containers with security.nesting=true because these containers have docker applications nest in them.

The really insidious problem occurred after rebooting the host which is the Ubuntu 24.04.3 LTS server. After a reboot:

mount | grep devpts

reveals devpts mounted with ptxmode=000

devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)

Worse yet:

ls -l /dev/ptmx

reveals that the symbolic link is GONE!

lrwxrwxrwx 1 root root 13 Aug 15 20:10 /dev/ptmx 

I have started from scratch rebuilding Ubuntu and reinstalling Incus 5 times now and each time this happens. It happens right after incus starts.

I don’t have to tell you that a corrupted PTS results in no ability to either ssh or sudo.

I put a band-aid on the problem by creating a service that remounts devpts properly and restores the symbolic link.

The issue is that apparently after the upgrade to one of the interim Incus 6.0 releases in the last three days, the mqueue issue starting popping up and that seems to be the thing that initiated this issue.

[stgraber (Stéphane Graber)]

Docker issues after 6.15 Upgrade

Can anyone provide assistance with the following issue: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "mqueue" to rootfs at "/dev/mqueue": change mount propagation through procfd: resolving path inside rootfs failed: lstat /var/lib/docker/overlay2/8c209f91d6403d5cb3b079487722a4087de4739c743dcaaf85c82c7a73665022/merged//dev/m…

But you’re mentioning 6.0 so that’s a bit confusing as the logic to use abi4.0 never made it to the 6.0 release.

[Scott_T]

My bad. It is the latest Incus 6.15.

[stgraber (Stéphane Graber)]

Possibly not quite latest enough?
The fix for this issue was pushed to the package on Wednesday.

[Scott_T]

I sort of understand that. However, I reinstalled Ubuntu 24.04 server from scratch this morning. I installed Incus from scratch this morning. The problem persists till now. If I didn’t have the band-aid and I rebooted it would still persist. I don’t understand this because I have eight other Incus servers and I update them all at the same time. They all updated fine. Only my most critical one saw this problem. All the others are fine.

[stgraber (Stéphane Graber)]

What’s the package version on the broken server?

20250814XXXX is current and at least here doesn’t result in abi4.0 being in the generate profile.

[Scott_T]

Here ya go…

incus:
  Installed: 6.0.0-1ubuntu0.3+esm2
  Candidate: 6.0.0-1ubuntu0.3+esm2
  Version table:
     1:6.15-ubuntu24.04-202508140159 500
        500 https://pkgs.zabbly.com/incus/stable noble/main amd64 Packages
     1:6.15-ubuntu24.04-202508121948 500
        500 https://pkgs.zabbly.com/incus/stable noble/main amd64 Packages
     1:6.15-ubuntu24.04-202508091853 500
        500 https://pkgs.zabbly.com/incus/stable noble/main amd64 Packages
 *** 6.0.0-1ubuntu0.3+esm2 510
        510 https://esm.ubuntu.com/apps/ubuntu noble-apps-security/main amd64 Packages
        100 /var/lib/dpkg/status
     6.0.0-1ubuntu0.3 500
        500 http://us.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages
        500 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Packages
     6.0.0-1 500
        500 http://us.archive.ubuntu.com/ubuntu noble/universe amd64 Packages

[stgraber (Stéphane Graber)]

Ah, you’re hitting the mess caused by Canonical always prioritizing their ESM repos over everything else on the system, so you’re not at all using the Zabbly packages.

[stgraber (Stéphane Graber)]

Zabbly Incus will not install on Ubuntu Pro subscribed hosts

… and instead only the Ubuntu version (LTS?) will install. It took me a little while to troubleshoot so I write this here to save time for anyone else struggling with this. This is because the ESM respositories are higher priority. To resolve this you can edit /etc/apt/preferences.d/ubuntu-pro-esm-apps and set the priority to less than 500 and then run sudo apt update and sudo apt upgrade again. This should upgrade the installed Ubuntu version with the Zabbly version plus a bunch of new depende…

Install Stable on 24.04 with Ubuntu Pro enabled - #2 by Lox has an example of the pinning to fix it

[Scott_T]

@stgraber very astute. I am now preferring the Zabbly packages as intended. Do you think that with this I will be able to remove my band-aid fix and be sure that devpts will not be messed with?

[stgraber (Stéphane Graber)]

Going from a release that’s well over a year old and onto a release that came out in the last two weeks sure should help with some issues

it will also make it a lot easier to actually debug/reproduce any issue you may hit.

[Scott_T]

@stgraber The thing is, the apt upgrades always called out 6.15 when I did sudo nala upgrade. However, apparently 6.0 was the one with priority. Both were on the system. Only 6.0 was being used. Something I need to call out on my channel. I am a bit paranoid. I would hate to remove my code that remounts devpts and restores the symbolic link. If it still persists in messing up the system, I need to go hook up a console again to get back in since sudo and ssh are rendered DEAD.