For some background I’m running software inside containers, inside each container is a daemon which accepts connections via a UNIX socket located at /var/run/crated.sock. The daemon runs as root, spawning and managing child processes.
This worked fine with the configuration like so without issues:
I’ve noticed however that my new install of LXD is returning an end of stream attempting to connect with the host-side UNIX socket. The proxy log looks like this:
Warning: Failed to connect to target: dial unix /var/run/crated.sock: connect: permission denied
Warning: Failed to prepare new listener instance: dial unix /var/run/crated.sock: connect: permission denied
I thought it might be a permissions error on the container side, but even setting the permissions to 777 didn’t seem to resolve the problem.
srwxrwxrwx 1 root root 0 Oct 14 22:22 crated.sock
Looking around can’t seem to find any information on what to try next here, some posts recently indicate it might be AppArmor tightening some poor security on my part. If I connect with socat inside the container the Unix socket is working fine. Any ideas? Thanks.
There have been some recent changes regarding forkproxy (the process that does the proxies) and the confinement of that process with AppArmor. Can you tell us the LXD versions involved?
I am on mobile now; you can check the release notes for LXD 4.5 (I think) for this. Also, see the LXD proxy documentation if you need to add extra flags for the proxy command.
I’m using LXD 4.6 17738 locally and LXD 4.6 17738 in production, oddly this problem doesn’t happen in production but this might be because I haven’t rebooted the box in a while so the new version hasn’t applied yet. I can’t reboot to verify.
As for the release notes, I can’t really understand what exactly would have caused AppArmor to intefere. I’m trying to proxy a root owned UNIX socket in the container to a UNIX socket on the host which is accessible by the host software user. Somehow it seems to be having problems making a connecting to the container side UNIX socket.
The process that provides the proxy, called fork_proxy was placed under AppArmor restrictions for security reasons, it may be that your usage of it is being blocked by the profile and we need to update it.
@stgraber do you have any idea why connect statement in the proxy is using /var/run/crated.sock yet apparmor is picking it up as /run/crated.sock? Could this be a symlink issue?
Confirmed with @stgraber that now we are protecting the fork_proxy process with AppArmor that we need to ensure the AppArmor profile specifies the fully resolved path to the unix socket, not the symlink.
However whilst we can resolve this on the host side, resolving it inside the container is more difficult as would require entering the container’s mount namespace to resolve the path.
For the time being these paths should be specified in the config as the resolved path rather than to a symlink.