Cntr: A container debugging tool

LXC
1

I wrote a tool to debug LXC containers. You can think of it as a lxc-attach replacement.
You can check it out here: https://github.com/Mic92/cntr
I would love to hear about your feedback.

1 Like
2

Thanks for writing this!

Have you tested with LXD containers?
I could not get it to work with the LXD container name:

$ ./cntr attach guiapps
no suitable container found, got the following errors:
  - ProcessId: not a valid pid: `guiapps`: invalid digit found in string
Exit 1

With process ID I tried with:

$ lxc list -c np
+-----------+------+
|   NAME    | PID  |
+-----------+------+
| guiapps   | 7575 |
+-----------+------+
$ ./cntr attach 7575
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: : EPERM: Operation not permitted', /checkout/src/libcore/result.rs:916:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.
^C
Exit 130
3

I have not tested lxd, because I assumed it works similar to lxc.
Do you run cntr as root? What is the output of lxc-info -n np in your case?
Looking at the error message, it also seems, you have no lxc-info in your path.

4

With sudo, I get

$ sudo ./cntr attach 7575
failed to get lsm profile: failed to get own security label: failed to read /proc/thread-self/attr/exec: Invalid argument (os error 22)

LXD does not have any of the lxc- commands. With LXD, you have just a lxc command, and then you invoke lxc with a subcommand. For example, lxc list will list the available LXD containers.
Therefore, there is no lxc-info and it is not expected to have those commands in a LXD installation.

5

Ok. The failure as root looks more promising. What distribution are you using? Do you have apparmor or selinux running? This means, cntr does not support lxd directly at the moment. But it should be easy to add. Does the lxc command as root provide the same output?

6

I accidentally broke apparmor support. I am working on a fix.

7

I use LXD 2.21 on Ubuntu 16.04 LTS. It has AppArmor.
The lxc command output is identical whether you run as root or non-root user (which is member of the lxd group).

8

@simos can you run test wether master has fixed the problem, when running cntr as root?

$ cargo install --git https://github.com/Mic92/cntr

It should also support older kernels now - I tested trusty, which is 3.13
If it works I can publish a new version.

9

I set up a KVM VM (using multipass, https://github.com/CanonicalLtd/multipass).
I ran the cargo command and installed the latest cntr.

I then set up LXD and created a container with Alpine Linux:

ubuntu@notable-goldcrest:~/.cargo/bin$ lxc launch images:alpine/edge alpine
Creating alpine
Starting alpine
ubuntu@notable-goldcrest:~/.cargo/bin$ lxc list -c np
+--------+------+
|  NAME  | PID  |
+--------+------+
| alpine | 7465 |
+--------+------+

I then ran cntr:

ubuntu@notable-goldcrest:~/.cargo/bin$ sudo ./cntr attach 7465
cannot create container mountpoint /var/lib/cntr: Permission denied (os error 13)
^C

The LXD container names are not recognized,

ubuntu@notable-goldcrest:~/.cargo/bin$ sudo ./cntr attach alpine
no suitable container found, got the following errors:
  - ProcessId: not a valid pid: `alpine`: invalid digit found in string
10

Yes. Attaching by name will not work until I added the lxc command as well to my list. The code for that is almost done. This permission denied looks interesting. When I ran lxd on my laptop I possibly skipped some security profiles as I just compiled it from source and setup the minimum. On ubuntu I have only tested docker + apparmor so far. So what is happening on your system is that the container is not allowed to access the fuse filesystem, that cntr mounts into a nested namespace in the container. At this stage only cgroups and the mount namespace have been applied so there not too many thinks that can prevent a root user from creating directories on a filesystem. I will try to reproduce your setup later this day.

11

At least apparmor support seems to be fixed in your setup. This means I will publish a new version because also other containers will profit from that.