LXD containers aren't showing up in sysdig

Joe_Mattie · March 26, 2019, 12:16am

I’m running LXD/C from snap, and for some reason none of my lxd containers show up in sysdig. My docker containers show up fine, and I can use ctop without a problem. Was wondering if anyone else has encountered this?

lxd version 3.11
sysdig version 0.24.2

I’m having the same issue as https://github.com/draios/sysdig/issues/1289 which hasn’t received any responses yet.

Joe_Mattie · March 26, 2019, 1:04am

Replying to my own thread:

I got this working by downgrading to lxd 3.0.3, still from snap

Joe_Mattie · March 26, 2019, 2:17am

It looks like the issue is with cgroup.find("/lxc/"); returning false on lxd 3.11 but true on 3.0.3

simos · March 26, 2019, 9:51am

Can you check in /sys/fs/cgroup/cpuset if is says lxc or something else?
It looks like in LXD 3.11 it’s split into lxc.monitor and lxc.payload.
If so, report back to github so that it gets fixed.

Joe_Mattie · March 26, 2019, 3:58pm

Yup, can confirm that LXD 3.11 only creates lxc.monitor and lxc.payload under /sys/fs/cgroup/cpuset

stgraber · March 27, 2019, 4:48pm

LXD itself didn’t make any chance here, it’s the switch to liblxc 3.1 which causes that change in paths, so any container that’s running through liblxc 3.1 or higher will have this problem.

simos · March 27, 2019, 5:30pm

Here is liblxc (master) on the cgroup naming.

github.com

lxc/lxc/blob/master/src/lxc/cgroups/cgroup.h#L31


* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/


#ifndef __LXC_CGROUP_H
#define __LXC_CGROUP_H


#include <stdbool.h>
#include <stddef.h>
#include <sys/types.h>


#define PAYLOAD_CGROUP "lxc.payload"
#define MONITOR_CGROUP "lxc.monitor"
#define PIVOT_CGROUP "lxc.pivot"


struct lxc_handler;
struct lxc_conf;
struct lxc_list;


typedef enum {
       CGROUP_LAYOUT_UNKNOWN = -1,
       CGROUP_LAYOUT_LEGACY  =  0,

The previous version 3.0 is at the stable-3.0 branch and does not have such defines.
Instead, the name is coded in the code, which I think is this part,

github.com

lxc/lxc/blob/stable-3.0/src/lxc/cgroups/cgfsng.c#L1924


	free(full_path);


	len = strlen(base_path) + STRLITERALLEN("/lxc-1000") +
	      STRLITERALLEN("/cgroup-procs");
	full_path = must_realloc(NULL, len + 1);
	do {
		if (idx)
			ret = snprintf(full_path, len + 1, "%s/lxc-%d",
				       base_path, idx);
		else
			ret = snprintf(full_path, len + 1, "%s/lxc", base_path);
		if (ret < 0 || (size_t)ret >= len + 1)
			goto on_error;


		ret = mkdir_p(full_path, 0755);
		if (ret < 0 && errno != EEXIST)
			goto on_error;


		(void)strlcat(full_path, "/cgroup.procs", len + 1);
		ret = lxc_write_to_file(full_path, pidstr, len, false, 0666);
		if (ret == 0)

While writing this reply, I noticed the following. Thanks, @Joe_Mattie.